Lawrence J. Schweinhart and David P. Weikart indicate in "Why Curriculum Matters in Early Childhood Education" (March 1998) that children who attended a Direct Instruction preschool program had a significantly higher rate of felony arrests as adults than children who went through the High/Scope or the Nursery School preschool curriculums. Schweinhart and Weikart assert that they based their conclusion on rigorous research.
This conclusion is contrary to extensive data on Direct Instruction, and it is questionable because of the indirect relationship of felony arrests at age 23 to anything that occurred in preschool. The conclusion is also suspect because it is based on woefully small groups of subjects and because the study is laced with inconsistencies, mathematically impossible numbers, and logic that is inadmissible in rigorous scientific endeavors.
Mathematical Inconsistencies
The High/Scope preschool comparison attempts to identify statistically significant differences in the performance of the curriculum groups and to infer from these data that the preschool curriculums caused the differences (Schweinhart & Weikart, 1997a, 1997b).
Each of the three preschool groups had only 22 or 23 students. The authors claim that 52 of the original 68 subjects were interviewed at age 23, but they repeatedly contradict this assertion and show that they interviewed only 49 subjects. For instance, they report that "19 subjects were not found." If 19 were not found and the original group numbered 68, no more than 49 subjects could have been interviewed. Also, the samples from the group were not well matched. Only 36 percent of the High/Scope subjects were male, compared with 47 percent for the Direct Instruction group.
Variations in Experiences
Another reason to question whether the preschool experience caused later problems is that not all subjects experienced the same duration of preschool. All members of the High/Scope group went through two years of preschool, as 3-year-olds and as 4-year-olds, but eight Direct Instruction children and eight Nursery School children attended only one year of preschool, as 4-year-olds.
The groups also had different experiences after preschool. Eighty-four percent of those who participated in Direct Instruction attended Ypsilanti High School, compared with 69 percent of the High/Scope children and 39 percent of the Nursery School children. The percentages who lived in Ypsilanti at age 23 were significantly different: 84 percent for Direct Instruction, 64 percent for High/Scope, and 44 percent for Nursery School.
The great difference in pre–high school mobility means that for many years, subjects experienced different environmental climates and influences. Given that where children grow up has a strong influence on behavior, and given that this influence is more recent, more pervasive, and of a longer duration than the preschool experience, these differences are more plausible causes of differences in felony arrests than the remote differences in preschool curriculums. What adjustments did the High/Scope study make to account for the differences in pre–high school mobility? Precisely none.
Errors and Discrepancies
Further, Schweinhart and Weikart's method of reporting tabular data is replete with errors and impossible numbers. For example, in the two publications where they report their research, at least 10 inconsistencies on data for felony arrests occur. The most obvious inconsistency is the number of subjects: 62 versus 68. Both accounts present the same percentages for felony-arrest data of the three curriculum groups. It's mathematically impossible for the group sizes to be different and for all the percentages to be the same from one account to the other. At least six other inconsistent values occur in the average number of adult arrests.
A far greater problem involves the apparent impossibility of reporting on either 62 or 68 subjects when the researchers located only 49. The authors' justification starts with the premise that "missing arrest records signify the absence of arrests." Applying this premise, the authors searched arrest records. If a subject's name did not come up, the subject was counted in the total number for the study and was classified as having no arrests. However, the authors searched only in Michigan. If a subject lived in another state, the search would reveal nothing. Despite this obvious limitation, Schweinhart and Weikart (1997b) conclude that "the 19 study participants who were not interviewed were retained in the arrest records sample" (p. 127). In other words, if the authors didn't know where somebody lived, or even whether the person was alive, they concluded not only that the person was a Michigan resident, but also that the total number of felony arrests for that person occurred in Michigan. The authors claim that 8 of the 19 missing subjects, or 49 percent, had adult arrest records in Michigan. However, the authors account for only 5 of these subjects, which is only 26 percent, not 49 percent.
A possible motive for Schweinhart and Weikart to include the 19 subjects was to increase the total number of subjects in the study so that statistically significant differences are manufactured. The true number for the group is possibly 43 or 48, but certainly not 62 or 68. With the more realistic number, there are no statistically significant differences in felony arrests between the Direct Instruction group and the others, even with the authors' statistical procedures.
Questions About Felony Records
Another illuminating fact is that the number of reported felony arrests for the Direct Instruction group was very similar to that of the High/Scope population in the Perry Preschool project. The average for the Direct Instruction group was 0.9, which is very close to the 0.7 estimated for the High/Scope subjects.
But even if we permit the inclusion of all subjects and accept the figures presented for felony arrests, we still face the problem of conviction data. The study shows that whether the number is 68 or 62, there are no statistically significant differences in convictions, only in arrests. So, unless arrest data is a more reliable indicator of guilt than conviction data, the study does not establish the suggested outcome that Direct Instruction causes crime.
Effect of the Duration of the Preschool Experience
Another contradiction in this study has to do with how Schweinhart and Weikart interpret the relationship between duration of preschool experience and later arrests. If a program effect existed, it would seem reasonable that the longer the duration of the preschool experience, the greater this effect would be. If there is no difference between the children who attended the preschool for one year and those who attended for two years, the second year is inert with respect to producing arrests. Therefore, in absence of strong data to the contrary, the first year may also have no impact on arrest records.
Schweinhart and Weikart (1997b) confirm that in the two year subsample, the mean number of felony arrests for each of the three curriculum groups was almost exactly the same as it was in the complete arrest sample. (P. 134)
Two conclusions are possible: that the preschool experience is perfectly uncorrelated with arrests or that a period shorter than one year might have the same effect as the two-year programs. Possibly a two-week exposure would produce the same difference. Given the unlikelihood of the latter conclusion, the former is strongly suggested.
Finally, the authors seem to go out of their way to implicate a particular program—Reading Mastery and its scripted presentations—as the cause of the supposed differences in felony rates. In other places, Weikart has called this program the thalidomide of reading programs, even though it probably has more evidence of effectiveness, particularly with poor children, than all other reading programs combined (Engelmann & Carnine, 1982). Two reasons suggest that the authors' attack on Reading Mastery is less than rigorous or objective. First, the program was not published until 1969, which means that a prepublication version was used for all but possibly one-third of the children in the Direct Instruction treatment. Second, the reading program was not used with any 3-year-olds and was not the primary program with the 4-year-olds. The DISTAR language program was the primary program used with both 3- and 4-year-olds. In fact, the authors' Direct Instruction curriculum was originally called the Language Training Curriculum. It used a prepublication form of the language program for two-thirds of the participants.
Schweinhart and Weikart's ostensibly "rigorous" research has many serious problems. They generalize from a hopelessly small number of subjects (possibly only 11 of them in the High/Scope model). The groups were not well matched in duration of preschool experience, in gender balance, in pre–high school mobility, and in neighborhoods where they grew up. The authors' definition of rigor uses statistical techniques that require only a .5 level of probability and presents descriptions that sometimes (and conveniently) do not match numbers, numbers that are not always consistent from one account to another, and arguments that liberally break rules of logic and common sense. *
