Urban legend: a good/captivating titillating/engrossing/incredible/worrying story that has had a wide audience, is circulated spontaneously, has been told in several forms, and which many have chosen to believe despite the lack of actual evidence to substantiate the story.—Urban Legends Research Centre
I want to call attention to a new urban legend, one that ranks right up there with the legend of alligators in the sewers of New York City. It is the Skyrocket Legend: As a result of dropping bilingual education, test scores in California “skyrocketed.”
This legend has had serious consequences. Many people have interpreted the Skyrocket Legend as demonstrating that immersion programs are superior to bilingual education as a way to educate English language learners, and some have pointed to this legend to advocate eliminating bilingual education in other states. But dropping bilingual education did not contribute to the improvement in California's test scores. There are more reasonable explanations for the increase.
Proposition 227, the mandate to eliminate bilingual classes in California public schools, took force in 1998, just as the new SAT-9 was introduced. Research has shown that after new tests are introduced, test scores tend to rise (Linn, Graue, & Sanders, 1990). This test inflation was especially strong in California because the same test had been given for several years in a row, and because state policy punished schools severely for lower scores and rewarded them generously for higher scores—at least while money remained in the budget. This pressure resulted in districts using extraordinary means to raise test scores, some of which have nothing to do with increased competence. For example, Asimov (2000) found that schools in California may have engaged in selective testing—excluding low-scoring students from taking the test. She reported that in many cases in which SAT-9 scores increased from year to year, the number of students tested decreased. These “questionable pairings” appeared in 22 San Francisco-area school districts.
Schools can also raise test scores without increasing competence by giving students extensive training in certain test-taking skills. For example, if the test imposes no penalty for guessing, then schools can raise scores by simply encouraging students to guess when they don't know the answer.
Using these bogus means to achieve higher test scores is like claiming to raise the temperature of a room by lighting a match under the thermometer.
Test Scores and 227
The Parrish Study
Parrish and colleagues (2002) compared test scores for California English language learners in 682 schools that kept bilingual education with scores in 1,184 schools that dropped bilingual education. This analysis included students currently considered English language learners as well as those who were formerly English language learners but who were subsequently reclassified as English-proficient.
The researchers provided two analyses. In a “quasi-longitudinal” study, they analyzed SAT-9 results for English language learners who were in grade 2 in 1998 and in grade 5 in 2001. The study design was considered only quasi-longitudinal because student transience prevented the two groups from being identical. The analysis found practically no difference in gains between students in schools that maintained bilingual education and students in schools that dropped bilingual education. In schools that kept bilingual education, the average score rose from 537 to 621, a gain of 84 points. In schools that dropped bilingual education, scores rose from 540 to 623, a gain of 83 points.
Parrish and colleagues also conducted a cross-sectional analysis. They found that in 1998, 5th graders in schools that planned to keep bilingual education programs scored 613 and 5th graders in schools that were about to drop bilingual education scored 614. In 2001, the 5th grade scores were 621 for the schools that had kept bilingual education and 623 for the schools that had dropped it.
Not all English immersion programs in the study practiced pure English immersion, however. Parrish and colleagues reported that among 75 districts that said they used “structured English immersion,” 68 percent reported that teachers used the students' first language occasionally or frequently for academic content instruction; 88 percent used it occasionally or frequently for preview and review; and 48 percent occasionally or frequently used instructional aides who spoke students' first language. Thus, this study actually compared bilingual programs with immersion programs that used aspects of bilingual education.
The Bali Study
Rossell (2002) rejects Parrish and colleagues' results because more students were tested in 2001 than in 1998. She cites with approval, however, Bali (2001), who provides data from one district—Pasadena—and controlled for different student testing rates. Bali compares the progress of two groups: students who had been in bilingual education but who were abruptly placed in all-English classes as a result of Proposition 227, and students who had been in all-English classes all along. Bali reported that students who had been in bilingual education gained 4.15 points in reading the year after leaving bilingual education, whereas those who had been in English immersion all along gained only 1.8 points. The study found no difference in math. Bali interpreted these data to indicate that dropping bilingual education did no harm, and in fact actually helped students.
But we could easily argue that the superior gains of students who had been in bilingual education demonstrate the superiority of bilingual education. In their bilingual education classes, students learned subject matter through their first language, which made the English instruction they subsequently received in those subjects more comprehensible to them. Bilingual education also developed students' literacy in their first language, which facilitated subsequent literacy development in English.
Thus, research has provided no evidence that dropping bilingual education resulted in a significant improvement in English development. California, it seems, dismantled a successful program that allowed for equivalent English language development with less time actually devoted to English. California gave up the advantages of bilingual education and bilingualism and received nothing in return.
Why Bilingual Education Works
High-quality bilingual programs introduce English right away and teach subject matter in English as soon as it can be made comprehensible, but these programs also develop literacy in students' first language and teach subject matter in that language in the early stages. Developing literacy in the first language provides a shortcut to English literacy. Children learn to read much more easily in a language they understand, and once they can read in the primary language, they can rapidly transfer these reading skills into English. Teaching subject matter in the first language stimulates intellectual development and provides students with valuable knowledge that will help them understand instruction when it is presented in English.
What Controlled Studies Say
A valid way to determine the effect of bilingual education is to perform controlled studies, which compare programs that differ only in the extent to which students are taught in their first language.
Such scientifically valid, controlled studies consistently show that students in properly organized bilingual programs acquire at least as much English as comparison students in all-English programs do. In fact, they usually acquire more.
Willig (1985), Greene (1997), and Slavin and Cheung (2004) conducted reviews of the research on bilingual education using statistical tools far more precise and sensitive than those used in other reviews. They all concluded that using the first language to teach English language learners has beneficial effects on English language development.
A research team headed by Oller and Eilers (2002) recently conducted a study on the effectiveness of bilingual education. Fifth grade students in a bilingual program (in which 60 percent of instruction was presented in English and 40 percent was presented in Spanish) performed as well on tests of English literacy as did comparison students in an all-English program (with an optional 10 percent of instruction in Spanish), and they performed far better on tests of Spanish literacy.
Studies from other countries yield results that are consistent with those from the United States. Students in well-organized bilingual programs acquire as much or more of the second language as students in immersion programs do. Studies confirming this have been conducted with Turkish- and Urdu-speaking students in Norway, Punjabi-speaking students in England, Turkish- and Arabic-speaking students in the Netherlands, Finnish-speaking students in Sweden, Gapapuyngu-speaking students in Australia, and Tzeltal- and Tzotzil-speaking students in Mexico (Krashen, 1999a).
Rossell and Baker (1996) also reviewed the research on bilingual education, but they concluded that bilingual programs are not as effective as all-English immersion programs. Their review, however, inappropriately excluded a number of valid studies and included studies that were not valid comparisons—for example, comparisons of different types of Canadian immersion programs. In addition, some of the programs that the researchers labeled “immersion” were, in fact, bilingual programs containing significant amounts of instruction in the primary language (Krashen, 1996, 1999b).
Even so, Rossell and Baker conclude that “additional, methodologically sound research needs to be conducted in order for the courts and policymakers to make intelligent decisions” (p. 39) and that “we are struck by how small the differences are . . . between programs with very different amounts of English instruction” (p. 43). The Rossell and Baker review is by far the most negative review of bilingual education published; in fact, it is the only review that I know of that claims that all-English alternatives are better. Yet even this review concludes that differences are not huge and that more research is necessary to ensure sound decisions.
Clearly, the published research does not support claims that dropping bilingual education causes scores to “skyrocket”—nor does this research support any movement to dismantle bilingual education.
The Legend Is False
There is no question that test scores went up in California, but dropping bilingual education had nothing to do with the increase. Test score increases in California appear to be a result of the usual test score inflation that occurs when new tests are introduced. In California, test score inflation has been particularly strong because of intense pressure to raise scores.
Test scores went up for everyone in California. There were no significant differences in gains between districts that kept bilingual education and those that dropped it. In fact, it is not even clear that districts that dropped bilingual education really dropped it; use of the first language remained substantial in many “structured immersion” programs.
Missing from nearly all discussions of the effectiveness of bilingual education is the fact that controlled studies consistently show that bilingual education works. The Skyrocket Legend is false.
