Once upon a time, a king had a great merchant fleet that traded with countries throughout the world. Assessments revealed that the fleet had a high rate of accidents, particularly ships running aground or hitting rocks and sinking. Thus, the king decided to put policies in place to solve this problem.
First, the king proclaimed "sailing standards" that specified the routes that ships should take. This helped, but there were still far too many accidents. Then he announced an accountability plan in which captains would be thrown overboard if their ships ran aground. This didn't work either, so the king required the whole crew to be thrown overboard if their ship had an accident. Finally, he chartered other organizations to sail ships for him, but their ships ran aground, too.
While this was going on, researchers at Kingdom State University approached the king. "Your Majesty," they said, "we have been doing research on navigation methods. Your sailing standards are a good start, but your crews need to learn better ways to meet the standards. We think we can help, but we will have to work one ship at a time to teach the captains and crews effective navigation methods."
The king was skeptical. "My fleet is large and its problems are great. How can you make a meaningful difference one ship at a time?"
The researchers proposed to start with one crew, then two, then four, then eight, and to keep doubling until they taught the entire fleet. The king agreed, and the researchers began to teach the captains and crews the methods their research had identified: how to use the sun and stars for navigation, read charts, determine ocean depth, work more effectively as a team, and refloat their ships if they did run aground. The crews that went through the program stopped having so many accidents, but at first the rate of accidents for the fleet as a whole didn't change much because the number of trained crews was small. As the training program grew, however, the rate of accidents diminished sharply for the entire fleet.
The king, pleased with this, summoned the researchers. "Why did my systemic reforms not work, when your ship-by-ship plan worked so well?" he asked.
"Sire," the lead researcher said, "your reforms were intended to motivate the crews. The crews were already motivated; they didn't want to run aground or sink. Your captains and crews have always been hardworking and committed to success. They needed better ways to sail, not just consequences for failure!"
Since A Nation at Risk was published (National Commission on Excellence in Education, 1983), education in the United States has been in a state of reform. Commission after commission, politician after politician, newspaper after newspaper have declared education to be in crisis, in need of drastic change to keep up with our economic competitors and the needs of our own economy.
The dominant strategy for school reform has been systemic reform: changes in standards, assessments, accountability, governance (charters and vouchers), and funding plans (Smith & O'Day, 1991). Some of these strategies may be beneficial—in particular, the idea that educators should have well-thought-out, well-justified standards describing what students should know and be able to do. Systemic reforms are policies that Washington, D.C., state capitals, or district offices can prescribe to large numbers of schools and classrooms, but their influence is indirect at best. A state standard, a school report card, or even the threat of reconstitution will have little direct influence on what Ms. Jones teaches her 3rd grade class, how well she teaches it, or what her students learn.
Principals and teachers have always been motivated to ensure student success; they need better methods and materials, not just consequences for failure. Whatever impact 20 years of systemic reform may have had or may have in the future, there is a limit to what can be achieved in education reform unless the reforms can improve the methods and materials every teacher uses with every student, in every subject, every day.
Comprehensive School Reform
Alongside the systemic reform movement of the 1980s and 1990s, a complementary but quite different approach to reform—referred to as comprehensive reform, whole-school reform, or design-based assistance—has grown. Comprehensive reform shares the goal of systemic reform: improved student achievement on a broad scale. But it starts from the opposite end, considering the school as the unit of reform and "scaling up" effective schools until large numbers of schools are using more effective methods in line with state and national standards. Usually, developers of comprehensive programs require clear buy-in from school staffs, such as 80 percent of teachers voting for a program in a secret ballot. Comprehensive reform models vary substantially, but they ultimately intend to affect every aspect of school functioning: curriculum, instruction, assessment, school organization, support for children having difficulties, parent involvement, and sustained professional development. Comprehensive models invariably provide extensive professional development to the entire school staff.
Nonprofit organizations have created and disseminated almost all of the comprehensive reform models. The earliest examples of comprehensive reforms were process models, such as Theodore Sizer's Coalition of Essential Schools (1992), James Comer's School Development Program (Comer, Ben-Avie, Haynes, & Joyner, 1999), and Henry Levin's Accelerated Schools model (Hopfenberg & Levin, 1993). These and other process models specify general principles of reform, but let educators decide how to put these principles into practice, with the help of consultants or "critical friends."
In contrast to the process models, curriculum-based programs such as our Success for All program (Slavin & Madden, 1999, 2001; Slavin, Madden, Dolan, & Wasik, 1996) introduce schoolwide changes along with specific curriculum materials, teachers' manuals, and training procedures. Another approach of this kind is Direct Instruction (Adams & Englemann, 1996), which built a schoolwide reform model around structured reading and mathematics programs.
In 1991, the New American Schools Development Corporation—now New American Schools (NAS), a nonprofit organization established with substantial corporate and foundation funding—funded nine comprehensive models, helping them both establish their methods and materials and build training corps and self-sustaining organizations.
The development, evaluation, and growth of a variety of comprehensive reform models have substantially changed the landscape of school reform. Previously, policymakers could ignore the comprehensive reform movement because the number of schools using these reforms was too small to make a difference. That is no longer true. No one has exact numbers, but as many as 5,000 schools are implementing comprehensive reform models, with more than 1,000 new schools added each year. The efforts of these schools suggest a profound and unprecedented development in the long history of school reform. Even more profound changes in policy and practice are currently taking place, however, and still more are on the horizon.
A Research Base
One reason that comprehensive reform models have such enormous potential for broad-scale school reform is that some of these models show substantial evidence of effectiveness in improving student achievement on the assessments that almost all states now use to evaluate students and schools. Reviews of this research have identified comprehensive programs with solid evidence of effectiveness in comparison to matched control groups (Herman, 1999; Slavin & Fashola, 1998; Traub, 1999). But all comprehensive models feel pressure to produce such evaluations, especially third-party evaluations, and within a few years a substantial body of evaluation evidence supporting a wide range of models for elementary and secondary schools will likely exist. In support of this endeavor, New American Schools is leading an effort to create standards of effectiveness and comprehensiveness for providers of these designs.
The research base for comprehensive models is an essential element of reform. Unlike systemic reforms in standards, accountability, and governance—which are often difficult to relate to improved classroom practices and student outcomes (Cohen & Hill, 1997)—school-by-school reforms have a direct and immediate impact on practices. If comprehensive reforms are to affect policy, we must prove that these practices, when implemented as intended, improve student outcomes.
Comprehensive School Reform Demonstration
The most important policy development in support of comprehensive reform is the Comprehensive School Reform Demonstration (CSRD) program. Introduced in the United States Congress in 1997, the program initially provided an annual appropriation of $150 million, mostly to be used in grants of at least $50,000 per year for up to three years, to help schools adopt "proven, comprehensive reform designs." States and territories received an amount of money, more or less proportional to their Title I funding, to make grants to schools. Each state had somewhat different policies and timelines, but from 1998 to 2000, more than 1,800 schools received Comprehensive School Reform Demonstration funds. In addition, Congress allocated funds to the U.S. Department of Education's regional laboratories to help with the awareness and adoption process. Somewhat later, existing reform models received money to help build their capacities to serve schools and to develop new programs for middle and high schools, which had few well-validated models.
In the first round of Comprehensive School Reform Demonstration funding, about half of the grants went to widely known reform designs and half went to locally developed or less well known designs. About 17 percent of the grants supported Success for All/Roots & Wings, 7 percent Accelerated Schools, and 11 percent (collectively) the eight New American Schools designs other than Success for All/Roots & Wings.
The impact of the Comprehensive School Reform Demonstration program has extended far beyond the schools that have received funding and the more than one million students they serve. For the first time ever, serious federal funding depends on demonstrated evidence of effectiveness of reform models that can affect very large numbers of students. In the 1980s and early 1990s, the now-defunct National Diffusion Network provided money to help schools adopt research-based programs, but the amounts available were a fraction of those in the Comprehensive School Reform Demonstration.
The Comprehensive School Reform Demonstration changes school reform and research and development at many levels. At the school level, it provides funding to help schools through the early years of program adoption, when the costs of all comprehensive programs are at their highest. Further, Comprehensive School Reform Demonstration funding provides an incentive to school staffs to look into a variety of reform models. The close connection of Title I and Comprehensive School Reform Demonstration (the great majority of Comprehensive School Reform Demonstration schools are high-poverty Title I schools) communicates to the staff of Title I schools that comprehensive reform is a legal, even encouraged, use of Title I funds, especially in schoolwide projects in which Title I funds can be used to benefit all children. As a result, even schools that do not receive Comprehensive School Reform Demonstration grants are more likely to learn about and adopt comprehensive models.
At the policy level, the Comprehensive School Reform Demonstration—and the broader comprehensive reform movement—are making an equally profound impact. Well-researched, readily replicable whole-school reforms give state and district policymakers a tool to help improve teaching and learning in underperforming schools. In many districts, low-achieving schools have been given either incentives or support to adopt proven, comprehensive reform models or have been required to select from among several designated programs. Such actions give low-performing schools a means to improve dramatically, without draconian solutions such as reconstitution.
The Comprehensive School Reform Demonstration may have its most important effect on educational research and development. It creates a market for programs based on strong research, so that researchers and developers can have confidence that if they develop and successfully evaluate a comprehensive program, schools will actually use it. Further, the Comprehensive School Reform Demonstration program has enormously increased interest in Congress in supporting the entire research and development process. Federal policymakers can clearly see a link between investments in research and development and the widespread adoption of proven programs. The Office of Educational Research and Improvement (OERI) has funded new federal research and development initiatives tied in varying degrees to the Comprehensive School Reform Demonstration program. These initiatives have begun to fund practical, solution-focused research and development in such areas as early literacy and numeracy, secondary reform, and bilingual education. Quite in contrast to the usual reluctance to fund educational research, new funds have come forward to add substantially to the minimal funding that the Office of Educational Research and Improvement has traditionally spent on research, perhaps doubling in a short time the total amount spent on research in elementary and secondary education.
In practice, Comprehensive School Reform Demonstration is far from perfect. In the first round of funding, schools adopted more than 280 different models. Many of the models had little or no evidence of effectiveness, and many were not remotely comprehensive. States and districts have much to learn about helping schools make informed and appropriate decisions among models and then ensuring effective implementation of the chosen models. No one could expect such a massive reform effort to work flawlessly the first time, however. Recognizing some of the difficulties, Congress and the Department of Education have introduced programs to build capacity and quality of designs and to create additional designs, and they are working to improve dissemination of information, technical assistance, and guidelines to state agencies. Congress increased funding for the Comprehensive School Reform Demonstration program to $220 million in 2000, and Senator Richard Lugar (R-Indiana) has introduced a proposal to increase funding to $500 million—still only one-sixteenth of Title I funding, but enough to transform U.S. education.
New Jersey and Beyond
Beyond the Comprehensive School Reform Demonstration, the most striking reform based on the existence of comprehensive reform models is the Abbott v. Burke funding-equity case in New Jersey. There, in a resolution of a 28-year case, the New Jersey Supreme Court required that schools in the 28 highest-poverty school districts select from among a set of proven, comprehensive models. For elementary schools, Success for All was named as the presumptive design, meaning that schools were urged to adopt it or explain why they did not. In practice, 34 percent of elementary schools that have made a choice so far chose Success for All, with the others selecting from among a half-dozen alternatives.
The Abbott reform links substantial additional funding for high-poverty schools to adoption of effective programs. It cuts through the murky question, central to all funding-equity litigation, of whether money makes a difference in educational outcomes, and moves the discussion to the more promising question of whether money can make a difference if invested in specific programs and practices known to be effective. Again, the potential impact of this precedent is profound. If courts can accept the idea that proven and replicable programs can reliably enhance student achievement, then they can maintain that there is no excuse for the low achievement scores of high-poverty schools. Withholding effective methods could be seen as being tantamount to withholding effective medicines from poor children.
Many other state and district reforms have also been built around comprehensive reform models. Memphis, for example, gradually phased in a variety of models to almost all of its Title I schools (Ross & Smith, 2001). New York, Cincinnati, Washington, D.C., Los Angeles, Hartford (CT), and other districts have implemented large-scale reforms based on comprehensive reform models. California identified more than 300 low-performing schools and funded them to implement a range of models.
At this writing, Congress is debating the reauthorization of Title I. Although many issues are contentious, perhaps the most significant element of the reauthorization has been relatively noncontroversial. The House and Senate drafts of the main Title I bill contain language patterned on the Comprehensive School Reform Demonstration program. In the future, the $8 billion Title I program could increasingly focus on programs with evidence of effectiveness, rather than continue the focus on remedial pullouts and classroom aides that have been the main uses of Title I since its inception.
The Promise of Comprehensive Reform
The systemic reforms of the 1980s and 1990s set the stage for the comprehensive reform movement, but systemic reform alone can only go so far in creating fundamental changes in the materials and methods used by all teachers every day with every student. Comprehensive reforms create the possibility that rigorously evaluated programs can become the standard of practice in all schools, especially those serving many at-risk students. Not every school needs to adopt a comprehensive program, but as these programs build both capacity and evidence, schools with low achievement should increasingly have to demonstrate that whatever they are doing is no less effective than the comprehensive models available to them.
The comprehensive reform movement and the Comprehensive School Reform Demonstration program have the potential to enable educational practice to be based on rigorous research and development, following the pattern established by all the successful sectors of our economy, such as medicine, agriculture, and technology. These fields once paid little attention to research and development, but in the 20th century, practice in each of these disciplines began to be based on research as the quality and usefulness of the research increased. Perhaps in the 21st century, the same dynamic will finally take hold in education.
Critics of comprehensive reform have largely focused on the state of evaluation evidence for current programs. Yet this criticism misses the point. Whatever one thinks about current programs, how can anyone seriously oppose the idea that educational practice and policy should be based on evidence regarding effective materials and methods? There is controversy about research in every area of medicine and agriculture, for example, but no physician or farmer denies the value of research and development or the idea that research drives progress in their vocations. Policies favoring the development, rigorous evaluation, and dissemination of comprehensive reform models have the potential to move educational reform toward progressive improvement of practices and outcomes. Our children deserve no less.
