As a teacher and consultant with 30 years of experience, spent mostly in the Chicago Public Schools (CPS), I appreciated James Harvey's eloquent critique of the current education accountability movement in the November 2003 issue of Educational Leadership. Harvey creatively used the title of the popular movie The Matrix Reloaded to describe the current state of education accountability:<BQ>Like the society portrayed in the film, No Child Left Behind creates an artificial environment that bears little resemblance to the real world of schools.</BQ>
He asserted that this federal legislation makes promises it cannot keep, is underfunded, makes a mockery of local control, and ignores best practice.
I share Harvey's concern that the education accountability movement in the United States is pursuing a harmful and irresponsible course of action. The more reports I read and the more classrooms consumed with test preparation I observe, the more I am convinced that NCLB will not produce effective school reform. In fact, this legislation will leave large numbers of children further behind and will continue to befuddle the public about how to judge education quality.
In Chicago this year, some teachers are already complaining about the “NCLB children” transferring to their schools. These children's parents have taken advantage of NCLB's requirement that students in schools rated as “in need of improvement” must be allowed to transfer to other schools within the district. Like most of the rest of the public, these parents have begun to equate test scores with school quality. The receiving schools often assume a reactive stance as they scramble to deal with a flood of new students—rebalancing classrooms, reassigning teachers to remediation programs, and fretting about the possibility of lower test scores. Because their enrollment declines, the schools that lose these students experience teacher cuts and have fewer resources to distribute to children already “left behind.” Welcome to the nightmare of NCLB in a large urban school district.
The situation in Chicago and in many other districts brings to mind another movie title: Something's Gotta Give. In fact, inspired by Harvey's idea, I propose a number of popular films that might provide a lens through which to view the perils and challenges of education accountability.
Lost in Translation
In the movie Lost in Translation, actor Bill Murray's character struggles to communicate in an unfamiliar language and culture (Japanese). In education, we currently struggle to translate best instructional practices—those that we know promote student learning—into practices that boost scores on high-stakes tests. Unfortunately, much of what we know about holistic child development, balanced and research-based teaching, and authentic assessment is lost in the translation.
Research provides little evidence that current accountability systems accomplish anything apart from superficially raising test scores. In Chicago high schools, we have 9th graders reading below the 5th grade level who nevertheless met the 8th grade promotion cutoff score last year. Somehow, their elementary and middle school instruction prepared them to pass the mandated tests without enabling them to acquire good reading skills.
Yet as educators, we cannot afford to ignore such tests. I first understood how accountability could threaten good public schools six years ago when I worked with charter schools in Chicago. As test-driven accountability became the norm and academic watch lists and probationary status for “low-performing” schools became more prevalent, I felt panic at the prospect that many good schools with dedicated teachers and administrators would be closed. I reacted by learning more about test preparation strategies. Although I think of myself as an advocate for multiple measures of student achievement, I currently help schools and teachers with test preparation. Why? Because I don't like seeing students and teachers unfairly classified as failures.
Many other educators with similar concerns have responded the same way. For example, Lucy Calkins, the founding director of the Reading and Writing Project at Teachers College, Columbia University, wrote a book about helping students succeed on standardized reading tests (Calkins, Montgomery, & Santman, 1998). The catalyst: A great principal of her acquaintance was about to be fired from her position because of her school's low test scores.
Some people would say that this reaction is exactly what's wrong with educators: In our determination to protect our schools, we respond to test-based accountability with short-term fixes. In low-performing schools, we target and provide extra instructional support to 2nd- and 3rd-quartile students, knowing that we have a better chance of getting these students, rather than lower-performing students, above the mandated proficiency cutoff score and thus of raising the school's aggregate test-score level. Helping teachers and students play the game and navigate this unrealistic system of testing seems more achievable than negotiating with policymakers or fixing the systemic ills of society. Have we as educators sold out? Or are we just doing the best we can—trying to help students and teachers survive in an externally imposed accountability world that makes little sense?
Although we are sometimes forced to play the test preparation game, we haven't given up our quest to translate the good intentions of accountability reform into authentic learning for students. One elementary school close to my heart enrolls children from nearby housing projects filled with gang activity and domestic violence. Every year, the school faces the question of whether its test scores will be high enough to keep it open. This year, I am observing the school's implementation of the schoolwide reading reform program Success for All (SFA). Our hope is that SFA will not only boost test scores but also begin to develop motivated, lifelong, fluent readers among the school's population of at-risk students. Our biggest fear is that success on SFA assessments and genuine student progress will not translate into higher student performance on the state and district tests.
The Lion King and Titanic
In The Lion King, the young lion Simba initially shrinks from asserting his power but finally returns home to reclaim his throne and save his kingdom. In Titanic, the majority of passengers in steerage class—those at the bottom of the social and economic ladder—do not survive. These two movies offer a lesson about the alternative futures facing public schools. I hope that educators will find the courage to take a stand and rescue students, teachers, and schools from the injustice of an education system based on test results. If not, I fear that our most vulnerable students may suffer the fate of the lower-class Titanic passengers.
Educators who vehemently disagree with test-based accountability must become savvier about student assessment and speak persuasively to the public about the limitations of standardized achievement testing as a measure of school quality. They must become knowledgeable about the research that verifies their concerns. For example, W. James Popham (2001) coined the term psychometric mischief to explain that standardized tests are purposely designed to rank-order students, not to provide information that informs instruction. The public should also know that scores on these tests reflect students' economic backgrounds more accurately than they indicate school quality.
The Consortium on Chicago School Research recently compared students' performance on the district tests with their performance on the state tests mandated by NCLB (Easton, 2003). Interestingly, students performed similarly on both, despite the fact that these two tests are publicized as being very different. The district test (the Iowa Tests of Basic Skills, or ITBS) is norm-referenced, whereas the state test (the Illinois Standards Achievement Test, or ISAT) is touted as standards-based. But if you read the ISAT technical manual, you will find that the test is normed on the Stanford 9 Achievement Test. This means that both of the tests use a psychometric design that ensures score spread—in effect, guaranteeing that some students will do poorly. The public has a right to know and understand such information—and who better than educators to spread the word?
Finding Nemo
- Becoming better informed about the purposes and design of standardized achievement tests;
- Communicating knowledgeably and persuasively, not defensively, to the public about the alternatives to test-driven accountability;
- Designing and implementing multiple internal and external accountability tools that focus on the whole child (including cognitive, social, and emotional attributes); and
- Demanding high-quality performance from all teachers and administrators.
As educators, we are more qualified than policymakers and politicians to design our own accountability programs. In my role as a national school accreditation commissioner for the American Montessori Society, I have reviewed numerous school improvement models and data management programs designed to guide internal accountability. I don't feel any discomfort when asked to report quantifiable data or standardization of measures across a school, as long as educators—not psychometricians—control the content. One excellent tool is the National Study of School Evaluation's data-driven school improvement model (found at www.nsse.org), which guides a school or district in developing learning standards and quantifiable reporting measures and rubrics. This model allows schools to measure not only cognitive learning but also important social behaviors, such as teamwork.
An example at the school level is the initiative of the Young Women's Leadership Charter School in Chicago, whose teachers have worked diligently to design and implement a performance-based student assessment system. Instead of letter grades, teachers use performance ratings to report student progress in academics and learning behaviors; students receive a rating of High Performance, Proficient, or Not Yet for each course outcome. The staff continues to refine its standards and rubrics to report student learning. This labor-intensive work pays off: When the students conduct PowerPoint presentations of their year-end school portfolios, they exhibit the kinds of meaningful learning that standardized tests cannot measure, such as verbal communication, oral presentation skills, and use of technology.
Time to Take a More Active Role
Clearly, the accountability dilemma has elements of many film genres, including science fiction, comedy, adventure, and drama. Many educators are ready to stop being spectators and to step into roles in which they can direct the action. But, like me, they may be torn about how to respond to the misguided movement to judge schools, teachers, and students on the basis of isolated tests.
James Harvey and several other authors in the November 2003 issue of Educational Leadership also struggle with this dilemma. Although their messages are diverse, they all reinforce the idea that we're in trouble with education accountability and the direction of school reform. I share Richard F. Elmore's belief that we need to build systemic school capacity and use strategic school improvement plans to enhance teaching and learning. I also agree with Craig Jerald that administrators, teachers, and students must hold themselves accountable. Frederick M. Hess is also correct that we must become more relentless about ridding the profession of ineffective teachers and changing inflexible work rules to get the job of educating done properly.
But Harvey's article offers the clearest critique of what he calls the “punitive, test-driven” accountability movement, and the strongest call for educators to take back ownership of our schools. To accomplish this goal, we must design more realistic standards that teachers can clearly comprehend and integrate into their instruction. And we must measure the achievement of these standards with multiple assessments that are valid, reliable, and aligned with curriculum and instruction.
