Most large districts have a wide range of schools, with a flagship school at one end of the performance spectrum and a bottom-of-the-barrel school at the other. People see the flagship as a paragon of excellence; teachers vie for assignments there. It is located in one of the district's most affluent neighborhoods, and its students lead the district on standardized test scores and most other academic indicators. The school is credited with a high degree of effectiveness, which is generally attributed to the quality of its administration, teaching staff, and educational programs.
By contrast, the bottom school is usually in one of the poorest parts of town and usually ranks last, or nearly last, in the district on most academic measures. The school is often labeled subpar or substandard, and its students are often presumed to be less intelligent or less talented than their flagship counterparts. Administrators and staff frequently find themselves defending the school's effectiveness and their own competence and dedication. In discussions about what's wrong with education, the bottom school is sure to be cited as an example. (Any mention of what's right is usually related to athletics.)
How do school districts address the gap between the two schools? All too often they begin with questions like, How can we make the bottom-of-the-barrel school more effective? or How can we bring its performance up to par? Yet that is perhaps the most inequitable approach of all: It validates and reinforces what may be a myth, namely, that the bottom school is inferior to the flagship school.
The Wrong Questions
If we are sincere about closing the equity gap, we must first distinguish between reality and myth. That requires going beyond traditional measures of school quality and effectiveness. So long as schools are judged solely on the basis of single-shot outcome measures, any resulting decisions or actions are bound to be inequitable.
Recently, for example, the Sweetwater Union High School District, a large urban secondary school district in southern California, began challenging the myths associated with its lowest achieving middle school—National City Middle School in National City, California. Despite talented leadership and exemplary teachers, that school consistently ranked at or near the bottom academically. Its poor reputation took a heavy toll on staff morale and, more important, on student self-esteem. But was that reputation in fact deserved? The answer surprised everyone.
Raising Writing Scores
National City Middle School's staff members were particularly concerned about student performance on a state-mandated demonstration of minimum writing competency. As a requirement for high school graduation, all district students must satisfactorily write several paragraphs in response to a given prompt. The district generates these prompts, and two or three trained readers grade the students' work. Final scores range from zero to 18, with 10 required for passing.
For many years, National City Middle School's pass rate and mean score on the writing exam were the lowest of the district's nine middle and junior high schools, while the flagship's were the highest. As a result, National City Middle School made improved writing performance a school objective. With a degree of dedication that has become a hallmark of the school, students and staff devoted a year of intensive effort to writing. As part of a project I was completing for my doctorate, I had an opportunity to study the school's efforts. I observed a process far more interesting and complex than I had anticipated.
Whole Language or Mechanics?
Although everyone at the middle school agreed on the need to improve performance, an enormous rift developed over how to do it. The Language Arts department was divided into two camps: whole language proponents, who sought ways to improve writing through a global reading, writing, listening, and verbal skills package, and writing mechanics proponents, who favored focusing on practicing and polishing specific writing skills.
These philosophical differences led to a certain amount of discord, but they also complemented each other. The whole-language emphasis led to significant changes in the curriculum. Language Arts teachers added multicultural literature to the core literature list. They adopted a variety of authentic assessment techniques, including a schoolwide portfolio program that stressed students' growth. And teachers employed specially designed academic instruction in English—instructional scaffolding—for students with limited proficiency in English.
Because most students read below grade level, staff members instituted two reading programs: a computer program for reading comprehension and a schoolwide program in which teachers read aloud to students or students read silently the last 15 minutes of each day. The staff also planned a reading institute to familiarize families with the school's programs and with ways to support reading at home.
The writing mechanics focus, on the other hand, resulted in a 10-hour grammatical usage course for selected 8th graders, who took a monthly test in the subject. In addition, two or three mornings a week, writing tips were televised on the in-house television circuit, reaching students schoolwide. Each broadcast lasted five minutes and was devoted to a specific writing or grammatical issue—run-on sentences, sentence fragments, subject-verb agreement, and so on. The faculty also held monthly writing contests, awarding prizes and certificates, and they en-couraged students to contribute to Vignettes, the school's literary magazine.
Dissension was not limited to the Language Arts department. Even as teachers in other departments voiced support of reading and writing instruction across the curriculum, many held the Language Arts teachers responsible for the students' poor writing performance. Additional tension surfaced when some teachers felt their competence was being questioned by administrators or other teachers. Resolving those issues required endless patience, tolerance, and perseverance.
Ultimately, the staff found ways to accommodate the various philosophies, beliefs, and emotions without compromising objectives or detracting from the quality of instruction. It was understandable, then, that after a year's effort, school personnel were utterly dismayed to learn that their students still had the lowest pass rate and mean writing score in the district. The disappointment was profound.
Gauging Growth
Although final outcomes are indispensable for determining students' achievement at a given time, they do not provide an accurate gauge of school effectiveness because they fail to register change. That's where Webster and Mendro's (1995) value-added approach came in. Like Astin's (1991) talent development approach, the value-added approach indicates students' academic growth over a given period of time. Using this approach, National City Middle School faculty members compared students' final outcomes on the writing test to their incoming performance. Only then were they able to see not only the students' present standing but also how far they had come.
The middle school staff members then used the value-added approach and conventional statistical procedures for analyzing matched student data, rigorously comparing their students' writing scores to those of students at the flagship school. The results were startling. As usual, the flagship's gain was the highest in the district. But National City Middle School trailed only slightly behind, with a lower gain that was statistically insignificant. In 1992-93, only 16 percent (104 students) of the National City Middle School student body had achieved a passing grade of at least 10 on the writing test. By 1996-97, that percentage rose to 30.8 percent (233 students). In other words, National City Middle School was as effective as the school considered the best in the district in raising its students' writing scores.
Indeed, National City Middle School's effectiveness extended well beyond writing performance. A comparison of like populations at the two schools—students who were not fluent in the English language—revealed that there were no significant differences between these students on most academic indicators. In fact, additional analyses revealed that once researchers controlled for incoming language ability, the advantage of being at one school or the other was virtually undetectable.
Researchers then conducted a statistical procedure to predict 8th graders' year-end writing performance based on their attributes and performance when they began the 7th grade. Considering only factors over which the schools had no control—English language proficiency, gender, and home language—it was possible to predict with 78.2 percent accuracy which 8th graders would pass the writing exam. Knowing which of the two schools the 8th graders attended increased the accuracy rate only slightly—to 78.6 percent. Thus, the two schools' effectiveness with students with similar characteristics differed very little.
Getting to First Base
- More than 37 percent of National City Middle School's students are not fully proficient in the English language, compared to fewer than 7 percent of the flagship school's.
- Nearly a third of National City Middle School's students live below the poverty line, compared to fewer than 5 percent at the flagship school.
- More than 20 percent of National City Middle School's students are from homes in which parents have less than a 9th grade education and annual incomes of less than $10,000, compared to about 3 percent at the flagship school.
These demographics do not mean that National City Middle School's students are less intelligent or less talented than the flagship school's. What it does mean is that students who already speak English fluently and whose parents can afford computers and modems, summer enrichment experiences, and individual tutoring generally enjoy a jump at the starting gate when entering school.
Fueling Incentive
The effects of reporting National City Middle School's value-added results were overwhelmingly positive. After years of being at the bottom, students and staff finally were recognized for the excellent job they did, and that reenergized the school. Teachers were now eager to be involved in writing instruction as members of a winning team. As for the students, where once they dreaded the writing exam and the reports of failure that inevitably followed, they were now asking teachers when they would take the exam again.
Perhaps the most telling change, however, was the public's perception of the school. "Our school used to be the school where nobody wanted to teach," one teacher observed. "Now teachers are transferring here from other schools."
Of course, growth did not make up for the final product. The bottom line remained the same: Students needed to pass the writing exam in order to graduate. The staff did not become complacent about the school's low pass rate or low mean writing score. On the contrary, when students and teachers saw that their efforts were paying off, it heightened their expectations, providing the fuel they needed on their uphill climb.
Assessing Schools Fairly
Partially as a result of state and federal reporting procedures, final outcome measures all too often have been equated with school quality. That simplistic approach not only perpetuates the inequity already present in the educational system, but introduces an additional element of injustice that further undermines the efforts of educators who are attempting to narrow the equity gaps.
But individual schools and their students are not the only victims. Entire school districts have been relegated to bottom-of-the-barrel status simply because, in comparison to their prestigious counterparts, they have a much higher proportion of students from families that don't speak English or whose earnings put them below the poverty line. So long as these districts are judged by final-outcome measures alone, without regard to academic gain, they will continue to be viewed as inferior to districts with more affluent students, even when students in the affluent districts don't register any academic gains.
Further, by giving the flagship school a false sense of progress, this kind of misjudgment of effectiveness can be equally harmful to its students. Which is better—for students to enter with a score of 9 and move up to a passing score of 10 two years later, or to enter with a 4 and move up to a 9? Final outcome is certainly critical, but should it be our only concern? Are we content to have students coast and then stop altogether once they achieve the minimum outcome required?
When we finally begin to assess school effectiveness fairly, we may see a shift in the way schools are popularly perceived and labeled. Many schools that have been relegated to the bottom may gain prestige, while some more highly regarded schools may need to redouble their efforts. We would then be providing an incentive for flagship schools to strive for growth, while also recognizing and reinforcing the accomplishments of many schools that work primarily with less advantaged students. For once, all schools would be accountable for what goes on in their classrooms, an accountability that would benefit all students.
