Outcome-based education (OBE) has an intuitive appeal that hooks people. Simply set the outcomes you expect students to achieve, then teach and reteach in as many different ways and for as long as it takes until everyone meets them. In its simplest form, the OBE process virtually guarantees every student an education. And in this light, what is surprising is not that 42 states are involved in some form of outcome-based reform (Varnon and King 1993), but rather that they have waited until now to try it.
Despite OBE's appeal, however, research documenting its effects is fairly rare. An earlier literature review reported that existing evidence was largely perceptual, anecdotal, and small scale (Evans and King 1992), and our recent search for additional published information led to the same conclusion. Testimonials, speeches, and narrative descriptions may be inspirational and helpful, but they provide little solid ground on which to build a reform movement.
Admittedly, researchers attempting to “prove” the effects of outcome-based education face various problems. First, OBE is an umbrella concept under which various reform efforts can be placed, and people who ask, “What exactly is outcome-based education?” may receive several answers. Block, Efthim, and Burns (1989) include OBE in their conceptual overview of mastery learning, but mastery learning is not the only way to implement OBE. The fact that people who practice open education also claim to engage in outcome-based education suggests the breadth of the concept.
Creating another complication, Spady and Marshall (1991) distinguish among traditional, transitional, and transformational approaches to schoolwide or districtwide OBE, noting that the first can operate within an existing school system, while the last requires the creation of a whole new system. Even more basic is the distinction between OBE and the setting of high school graduation outcomes. Just because a state requires its high school graduates to achieve specific outcomes does not mean that schools necessarily engage in outcome-based education to prepare students for graduation assessments. And finally, a movement that purports to develop “complex thinkers,” “responsible citizens,” and “community contributors” faces complex measurement challenges, both conceptual and practical (Minneapolis Public Schools 1992).
Despite these limitations, a small but growing body of OBE research does exist. Relevant research includes information on classroom-based mastery learning; over 20 years of evidence from the Outcomes-Driven Developmental Model in Johnson City, New York; and studies from state-level OBE projects in Utah, Missouri, and Minnesota.
The Effects of Mastery Learning
Although OBE does not require mastery learning as an exclusive instructional model, many people consider mastery learning to be an integral part of OBE beliefs and practice (Burns 1987, Schleisman and King 1990, Spady 1982). In a comprehensive meta-analysis, Kulik, J. Kulik, and Bangert-Drowns (1990) integrate the various findings of the last decade (Guskey and Gates 1985, J. Kulik et al. 1979, Slavin 1987) and address inconsistencies about the effects of mastery learning. They examine 108 studies on Bloom's Learning for Mastery and Keller's Personalized System of Instruction. Both approaches present material in short units, and students take formative tests on each unit.
The meta-analysis indicates that the average student in a mastery learning class performed at the 70th percentile, whereas the average student in a control class performed at the 50th percentile. The authors conclude that mastery learning does have positive effects on student achievement. Results were better for social science classes, on locally developed tests, in teacher-paced classes, when the required level of performance was high, and when the control group received less feedback. They also note that lower-aptitude students enjoyed a greater gain than higher-aptitude students. The most consistently negative effect was on course completion. Students in mastery learning classes completed fewer courses than students in control classes.
Although the authors argue that the positive effects are not as great as Bloom predicts, they do state that the overall effect of mastery learning is impressive when compared with other educational treatments: “Few educational treatments of any sort were consistently associated with achievement effects as large as those produced by mastery teaching.”
The Outcomes-Driven Developmental Model
One current working model of an outcome-based educational program at the district level is the Outcomes-Driven Developmental Model (ODDM) begun in 1972 in the Johnson City, New York, School District under Superintendent Albert Mamary.
The developers of ODDM make clear that it is an “empowering, participatory, and noncoercive” change process (Alessi et al. 1991). Johnson City's ODDM, using an instructional model similar to Learning for Mastery, has been so successful it is the only total school curriculum model validated by the National Diffusion Network (Vickery 1990). Adding to its credibility is the fact that Johnson City is a lower-middle-class community with few professional citizens and the second highest poverty rate of 10 urban districts in its county. Over 20 percent of its school population qualifies for free or reduced-price lunch, and it has a sizable Asian immigrant population with limited English proficiency. (See “On Creating an Environment Where All Students Learn: A Conversation with Al Mamary,” p. 24).
When Johnson City began its program in 1972, it ranked 14th out of 14 districts in its county on academic achievement as measured on standardized tests. Approximately 45 to 50 percent of its students scored at or above grade level in reading and math in grades 1 through 8. By 1977, the percentages rose to about 70 percent, and by 1984 ranged between 80 and 90 percent.
To have a consistent measure for tracking student progress, in 1984 the district chose to identify the number of students whose scores indicated achievement six months or more above grade level on the California Achievement Tests (CATs) in reading and math. They found that in 1976, 44 percent of all students had performed at six months or more above grade level in reading, and 53 percent had done so in math. By 1984 these figures had increased to 75 percent in reading and 79 percent in math.
Other indicators of success in Johnson City include performance on the New York State Regents exams and attainment of the Regents diploma. In 1989, for example, Johnson City students, on every exam, always surpassed the state performance and either equaled or surpassed the county performance (with 70 percent of Johnson City students participating, 58 percent in the county, and 40 percent of students statewide participating in the exams). In 1986, 77 percent of Johnson City students received a Regents diploma, compared with 43 percent statewide and 59 percent countywide.
In 1988, the New York Board of Regents instituted more rigorous requirements for a Regents diploma. In 1989, Johnson City still outperformed the state and county, with 55 percent of its students receiving the diploma, compared with 33 percent statewide and 47 percent countywide. This placed Johnson City in the top 10 percent of schools statewide in percentage of students receiving Regents diplomas. These figures aside, however, perhaps the most convincing evidence of Johnson City's success is the 100 percent enrollment, in 1991–92, of students in 9th grade algebra.
Lessons from States
At the state level, documentation of the effects of OBE is difficult to find, and what is available is largely perceptual. Nevertheless, data collected in Utah, Missouri, and Minnesota provide useful insights.
- Implementation of OBE generally requires a restructuring of the entire educational system and consequently takes a significant period of time.
- More OBE implementation takes place in districts that have adopted ODDM as a model than in other districts.
- More OBE implementation takes place in elementary schools than in secondary schools, and in smaller districts than in larger districts.
- Although the evidence is limited, districts with more complete implementation of OBE also appear to demonstrate higher student achievement gains (Applegate 1992).
- Districts using ODDM seem to be experiencing the most successful implementations.
Another state effort noted by OBE proponents is Missouri's Statewide Project for Improving Student Achievement (Cohen and Hyman 1991, Guskey and Block 1991). This project, called the Instructional Management System, involves the following components: (1) a statewide curriculum; (2) three state-endorsed instructional programs (mastery learning, outcome-based education, and cooperative learning); and (3) a criterion test, the Missouri Mastery Achievement Test (MMAT) that precisely measures the curriculum's outcomes.
Beginning in 1986–87, scores on the mastery test have significantly risen statewide each year in nearly every subject area. At the same time, scores have increased on norm-referenced tests, including the Iowa Tests of Basic Skills for grades 2 through 8 and the Test of Achievement and Proficiency for grades 9 and 10.
One example of the Missouri project's success is an “Academic Achievement Demonstration Site,” the Thorpe Gordon Elementary School in Jefferson City. In 1987, approximately 40 to 60 percent of the students in this inner-city school ranked in the bottom two quintiles in language arts, mathematics, and science on the MMAT. By 1989, 10 percent or less were in the bottom two quintiles, with few students placing in the lowest one. In addition, 70 to 90 percent now rank in the top two quintiles, with 50 to 75 percent in the highest (Guskey and Block 1991).
In Minnesota, the Department of Education's Office of Educational Leadership worked in 10 project sites across the state from 1989 through 1991 to determine the effectiveness of an outcome-based system of education in improving student learning (Minnesota Department of Education 1990, Center for Applied Research and Educational Improvement 1991, Bosma and King 1992). Research activities, including interviews with students, teachers, administrators, and parents, sought to document the perceived effects of the changes made, that is, to provide initial evidence about what was happening to students as a result of the transformational OBE approaches being implemented (King et al. 1992).
The results across 37 schools involved during 1990–1991 included three perceived effects on student learning. Forty-nine percent of the respondents reported more and better learning. (“I've gotten a lot more out of class than the last few years.” “There's been a tremendous increase in student learning.” “We have set higher expectations, and students are achieving more.”) Forty-three percent reported increased student involvement in learning. (“Kids really take a stake in learning and are more responsible.” “I'm pushing myself more.”) Thirty-five percent reported different effects for different student types. Many parents expressed a sense that OBE “works for the average and unmotivated learner,” both because these students are allowed sufficient time and opportunities to succeed and because some become part of regular instruction for the first time. But many respondents reported negative perceptions for students who have succeeded in the traditional system. (“Admittedly we have picked up some we would've lost, but we are losing some at the top.” “We feel the higher students won't be challenged enough.”)
OBE's Possibilities
What, then, can we conclude about OBE as a restructuring effort to date? Acknowledging the paucity of hard data, we find at least three themes. First, the data from research on mastery learning; Johnson City, New York; Missouri; and Utah suggest that mastery learning and its ODDM implementations are effective at the classroom and building levels. Second, experiences in Johnson City and Utah indicate that the Outcomes-Driven Developmental Model can work and is readily adapted into traditional systems. Third, the mastery learning and Minnesota data document that OBE appears to benefit low-achieving students while having questionable effects on high-achieving students (Evans and King, in press). These three points speak cogently to the emerging possibilities of OBE within the traditional system. However, whether transformational OBE can effect similar changes remains to be seen.
What the data—or lack of it—suggest is the compelling need for more research. But not just any research will do. First, we must be clear what we mean by OBE. Second, we must determine what it is we want to do well in schools and how that can be best documented. For example, are we committed to “authentic” learning, with measures that tap such achievement (Newmann and Archbald 1992), or will we settle for improvements on standardized tests?
In our work with OBE and its increasing numbers of dedicated educators, we have become convinced that traditional studies are simply not rich enough to portray the changes that an OBE system may inspire. And so, we challenge researchers to devise innovative evaluation methodologies that truly capture the excitement of real and lasting change in schools.
