Although only about 10 percent of schools thus far have extensively restructured themselves, observing those schools has led researchers to gain important new insights about authentic pedagogy.
After studying dozens of schools that have attempted to restructure themselves, researchers concluded that schools interested in serious reform should begin by addressing two questions: How can learning have intellectual quality? How do we build a community of learners?
During their five years of work, these researchers also gained some important new insights about assessing the quality of instruction and evaluating the quality of student work. Here, Fred Newmann, director of The Center on Organization and Restructuring of Schools, describes the five different studies that led to these insights.
Your research center—The Center on Organization and Restructuring of Schools—is wrapping up five years of study. What did you hope to learn from your research?
The Center's work was organized around five major projects. The most ambitious project was an intensive study of 24 schools across the United States that were deeply involved in restructuring. A second project, focused on professional community, was a four-year study of eight schools. The third project used data from the NELS (National Education Longitudinal Study of 1988) database, which is a sample of students from about 1,000 U.S. schools that followed their 8th grade students all the way through high school. We did a number of statistical analyses of that huge database.
The fourth project looked at the delegation of authority to local school councils in Chicago, the most radical decentralization reform the United States has tried. The fifth project studied community collaboration: efforts by school systems to work with community organizations to coordinate the delivery of social services to students and their families.
We wanted to look at the effects of innovative organizational changes on what we call authentic instruction and authentic student achievement. We were curious about equity—the equitable distribution of opportunities to learn. We wanted to know how restructuring was affecting efforts to empower various stakeholders: teachers, administrators, parents, and students. We were interested in development of more cohesive communities, especially professional communities in which teachers have the opportunity to pursue change through reflective dialogue rather than simply following mandated change. Finally, with all the talk about accountability, we wanted to know whether new structures included useful new approaches to being accountable. So the major themes of our research have been authentic learning, equity, empowerment, professional community, and accountability.
You're saying these ideas were discussed in the professional and public policy literature as the purposes for restructuring schools, and you wanted to find out whether restructuring really contributed to achieving these aims.
Right. And if so, how.
How many people were involved in your research?
At various times 86 people have served as researchers or support staff. Usually, there were at least 10 experienced faculty-level researchers and about 25 experienced teachers/researchers, including graduate students and academic staff. Besides those at the University of Wisconsin, these researchers were at the University of Michigan, the University of Chicago, the University of Minnesota, and Hofstra University.
You said your centerpiece was a study of 24 restructuring schools.
We put most of our resources into that study, but the other projects were critical for cross validation of key findings. The School Restructuring Study (SRS) presented us with difficult choices. Ideally, one would want to compare a set of “restructured” schools with a set of “conventional” schools in matched control design, over a period of several years. But the restructuring movement has been so ambiguous, and resources for research so limited, that we didn't see a way to divide schools into two separate groups and compare them.
Even trying to define restructuring is problematic, as you know. Based on the restructuring literature, we defined a restructuring school as one that had made major departures from conventional practice in numerous ways, which could include anything from site-based management to heterogeneous grouping. We published a list of 36 such departures. Instead of trying to find a control group, we chose to seek the best examples of educational innovation in public schools and examine how they contributed to the themes we had identified.
How did you go about finding the 24 schools?
We asked for nominations from all the national professional organizations, including ASCD; all the reform organizations we knew about; and all state departments of education. We sent a questionnaire to schools that had been nominated or nominated themselves. We then followed up with a detailed interview with the principal or other informed person at the school. Finally, we made site visits to schools that seemed to have the most substantial restructuring—about 70—and on the basis of the site visits, chose eight elementary, eight middle, and eight high schools.
Would it be accurate to say that your search found that the number of U.S. schools that were extensively restructuring was quite small?
Yes. It depends how you define extensive restructuring, but my estimate is fewer than 10 percent.
When your researchers went to the 24 schools, what were they looking for, and how did they look for it?
Our main interest was the quality of instruction and student learning. From our point of view, there's no point in restructuring unless it helps kids learn. We couldn't study all subjects, so we decided to look at just mathematics and social studies. We couldn't study all grade levels, so we chose grades 4 or 5 elementary level, grades 7 or 8 middle school level, and grades 9 or 10 at the high school level.
In each school we observed three math and three social studies classes four times during the year on two separate visits. (We studied eight schools each year in this way, so it took us three years to visit all 24 schools.) Besides the classroom observations, we spent a lot of time interviewing teachers, parents, and administrators. Our observations and interviews focused on equity, empowerment, professional community, and accountability, as well as teaching and learning.
As our researchers asked questions related to the five themes, we looked for explanations: why was something happening? We used different categories to guide our inquiry. For example, had there been some structural change, such as longer planning periods for teachers, that helped explain what we were seeing? What technical attributes of the school, such as program content, might explain it? For example, could use of a particular mathematics textbook help account for the quality of instruction we were seeing? Finally, what cultural attributes, especially norms and beliefs people held, could account for the kinds of teaching and learning we observed?
Some people might ask, “If you want to find out the effects of restructuring, why not just look at student achievement?” I believe you were one of the first to use the term authentic assessment, and I know you have serious reservations about the validity of standardized test scores as measures of student learning. That must have really complicated your work.
It did. But even if you wanted to use standardized tests, there was a real logistical problem. We were studying schools all over the country, and there's no test given by every school across the country. One option would be to choose a conventional test and ask all the schools to give it, but this would impose an unfair burden on the schools. And our main interest was student performance that demonstrated thinking and depth of understanding.
Another option was to create our own more authentic measures. But that would also be an imposition on the schools—and any such assessment tasks would not necessarily match the curriculum of the diverse schools.
We decided to ask the teachers to choose and send to us key assessments that they had given in the course of their instruction—assessments they thought were valid and meaningful indicators of their students' achievement. We would figure out how to evaluate the quality of student work done to meet the school's own assessment objectives.
As I understand it, that decision not only affected how you would study the effects of restructuring but led to an important outcome of the project.
Right. It forced us to develop ways to assess the quality of assessment tasks, and the quality of student responses. And because we were observing the same teachers' classes, we also had to have standards for assessing the quality of instruction.
Several years ago we published an article in which you and Gary Wehlage explained your definition of authentic instruction.
Yes. But we looked at more than instruction. Teachers' practice includes lots of complex work, but there are two critical parts. One is instruction—deliberately teaching, explaining, giving feedback; the other is assessment: what teachers do to find out what students have learned. They aren't easy to separate—because assessment isn't done just at the end of the instruction or apart from it—but you can distinguish between them. When you put assessment together with instruction, you have pedagogy. So our effort has been to define and study what we call authentic pedagogy.
And the result is authentic student performance?
Yes.
When you look at a piece of student work, how do you judge whether it's authentic?
We think authentic academic performance has three characteristics. The first thing we look for is construction of knowledge. Has the student successfully analyzed and interpreted information as opposed to simply reproducing it? This is similar to higher-order thinking.
But there are some kinds of knowledge construction—like making up jokes—that we wouldn't necessarily value academically. So the second standard we emphasize is disciplined inquiry. That means working within a knowledge base, investigating something in depth, and using elaborated communication to convey what you're learning. The third standard for authentic performance is that the discourse, product, or performance that students produce have some value or meaning to the student or to others beyond certifying mastery in school.
And you developed ways to judge student work in terms of those three criteria?
Yes, and because we think these ideas might be useful to educators interested in the quality of instruction and student work, we've published a guide that lays out the rationale for authentic pedagogy and authentic student performance along with standards for evaluating instruction, assessment tasks, and student work [See p. 73.] The guide has examples of high-quality lessons, assessment tasks, and student work that we found in the 24 schools.
I know it's very difficult to summarize the results of five years of research on several different matters, but I'll ask, anyway. What, in general, did you learn?
The good news is that school restructuring can promote both authentic learning and higher scores on conventional measures of achievement.
You were able to establish a direct connection between restructuring and improved student learning?
Yes. Our School Restructuring Study showed that authentic pedagogy in restructuring schools enhances authentic achievement, and the NELS studies showed that restructuring high schools, in contrast to traditional ones, enhance conventional achievement.
Another part of the good news has to do with our concern for equity. First, we found that restructured schools can deliver authentic pedagogy equally to students regardless of gender, socioeconomic status, race, or ethnicity; that is, they can offer equal opportunity to learn. Second, restructuring can have equitable effects on achievement. The School Restructuring Study found that authentic pedagogy brought equal achievement benefits to students of different gender, economic status, race, and ethnicity. NELS showed that restructuring can even reduce inequalities in achievement between students of higher and lower socioeconomic status.
So that's the good news. Is there some bad news?
Not necessarily, but there's news that complicates the picture. Even though we found that restructuring can boost student achievement and do it equitably, we've concluded that it's not structural changes alone that make for success. Even in our select sample of restructured schools there was tremendous variability in the quality of instruction and the quality of student work.
So why doesn't restructuring help all schools improve student achievement?
Most restructuring efforts—site-based management, team teaching, cooperative learning, alternative scheduling—are simply tools educators can use to get more authentic learning. But like hammers and screwdrivers, the effectiveness depends on your purpose and on how you use them. Not all schools use the tools the same way.
But those tools are what most people mean by restructuring. If it's not the structures that explain the success of restructured schools, what is it?
It's largely culture, particularly two parts of culture. The first is adults' depth of concern for the intellectual quality of student learning, in contrast to concern for techniques, such as whether to have portfolios or whether to eliminate all ability grouping. The second is the extent to which the school creates a professional community that harnesses and develops individual commitment and talent into a group effort that pushes for learning of high intellectual quality. We found in both the select restructuring schools and the large sample of NELS high schools that the level of professional community in a school contributed to authentic pedagogy and to student achievement.
We hope our research will help to change the conversation from being concerned with tools and techniques to focusing on these two central ideas: how do we get learning of high intellectual quality in our schools, and how do we create professional community in schools in order to achieve intellectual quality? Once these questions are the starting point, then it becomes important to think about the tools and structures needed to answer them.
How do you think educators can use the results of your work?
First, we've found that restructuring can move schools in the right direction. When they provide authentic pedagogy, they boost student achievement—for kids of all socioeconomic backgrounds.
Second, we think our guide to authentic instruction and assessment can be used to build professional community aimed at high standards. Educators can use our findings to redirect attention from the managerial/logistical issues raised by new practices, new procedures, and new structures. Our research can help them focus instead on what's really important: defining standards for high-quality student learning, and building a professional community that supports intellectual quality.
