How does mathematics instruction differ from country to country? What do these international comparisons tell us about how to improve mathematics achievement?
We have been working for 10 years on a research program aimed at answering these questions. The TIMSS video studies document typical teaching practices in various countries. These studies employ the video survey, a novel methodology that combines two research traditions: qualitative classroom research and large-scale survey research. The video studies capture close-up pictures of the classroom processes used by national samples of 8th grade mathematics teachers in different countries. These teachers are not necessarily experienced or effective. They are ordinary teachers, teaching lessons that they routinely teach.
Why would we want to study a random sample of ordinary lessons? First, these lessons together represent what average teaching looks like in different countries. If we want to improve student learning, we must find a way to improve teaching in the average classroom. Even slight improvements in the average can positively affect millions of students. This concept represents a new way to formulate the question of how to improve teaching.
Second, studying a national sample of classroom lessons can help us discover whether policy initiatives have influenced classroom practice. All reform efforts to improve teaching and learning must pass through a final common pathway: the classroom. Most reforms get stopped short at the classroom door; all available evidence suggests that classroom practice has changed little in the past 100 years.
Finally, studying lessons from different cultures gives researchers and teachers the opportunity to discover alternative ideas about how we can teach mathematics. Watching lessons from other countries prompts questions about the assumptions that guide common practices in our own country. It is often a startling experience to journey back and forth, looking first at foreign videos and then back at our own.
The First TIMSS Video Study
The TIMSS 1995 video study (Stigler & Hiebert, 1999) examined national samples of 8th grade mathematics lessons from three countries: Germany, Japan, and the United States. Several findings from the first study provide important background information.
Lack of a Shared Language to Describe Teaching
The lack of a shared language for describing teaching makes it very difficult to generate and disseminate professional knowledge. Even before beginning the video studies, we suspected that this problem existed; indeed, that suspicion was one of the reasons we chose to document teaching through videotapes instead of questionnaires. As the tapes started to arrive and we discussed what we saw on them, it became obvious that different people saw different things and described what they saw in different ways.
For example, we tried in the first study to mark where on the video each mathematics problem started and where it ended, a process that we thought might simplify our task by enabling us to analyze each problem separately. We could not agree on what a problem was (although we did manage to do so in the later study). Some observers would only count an activity as a problem if it involved students in sustained thinking over a long period of time. Others might count as a problem a brief exercise that students could solve quickly by recalling a solution that they had previously been taught. The word “problem” clearly means different things to different people. Other words and phrases, such as “develop concepts” or “teach for understanding,” pose similar challenges.
Slippage Between Policy and Classroom Practice
In part because we lack a shared language, attempts by policymakers to change what happens in classrooms often achieve either no results or unintended results as reform efforts get filtered through the weak communication channels we rely on to disseminate policy (Elmore, 2000). In our first video study, we asked teachers whether they had read mathematics education reform documents (for example, those published by the National Council of Teachers of Mathematics) and whether they implemented the documents' recommendations in their classrooms. Most teachers said that they had read such documents and that they used the reform ideas in their classrooms. However, the videos revealed great unevenness in how teachers interpreted the reforms and showed little evidence that classroom practices actually reflected the goals of the reforms.
The Cultural Nature of Teaching
We concluded from our first study that teaching is a cultural activity: learned implicitly, hard to see from within the culture, and hard to change. We were struck by the homogeneity of teaching methods observed within each country and by the striking differences in methods we observed across Germany, Japan, and the United States. Even in the United States, a country with great diversity in language, ethnicity, and economic conditions and an education system controlled by local governing boards, the nationwide variation in 8th grade mathematics teaching was much smaller than we had expected.
The 1999 TIMSS Video Study
The TIMSS 1999 video study expanded on the first study. In addition to the United States, we included Australia, the Czech Republic, Hong Kong, Japan, the Netherlands, and Switzerland. Each of these countries performed significantly higher than the United States did on the TIMSS 1995 mathematics achievement test for 8th grade.
The design of the 1999 video study was simple. We selected a random sample of 100 8th grade mathematics classrooms from each country and videotaped them at some point during the school year. We digitized, transcribed, and translated the tapes into English, after which an international team of researchers analyzed them. Coding and analysis focused on the organization of lessons, the mathematical content of lessons, and the ways in which the class worked on the content as the lessons unfolded. Here are some of the most interesting findings.
Effective Teaching Takes Many Forms
In both the 1995 study and the 1999 study, teaching methods in Japan differed markedly from what we observed in all of the other countries. Japanese students, for example, spent an average of 15 minutes working on each mathematics problem during the lesson, in part because students often were asked to develop their own solution procedures for problems that they had not seen before.
Because Japan was the only high-achieving country in the first video study (as indicated by students' performance on the testing component of TIMSS), many researchers assumed that the United States would need to copy Japanese methods to produce the levels of learning displayed by Japanese students. The 1999 study, however, makes it clear that despite the well-crafted nature of the Japanese lessons, high achievement does not necessitate a Japanese style of teaching. Other countries posted high scores with lessons that looked decidedly un-Japanese. For example, in contrast to the relatively long time spent on each problem in Japan, every other country in the study spent only up to five minutes on the average problem.
The videotapes from each country reveal a unique combination of features. Many teaching methods that are hotly debated in the United States vary among the six higher-achieving countries. For example, the Netherlands uses calculators and real-world problem scenarios quite frequently. Japan does neither. Yet both countries have high levels of student achievement.
As another example, consider the debate over what kinds of problems students should work on during the mathematics lesson: basic computational skills and procedures (using procedures problems) or rich mathematical problems that focus on concepts and connections among mathematical ideas (making connections problems). Figure 1 shows the percentage of each kind of problem observed in six of the seven countries.
Figure 1. Types of Math Problems Presented
Japan is an outlier; 54 percent of the problems observed in the country's classrooms were making connections problems. But note that Hong Kong, one of the highest-achieving countries in the study, is at the opposite end of the continuum, with only 13 percent of problems coded as making connections. Classrooms in all of the countries spend time both on problems that call for using procedures and on those that call for working on concepts or making connections. The percentage of problems presented in each category, however, does not appear to predict students' performance on achievement tests.
Implementation Is Important
What, then, do the higher-achieving countries have in common? The answer does not lie in the organization of classrooms, the kinds of technologies used, or even the types of problems presented to students, but in the way in which teachers and students work on problems as the lesson unfolds.
In the 1999 video study, we coded each problem twice: once to characterize the type of problem and the second time to describe how the problem was implemented in the classroom. The teacher could implement a making connections problem as a making connections problem, or the teacher could transform it into another type of problem—most commonly, a using procedures problem. For example, a teacher might transform a making connections problem designed to have students figure out a method for calculating the area of various types of triangles into a using procedures problem by giving students, at the outset, the formula (1/2 Base × Height) and telling students to simply plug in the relevant values.
Figure 2 shows how the teachers in the study actually implemented the making connections problems in the classroom: the percentage implemented as making connections and the percentage implemented as using procedures. Unlike Figure 1, this analysis reveals a pattern in which the highest-achieving countries resemble one another. Hong Kong and Japan, the two countries that differed most in the percentage of making connections problems presented, show a new similarity. In both countries, the majority of making connections problems are implemented as making connections problems; a much smaller percentage are transformed into lower-level using procedures problems. Here is the most striking finding of all: In the United States, teachers implemented none of the making connections problems in the way in which they were intended. Instead, the U.S. teachers turned most of the problems into procedural exercises or just supplied students with the answers to the problems.
Figure 2. How Teachers Implemented Making Connections Math Problems
The debates over mathematics education in the United States often pit two views against each other. One group believes that U.S. classrooms do not focus enough on concepts and understanding. The other group believes that U.S. classrooms overemphasize concepts at the expense of basic skills, thus holding back student achievement (Loveless, 2003).
Our research indicates that the lower achievement of U.S. students cannot be explained by an overemphasis on concepts and understanding. In fact, U.S. 8th graders spend most of their time in mathematics classrooms practicing procedures. They rarely spend time engaged in the serious study of mathematical concepts.
Improving Teaching
On the basis of this brief tour of the TIMSS videotape studies, three broad ideas can inform our efforts to improve classroom teaching of mathematics in the United States.
Focus on the Details of Teaching, Not Teachers
Most current efforts to improve the quality of teaching focus on the teacher: how the profession can recruit more qualified teachers and how we can remedy deficiencies in the knowledge of current teachers. The focus on teachers has some merit, of course, but we believe that a focus on the improvement of teaching—the methods that teachers use in the classroom—will yield greater returns.
The TIMSS video studies reveal that teaching is cultural; most teachers within a culture use similar methods. Indeed, within our study, teachers with strong mathematical knowledge showed the same cultural patterns of teaching as teachers with weaker knowledge. We must find a way to improve the standard operating procedures in U.S. mathematics classrooms—to make incremental and continuous improvements in the quality of the instruction that most students experience.
A focus on teaching must avoid the temptation to consider only the superficial aspects of teaching: the organization, tools, curriculum content, and textbooks. The cultural activity of teaching—the ways in which the teacher and students interact about the subject—can be more powerful than the curriculum materials that teachers use. As Figure 2 shows, even when the curriculum includes potentially rich problems, U.S. teachers use their traditional cultural teaching routines to transform the problems and reduce their instructional potential. We must find a way to change not just individual teachers, but the culture of teaching itself.
Become Aware of Cultural Routines
We can only change teaching by using methods known to change culture. Primary among these methods is the analysis of practice, which brings cultural routines to awareness so that teachers can consciously evaluate and improve them. A recent study by Hill and Ball (in press) of a large-scale professional development program found that analysis of classroom practice was one of three factors predicting growth of teachers' content knowledge.
Analysis of classroom practice plays several important roles. It gives teachers the opportunity to analyze how teaching affects learning and to examine closely those cases in which learning does not occur. It also gives teachers the skills they need to integrate new ideas into their own practice. For example, by analyzing videotaped examples of other teachers implementing making connections problems, teachers can identify the techniques used to implement such problems, as well as the way in which teachers embed these techniques within the flow of a lesson.
Attempts to implement reform without analysis of practice are not likely to succeed.
Build a Knowledge Base for the Teaching Profession
Finally, educators must find a way to inject new knowledge into the system of improvement and to share that knowledge with future generations of teachers (Hiebert, Gallimore, & Stigler, 2002). As John Dewey pointed out long ago, one of the saddest things about U.S. education is that the wisdom of our most successful teachers is lost to the profession when they retire.
What kind of knowledge do teachers need? They need theories, empirical research, and alternative images of what implementation looks like. U.S. teachers who want to improve their implementation of making connections problems, for example, will run up against a formidable challenge: They might never have seen what it looks like to implement these problems effectively.
Teachers need access to examples, such as those collected in the TIMSS video studies. They need to decide how they can integrate these examples into their own practice. They need to analyze what happens when they try something new in their own teaching: Does it help students achieve the learning goals? Finally, they need to record what they are learning and share that knowledge with their colleagues.
Teachers have a central role to play in building a useful knowledge base for the profession. Enabling teachers to learn about teaching practices in other countries and to reflect on the implications of those practices holds great promise for improving the mathematics instruction provided to all students.
