From portfolios and mentoring to the National Board for Professional Teaching Standards, the eval- uation of educators has undergone many changes over the years. Lee Shulman, president of the Carnegie Foundation for the Advancement of Teaching, has been on the cutting edge of those changes—influencing, witnessing, and analyzing new developments.
Before we talk about meaningful measures with which to evaluate teachers, let's talk about what good teaching is. What is your vision of good teaching?
My vision of good teaching includes nurturing the moral and spiritual development, the civic engagement, and the socialization of students—a nurturing we usually associate with parents, clergy, social workers, librarians, all the folks who jointly accept the responsibility for both raising and protecting the young. But teachers, in particular, play a singular role in our society in nurturing students.
One of the qualities that make us fully human and distinguish us from other species is our capacity to invent and discover both knowledge and beauty and to pass our understanding on to successive generations. Every other species begins, in effect, at a Darwinian ground zero with respect to their predecessors. Offspring may be a little different from what their ancestors were 10 generations ago, but different for evolutionary reasons, not because the previous generation carefully nurtured, nourished, preserved, and transformed information and then passed it on. Teachers are the mechanism by which our societies pass on knowledge and values. Teachers, in that sense, are uniquely responsible for carrying on our cultural, intellectual, and aesthetic achievements.
That's not always easy to measure and evaluate.
Nor is it impossible. We're about to start a study at the Carnegie Foundation that will look at the preparation of clergy—rabbis, ministers, priests. If I had to choose between measuring the impact of teachers on their students or measuring the impact of clergy on their congregations, I'd say the former task was easier. It's all relative.
My conception of good teaching also relies on whether teachers have a deep and flexible understanding of what they are teaching. This emphasis on understanding stems from my deep commitment to the quality of ideas that teachers help students construct. When a teacher doesn't have a deep understanding and affinity for mathematics or science, for example, it's hard to imagine how that teacher will help students understand and get excited by these subjects.
As I said, teaching is about the ability of each generation to build on the previous generation's understandings and creations to sustain our humanity. So if teachers are not themselves lovers of wisdom, of beauty, and of the values that make these subjects worth pursuing, then I worry about their ability to be effective. That's why at the heart of my work on good teaching is the notion of a teacher as an enlightened, passionate intellectual.
Do you find that this notion goes against the trend these days to measure the success of students—and their teachers—by standardized test scores?
The confusion stems from valuing standards, on the one hand, and embodying those standards in high-stakes assessments, on the other. I'd like to see us disentangle these priorities. I'd like us to develop high standards that describe in compelling, vivid, and persuasive terms what we would like both our teachers and our students to know, to be able to do, to understand, and to enact—without then tying accountability for achieving those standards to high-stakes assessments. The assessments end up corrupting the value of the standards. The standards get modified to be consistent with what we're able to measure in a high-stakes assessment. Almost inevitably, we have to ratchet down the standards and squeeze out all of the creative diversity because we want to be able to develop scoring keys that nobody can complain about or challenge.
An example of a set of high stakes or high standards that is not tied to high-stakes assessments is the Ten Commandments. People of many denominations use those ten standards, if you will, as crucial benchmarks for their lives. Those standards don't derive their power because they have high-stakes tests associated with them. Their strength comes from their wide dissemination and discussion. They are subject to interpretation and explication, yet that hasn't weakened them. Thou shalt not kill—What does that statement say about self-defense? What does it say about just wars? The Ten Commandments are only a starting point, yet look how powerful they are.
As you know, a perennial debate in education involves the connection between evaluation and improvement. Some argue that evaluation needs to be treated separately to enhance teachers' knowledge and skills. Others think that given the current emphasis on measurable results, we must align evaluation with professional development and school improvement. How do you feel about making these distinctions? Do you think there will always be such tensions?
Let me answer your last question first: There will always be tensions. Somebody once asked me, Why are there tensions between theory and practice in the education of professionals? And my answer is that there have to be tensions; sometimes tensions are just a natural condition. In raising children, for example, there is always a tension between disciplining them and encouraging them to be spontaneous, playful, and outgoing. That's built into the job.
On the one hand, we're trying to discover ways to sustain and support the continued development of practitioners, which, as in the case of child-rearing, often requires that practitioners sometimes feel free from strict evaluation. They need to feel that they can step back, that they can make a mistake, and that they can take a chance. On the other hand, we want to protect the society, the community, and the students from the insufficient preparation or limited motivation of some practitioners.
This tension is built into the role of the educator. One of the ways we make the tension worse is to put both the job of the nurturer and the task of the monitor in the hands of the same person or agency. I will go back to my example about parents. One of the things that makes parenting really hard is that, as parents, we both nurture and evaluate. When evaluating teachers and tying rewards to those evaluations, we should try, within the limits of responsibility, to separate and delegate the twin functions of monitor and mentor to different individuals.
So a mentor should never have an evaluative role?
Well, never say never. If you were my mentor, and it was your job to help me learn to be a really accomplished teacher, you would have access—through observation and conversation—to views and perspectives of me that nobody else had. From that perspective, if you came to understand that I was a psychopath or that I had areas of deep ignorance, which I gleefully and cluelessly passed on to my students, there is a point at which it would be unethical for you to withhold that information from those who represented the accountability side of the equation.
But as a mentor, you would have to explain that to me. You would have to remind me that, 99 percent of the time, you are my buddy, my friend, my supporter. But there could be a case in which you observed me doing or not doing something that so violated the fundamental expectations of the role that you would have no recourse but to make that observation part of the more general evaluative data.
I'm reminded of the comparisons you have made between the teaching and medical professions. It's interesting that the trend now is for educators to look to the business world for models. What do you make of those connections, particularly as they relate to teacher evaluations?
Some of them are useful. The most corrupt comparison—the one that is most dangerous and, unfortunately, widespread—is to treat students' test scores as equivalent to the bottom line of a business's balance sheet. That practice reflects a fundamental misunderstanding of what test scores measure. It would be the equivalent of a business like Hewlett-Packard or Intel using an inadequate measure of productivity as a way of evaluating the performance of individuals in the firm who have only a modest opportunity to influence it.
But if we could identify benchmarks that teachers really have some control over and that are tied to what they teach, then I wouldn't be too unhappy with such measures as a basis for teacher evaluation and merit pay. Look at teachers of advanced placement courses. When an unusually large percentage of students in a given AP class get ratings of 4s and 5s on their AP biology or history tests, the teacher takes enormous pride in the accomplishment and is prepared to talk about her practice. Even though there may have been a lot of variation in the students who first entered the class, the students were measured by an external set of assessments, and the teacher taught consciously to that test. In these cases, the variations among scores are substantially teacher-related, and I don't see this use of data as a bad way to hold teachers accountable.
William Sanders's work, which has gained popularity and media attention, focuses on value-added assessment. Would you like to see more value-added assessment?
The underlying principle is an interesting, even compelling, one. Once we've controlled for such variations as the composition of the classroom, if teachers are consistently able, over a period of three or more years, to have a positive effect on student achievement, then these data mean something. I would be happier if the value-added assessments were more like advanced placement assessments, however. Standardized tests are designed to assess knowledge that is relatively independent of what the teachers are actually teaching. If Sanders had better assessments, then I would be able to take more seriously what it meant for a teacher to be consistently "adding value."
What about pay-for-performance incentives?
I oppose any system that uses its money to pay teachers for performance across the board instead of using money as an incentive to attract and retain our very best people in teaching. If that's the trade-off, then I don't support a reward system. I make a real distinction between incentives and rewards. Where we are really hurting now is in providing incentives. In some respects, that's what the National Board for Professional Teaching Standards (NBPTS) now provides.
The National Board is not really a system of merit pay but a set of incentives for good teachers to become better through improving their own competence and documenting it with their students' achievements. Then they are recognized for having done so. This incentive is a carrot that encourages them to do more and that helps keep them in the profession.
I know you had a lot to do with developing the National Board from the ground up. Do you see these incentives—not just money, but also improved professional recognition—as one of the major highlights of the NBPTS?
Yes. Teachers from around the country are coming together to prepare for National Board certification, certainly with the incentive of financial bonuses and salary increases, but also just for the chance to demonstrate to themselves and to others that they are really, really good at what they do. The way in which we see groups of teachers voluntarily preparing one another and themselves, and the extent to which their NBPTS work fits into their classrooms in the portfolio-making process, demonstrate the major impact of the National Board. I believe that the impact of certification will only get more powerful.
Have there been many challenges or even disappointments with the establishment of the National Board?
I've seen very few disappointments. When we began this work in the mid-1980s, we didn't dare dream that in the year 2001, the National Board would have the kind of national visibility, credibility, momentum, and support that it has. Of course, we couldn't have started this work without a dream, but I would not have staked my kids' college educations on its success.
Overall, I am exhilarated by the National Board and its extraordinary leadership. If I have any disappointments, they have more to do with the unavoidable trade-offs that must be made when idealism and budgets collide. We simply can't afford the kind of National Board assessment that I had originally conceived. It takes too much investment of time and money. But Mercedes-Benz and General Motors are filled with visionary engineers who dream up cars that nobody can really build because nobody can afford to buy them. Eventually, however, the engineers see a different version of what they first envisioned. And, if they are lucky, their dream car eventually gets on the road and is driven by lots of people.
That's how I feel about the National Board. The ideal vision of the portfolio was that it would be even more elaborate and display an even richer range of teacher performances and understandings than it does now. The original idea of the assessment center was a much longer and more thorough evaluation—not that it would be a harder test, because it is already a very hard test.
But in the real world, I'm grateful for what the Board was able to do. Let's face it, a lot of what we want to see happen in the preparation of teachers won't happen just because we test for it. By enunciating the standards to which we want all teachers to aspire, the NBPTS helps teacher education programs, colleges and universities, and alternate routes to teaching ratchet up their standards and their preparation programs.
In a sense, the National Board raises the bar for everybody?
Yes, even though we don't make all of them jump over the bar. To use a negative example, it's a bit like knowing that police with radars are patrolling a stretch of interstate highway. Even though only a tiny percentage of the drivers ever get stopped, most of the drivers change their driving behavior because they know that standard police enforcement is there. Putting on seatbelts is another example. Most of us no longer just put on seatbelts because we're afraid that a cop is going to pull us over and give us a ticket. We now put on seatbelts because we have learned—we've become convinced—that we and the rest of the people driving on the road will be safer.
I see the National Board standards as having that kind of influence. Over the years, what happens is that those standards, and the ubiquity of the assessments that go along with them, will have an impact on other teacher- preparation programs and professional development. Once Board-certified teachers turn around and mentor other teachers, the influence will transcend the actual assessments. Already, colleges are redesigning their master's programs for teachers around National Board certification standards.
We talk a lot about the evaluation of teachers, but not so much about the evaluation of principals and administrators. Do you know of any moves afoot to revamp the evaluation of those educators?
Again, we have to distinguish among the broad kinds of evaluation: evaluation for purposes of selection, certification, and appointment; evaluation for judging the quality of current performance for purposes of review, promotion, and firing; and evaluation for self-monitoring purposes. A profession evaluates its own to assume the responsibility for the integrity of the practice. In some sense, this is what bar associations do for the law profession and what the National Board is beginning to do for educators.
It used to be that nobody became a superintendent unless he or she had been a principal, and before that, a teacher, and before that, certified to teach. Now, we're hiring a superintendent who used to be the governor of Colorado or a retired military leader or an attorney general. What this trend tells me is that it may be unwise to insist on treating the superintendency in the classic sense; the superintendency is becoming less a professional role and more a political role. Elected or appointed boards of laypeople may then be responsible for evaluating superintendents, using various kinds of accountability.
With regard to principals, my own view is that the principal is the lead teacher. The word principal is not only a noun, but also an adjective, the first word of a two-word title, principal teacher. Principals, therefore, should be highly accomplished teachers who have the capacity to manage, organize, lead, and develop the capabilities of other teachers. That concept ought to characterize both programs of preparation and programs of assessment.
One of my dreams is that a day will come when no one will be considered for a principalship who is not first a National Board–certified teacher. That could happen. It would certainly shut down several routes to the principalship whereby principals may never have been in a classroom.
I was rereading an EL conversation that you had with Ron Brandt in the early 1990s ("On Research on Teaching: A Conversation with Lee Shulman," April 1992). I expected that the piece would feel at least mildly outdated, but the issues that you talked about—such as instructional leadership, research on effective teaching, and content-specific pedagogy—still seem so relevant today. Do you find that encouraging or depressing?
I find that encouraging. I remember that Ron Brandt and I had that discussion publicly at a plenary session of an ASCD conference held in San Francisco. We were raising and emphasizing questions that weren't yet part of the daily discourse of the education profession. Most educators were still strongly oriented toward thinking of teaching in a nondisciplinary, generic way.
Not only are we still talking about many of these topics, but today we are also talking about them in public and as part of the mainstream, not as a —al view. What's fascinating is that when the National Board went into action, people raised all kinds of questions about why we would want to design more than two dozen different boards. They argued that teaching is teaching is teaching. You never hear that anymore. Now, people take for granted that teaching is discipline-specific, age-specific, and, to some extent, context-specific.
When we started out, most people found the notion that teachers should begin with a teaching portfolio rather than with a decontextualized set of test scores to be a loony idea. Now, if there's anything I worry about, it's that everybody is using portfolios, and teachers can throw any stuff into a box and call it a teaching portfolio. I don't want the idea to be cheapened. I want people to continue to take teaching portfolios seriously as an instrument, not as a ceremony.
But I am very encouraged that almost a decade later, we're still talking about these issues. Let's face it: So many things that we talk about with great enthusiasm in one decade are gone and forgotten by the next decade. This is, unfortunately, the fate of many trends in education. The fact that these ideas have moved from the warming oven to the front burner is actually very good news.
