When I first got involved in the Higher Order Thinking Skills (HOTS) program, I did not start out to develop a network or even a national program. Rather, I had been invited by a school to develop a Chapter 1 intervention around some ideas I had proposed in a speech. At the time I had no background in working with Chapter 1 students or in developing thinking skills. I remember sitting around a table with a blank sheet of paper, and everyone looking at me and asking what they should do when the school opened in September.
That was 14 years ago. Today, approximately 2,000 schools in 49 states are using HOTS as their Chapter 1 and/or LD programs in grades 4–7. This makes HOTS one of the most successful reform networks developed to date. This has all been done without a sales force. Why has this nontraditional program and network been so successful? In retrospect, the humbleness and uncertainty of the beginning became a strength. Not knowing what “we were supposed to do” let us design an entirely new approach.
Over the years, we developed and refined a sophisticated and creative HOTS curriculum, along with a model of Socratic teaching and teacher training. While the literature is replete with the promised benefits of new paradigms, the biggest payoffs in education, as well as in business and life, come from the ongoing refinement of powerful techniques.
The Benefits of a Network
As the number of HOTS sites grew, it seemed natural to form a network to exchange information and ideas. The network makes possible a more realistic type of research than is possible in the highly controlled (contrived) settings of limited scope and limited duration, in which most educational research takes place. Spontaneous feedback from hundreds of teachers around the country ensures a continuous flow of valuable data. Such information flows, a normal part of the HOTS network, would be tremendously expensive under normal conditions.
Over time, patterns emerged in our data that lead to precise conclusions about the conditions and reasons for HOTS' success. I call this type of research around large communications flows “pattern sense making.” Our conclusions are then transmitted back to the teachers through a newsletter and incorporated into updates of the HOTS curriculum and teacher training techniques.
Even though the primary motive behind the research is to make HOTS more effective, pattern sense making has generated fundamental new knowledge about the nature of the learning needs of educationally disadvantaged students. In addition, this approach to research has generated very different conclusions from those of conventional research—conclusions that I believe are more valid and valuable for making national and school policy than those generated from either the prevalent quantitative or qualitative research techniques.
Why the HOTS Network Is Successful
The primary reason for the large-scale success of HOTS is that it works consistently for a wide range of students and circumstances. That success has crossed regions and cultures. The fundamental concepts that make HOTS an enduring reform are probably critical for making any reform work with kids on a large scale. Some generalizable lessons are as follows.
For a reform to work on a large scale over an extended period of time, it must have a state-of-the-art curriculum. If an intervention does not have a sophisticated and creative curriculum, it does not provide anything that teachers cannot develop on their own. In my research, I've found very little in the way of exemplary curriculums for the middle school grades, despite more than 35 years of reform rhetoric (Pogrow 1993). Most of the available curriculums are simplistic variations on a theme, designed to be consistent with the marketing techniques of publishing houses.
New forms of curricular materials are the fuel of effective reform. One advantage of a network is that it provides an alternative mechanism for disseminating curriculums. Unfortunately, most reform networks either do not provide curriculums or use prosaic materials. The rhetoric of excuses for not developing state-of-the-art curriculums are: (1) it is undemocratic to impose a curriculum, and (2) a curriculum is an individual affair that must represent local needs. Ultimately, however, networks that focus primarily on disseminating rhetoric and enthusiasm for change get bogged down when the enthused have to build the substance of the change on their own, or when the simplistic curriculums do not perform.
Most advocates of reform consistently underestimate the importance of new forms of curriculums, or the difficulty in generating them, largely because they have never tried to develop them. It is easy to physically move students from one part of a building or classroom to another to implement cooperative learning, inclusion, and schoolwide approaches—but where is the new curriculum to make them work?
In addition to a state-of-the-art curriculum, a successful reform needs an effective pedagogical approach and intensive training for teachers in these pedagogical techniques. Good curriculums offer the potential for new types of interactions between students and teachers. This potential is seldom realized on a consistent basis. I often see whole-language teachers, for example, teaching literature using the same approach they used with basals, or teachers presenting hands-on science activities using direct instruction techniques. The results from mixing new techniques with old pedagogy are usually awful.
The keys to developing better pedagogical techniques are to have: (1) a clear set of appropriate teaching practices, and (2) state-of-the-art training in these techniques. Teacher training must emphasize the development of new talking and listening reflexes around the use of the curricular materials, as opposed to training in the philosophy and theories of teaching, or how to use the materials. The teaching techniques and training must develop skills in talking and listening to students in new ways.
To be successful, a reform must also focus on those grade levels and students who can benefit from it. Contrary to popular myth, the most effective interventions are those that specialize in particular grade levels and types of students—as opposed to those that pretend to be for everyone. This is especially true for interventions designed for educationally disadvantaged students. For example, Reading Recovery is effective with 1st graders, and is so specialized that its benefits carry over for several grades thereafter.
Contrast this with an unfocused reform such as the use of manipulatives in math. Because they seem to be effective in the early primary grades, manipulatives are now being advocated for all grade levels. As a result, traditional manipulatives have become almost as boring for older students as textbooks.
Most reforms used after the 3rd grade are based largely on success in the earlier grades and when used inclusively simply do not work. In addition, while whole school reform remains an ideal, and has been attempted for at least 100 years, it has never been accomplished successfully in more than a few schools in this country.
Recommending that complex reforms be highly focused is at odds with the current rhetoric, which advocates all-inclusive approaches; that is, lump all students together, treat everyone the same, get everyone learning cooperatively, do things schoolwide, and so on. All of our research and experience, however, suggest that educationally disadvantaged students are even more heterogeneous than previously suspected and require very different interventions for at least part of every day for one to two years.
For example, our research suggests that there are three very different learning problems and needs even within the Chapter 1 population. From 60 to 80 percent of the students suffer from metacognition deficits, about 5 to 10 percent have undiagnosed severe dyslexia, and the rest are borderline mentally handicapped. I am as idealistic about achieving inclusion as anyone. By overcoming students' metacognition problems, HOTS provides the specialized help that can enable the majority of Chapter 1 and LD students to not only be physically included but, more important, to be academically successful in heterogeneous settings. Such “academic inclusion” should be the real goal.
Unfortunately, I have no idea how to meet the need of severe dyslexics or of the borderline mentally handicapped, so I have to wonder whether it is possible for any teacher to deal effectively with all three Chapter 1/LD learning problems—not to mention all the other students' special needs. Inclusive rhetoric about how it is wrong to label students, how “all kids can learn,” or how kids do not have deficits do not make these special problems and needs go away.
The results from our pattern sense making research suggest that those who argue that all students should be treated the same, and those who argue that all have different needs, are both wrong. It appears that we can develop specialized interventions and make rational decisions about placement for a clearly identifiable, manageable number of students with special learning needs. This is very good news for designing practical large-scale reforms.
At the same time, the specific nature of the different learning problems suggests that there is no such thing as an intervention or curriculum that is effective across the board, any more than miracle drugs work under all conditions. Even highly effective programs are beneficial only under a limited set of conditions. Ignore those conditions, and the programs fail to provide substantial help. For example, we know that HOTS does not work with borderline educationally mentally handicapped or severely dyslexic students, but the program does work with metacognition-deficient students, regardless of whether they are in Chapter 1 or LD programs. Such a focused, knowledge-based “truth in disseminating” approach is the only way for reforms to work on a large scale with students from different backgrounds.
In addition, successful reforms maintain high levels of quality control every step of the way. Weak teachers, weak trainers, and inadequate materials and communication processes can quickly render even the best program ineffective. Maintaining and improving quality in all the details has to be a mindset.
HOTS has been blessed with wonderful teachers, trainers, and staff. HOTS also invests in a great deal of up-front communication with principals, coordinators, and teachers about the conditions that must be followed if the program is to succeed. To be eligible to become a HOTS site, schools must first sign an agreement as to how they will implement the program. While such communication dissuades about two-thirds of those interested in the program from using it, such communication also ensures a high success rate among those who choose to do it.
Finally, to work on a large scale, a reform has to be structured and detailed—and also highly creative. Reforms such as career education or whole language, which are so amorphous as to be officially undefined, often run out of steam or survive in a form that has little resemblance to the original conception. HOTS combines a high level of creativity with a high level of specificity—the same strategy successfully used by the performing arts.
Why Aren't There More Effective Reform Networks?
The experience of the HOTS network has generated the type of knowledge needed to help educationally disadvantaged students in a scientific manner. We have been able to define more precisely what the real learning problems are, and how to design and focus needed services.
Unfortunately, effective large-scale reform networks are rare. The only two networks I know that practice the principles of reform effectiveness just outlined are HOTS and Reading Recovery. The new version of the Junior Great Books comes very close, however, and is making the types of changes needed to make it a focused intervention.
Why aren't there more successful networks? First, it takes a lot of hard work and luck. Second, the reform climate isn't right. Ultimately, the existence of reform networks rests not only on the effectiveness of the interventions, but also on the attitudes of practitioners and reform advocates. Most reforms today are based on good intentions, idealistic philosophy, and tons of advocacy. The focus in the '80s was on increasing students' self-concepts, and the simplistic stratagem in the '90s is empowerment. To maximize buy-in, reformers now suggest that the best approach is to encourage everyone to do their own thing.
Such conceptions of reform, however, are mostly fantasy. The fact that a reform such as empowerment works in a few places does not mean that it can work in many or most places. The reality is that successfully helping most children to learn and to develop a sense of understanding is a very complex process; it requires sophisticated and specialized interventions with well-designed curriculums in the hands of good, well-trained teachers. Increasing learning on a consistent basis requires new tools developed with a blend of hard work and creativity.
In addition, the primary instinct of most teachers is to help their students, not to develop their own materials and tools. Most teachers prefer to use existing programs instead of developing their own if: (1) they are allowed to decide whether to buy in to the program, and (2) the program helps children learn more powerfully.
Anticipated changes in federal legislation and current conceptions of political correctness threaten the existence of even successful networks. The notion of a highly focused intervention is politically incorrect at the present time. Much like in the early '70s, we are at the apex of a historical cycle where reformers are hell-bent to change whole schools, and anyone trying to do something else is considered reactionary. In addition, the drive for democratic participation at any cost makes organized programs highly suspect.
The formation and success of substantive reform networks require that the profession move away from generating reform primarily through philosophy and advocacy to an orientation of producing far more powerful, creative tools and using them more precisely. For example, instead of saying “all kids can learn,” HOTS specifies the conditions under which educationally disadvantaged students will learn. This reorientation of reform perspective requires very different policies coming out of the profession and Washington from those now coming down the pike.
It's somewhat ironic that when we started HOTS 14 years ago, most of the criticism came from conservatives who were suspicious of anything that was not basic skills. Now the criticism is coming from progressives who reject focused approaches—regardless of whether they help kids. Our network will be ready four to five years from now when the field becomes disenchanted with the current wave of simplistic reforms. Indeed, that is how this network got started in the first place—as an alternative to the failed reforms of the time.
So You Want to Be a Reformer?
- Ignore conventional university wisdom, and proceed in a scientific way with your own hunches and observations. Trust what you see happening with kids and teachers.
- Remind yourself that you do not have expertise until you are able to make something actually work with teachers and students on a reasonable scale. This perspective keeps the focus on producing student learning as opposed to supporting one's ego or proving a pet theory or philosophy. Also, when something does not work, don't blame the test. Instead, make the curriculum and training better.
- Redefine your role to balance expertise and entrepreneurism with integrity. One of the things that I am proudest of is that whenever financial gain conflicted with the data about the best way to implement the program, we made all decisions on the basis of the data regardless of whether that meant there would be fewer sites and less revenue. This is something that conventional entrepreneurship cannot achieve.
What's Next?
Our research can serve as the basis of substantial reform. To achieve academic inclusion for the 60 to 80 percent of those Chapter 1 and LD students who have poor metacognition skills requires separation for a period and a two-stage process over three to four years. First, the students need to be taken aside for 35 minutes a day for two years to participate in a general thinking development program such as HOTS. Then, in the second or third year of the sequence, these students should be placed in an exemplary, problem solving-oriented course one period a day for two years in a heterogeneous setting. This course could be in any content area. Some possibilities are the use of Junior Great Books, our new SUPERMATH program (Pogrow 1994), or any other process approach to content learning—with a terrific teacher. The process could start as early as the 4th grade and as late as grade 6 or 7.
I'm interested in starting a new network with educators who are willing to select a group of metacognition-deficient Chapter 1/LD students and make the commitment to keep them together and provide the needed focused services over three to four years. Any takers?
Is all the work involved with building a network worth it? I think so. Professors who satisfy themselves with generating ideas—and not going the next step of trying to make their ideas work in the real world—are missing out on one of the great experiences in education. There is nothing more satisfying than watching a good teacher transform a group of students over one to two years and knowing that your ideas helped make it possible.
