Question: I am the research and development coordinator for the Extramural Education Project, a nongovernmental group working in township schools around Cape Town in South Africa. Our project provides disadvantaged learners with opportunities to engage in extramural activities, such as arts, crafts, and sports. Ultimately, we hope to transform schools into community hubs around the nation. Do you have any advice regarding how we should evaluate the effectiveness of our efforts?
—Sarah Bliss, Cape Town, South Africa
Answer: Be sure to focus your evaluative efforts, Sarah, on what your extramural program can realistically accomplish. The extramural activities you provide may indeed benefit the disadvantaged students you serve. And the program may be exciting, especially to those of you who are dishing out a spiffy set of activities for kids to do. But don't become so entranced with the program that you think these extramural activities will have an immediate and profound effect on students' academic achievement. To make a meaningful dent in students' academic progress requires a potent, highly focused instructional intervention. In time, your program may well produce payoffs in students' academic achievements. However, during the early stages of your evaluative efforts, I encourage you to concentrate on measuring the program's impact on your students' affect.
You're dealing with disadvantaged learners, and these are often the very students who most need to feel good about themselves and schooling. If your program could focus on promoting more appropriate attitudes (such as heightened self-esteem) and interests (such as a positive disposition toward school) among the students with whom you work, this would be wonderful. By gathering evidence from students using anonymously completed self-report inventories, it is possible to arrive at reasonably accurate inferences about the affective dispositions of agroup of students. If you and your colleagues are interested in learning more about affective assessment, hop aboard the Internet and troll for information dealing with the measurement of attitudes or interests.
I realize that your long-term goal is to make schools "community hubs" in South Africa. In a decade or so, you might see whether that's happening. But in the early days of a program like yours, focus on the attainable. In your case, this would be those affective consequences likely to flow from a well-conceived extramural program.
Question: I recently estimated that all the tests that the state, district, and school oblige me to administer eat up a full month's worth of teaching time. Does all this testing really improve student achievement?
—Lorraine Schmitt-Pugaczewski, Christa McAuliffe Middle School, Elizabeth, New Jersey
Answer: Your concern about lost instructional time is significant, Lorraine. When unnecessary testing takes time away from teaching, we shortchange students. But what is "unnecessary" testing? If assessments are properly conceptualized, they can supply the evidence that teachers need to make better instructional decisions. Research has shown thatinstructionally oriented classroom assessments can help students learn. To know which tests are instructional winners or losers, however, you'll probably need to beef up your own assessment literacy so you can advocate the adoption of more instructionally useful tests and the elimination of instructionally useless ones.
Unfortunately, even if you use instructionally oriented tests, students' in-class achievement gains may not show up on your state's accountability tests. Most states have adopted tests that chiefly measure the composition of a school's student body rather than the effectiveness of the teaching that takes place in the school. When accountability tests are instructionally insensitive, the chances of even spectacular instruction boosting students' test scores are minimal.
Question: In our school, we typically employ multiple measures to assess a student suspected of having a learning disability. However, we persist in evaluating our students in general education, as well as our teachers, using only one or two traditional types of assessments. The complexities of such evaluations require diverse assessments. Why aren't we training teachers and administrators in multidimensional appraisal techniques?
—Don McMahon, The John F. Kennedy American School, Queretaro, Mexico
Answer: There are three hefty impediments to providing such training. The first concerns our quest for simple solutions. Faced with a multivariate world, many of us—including me—would prefer uncomplicated ways of dealing with that world. In the field of education, this tilt toward the simple is confounded by a second impediment: educators' astonishing lack of knowledge regarding education assessment. When people don't understand something, they often defer to those who do. That's what frequently happens when people are willing to base all their evaluative conclusions on the results of a single assessment. The perception that our education tests are precise because they yield numerical scores is fundamentally mistaken.
In your communication, you indicated that you typically employ multiple measures to assess a student suspected of having a learning disability. Clearly, this is the proper way to play the appraisal game. This brings us to the third impediment: resources. Unfortunately, going to the trouble of using multiple data sources takes both time and money.
