The driverless 1964 Impala convertible cruises down a sunny highway. A man and woman—he with a carefree smile, she holding onto her hat with one hand—float down from the sky into the empty front seat, while the Hi-Los croon, "Let Hertz put you in the driver's seat."
I'm reminded of that iconic commercial every time I hear about the need for schools to be driven by data. I imagine a number 2 pencil and a bubble sheet coming down from the "assessment clouds" to drive curriculum and instruction.
Let me confess right off, I don't want to be driven by data! Any data set is a welcome passenger in the back seat for an occasional route confirmation. But I'll have love of learning or a passion for excellence in my driver's seat. Data can serve those drivers only when (1) the data measure something more significant and enduring than discrete facts and skills, (2) they take into account the whole child, and (3) students own the process of collecting and analyzing data and set their own goals.
I'll offer examples of how data analysis can serve a passion for learning. But first, let me explain why I don't trust it in the driver's seat. I don't doubt data's effectiveness in improving performance. Increasing student achievement and closing the achievement gap are crucial goals for schools. The act of collecting and analyzing data has the potential to play a significant role in helping teachers plan lessons and design projects that target specific student needs. I've seen data's efficacy in holding our intuition accountable and helping set goals for improvement. But I've also seen it unintentionally obscure the broader school vision and drive essential dimensions of school life to the periphery.
Too often the complex terrain of student achievement is reduced to the set of indicators that is easiest to assess with standardized tools. As such indicators focus our attention on easily quantifiable skills, they can cause us to lose sight of the forest for the trees, to focus on a narrow band of what it means to be human. Because data analysis is so effective in addressing the mastery of discrete skills, the process tends to see everything as a discrete skill. It assumes that learning is a linear process, skill by skill, bit by bit, starting in kindergarten at A and ending with a PhD at Z.
Data analysis might play the role of a servant who helps us help students learn when we begin the journey of educating them. But once aboard, it likes to become a master, taking over other dimensions of education's purpose. I'm reminded of a cartoon where a knight confidently encounters what seems to be a dragon not much bigger than himself—but is really a puppet held by a huge dragon hiding behind a cliff.
The problem isn't in data itself, or even in how we use data—it's in what we decide to measure. As Mike Schmoker wrote, "our current data-driven decision making is to a great degree standardized-test-data-driven decision making." How can we invite data into schools without letting data usurp the fullness of our humanity?
What Can We Measure?
What if we could reliably measure characteristics and competencies beyond the realm of content and skills—intangible qualities like beautiful work, or traits like perseverance? I like to ask students, "What can you measure?" They usually think of using a ruler at first, measuring length or height. Then maybe surface area, volume, or speed. Some wonder, How about brightness? Or hardness?
Reminding students that measurement is a key to scientific discovery, I challenge them to think of more things they would like to evaluate and design scales to measure them. Could you measure the wind? (Consider the Beaufort scale.) Or how loud it is in the cafeteria? How about time, without a watch? Yes, we could use the sun, but could you use water, sand, or the wind? Why might the sun—but not the wind—work to measure time?
Then I ask, "Is there anything we can't measure?"
Students say things like, "the pride of accomplishing something," or "how much my mother loves me."
"Really?" I sometimes challenge. "We can't measure those?"
People can design surprising tools to measure even complex emotions. I helped Jenna Gampel's class at Conservatory Lab Charter School in Boston with a project about snakes. These 2nd graders' goal was to convince people not to be afraid of snakes. Jenna asked, "How will you know if you succeed?" and discussed the challenge of measuring how afraid someone is.
We asked each of the kids to describe his or her own level of fear of snakes. They crafted statements to describe various relationships to snakes, then voted for the final descriptors to use in their survey that assessed how afraid people were before and after they taught them why they shouldn't fear snakes (see fig. 1). You can see a video of Jenna's class in action.
Figure 1. Fear Scale
Once students leave school, they'll rarely be measured by how well they score on a test. They'll be judged on their character and the quality of their work. So wouldn't it be great if there were a school where educators collected data on these things? There is not only a school, but a national network, Expeditionary Learning, that expects students and schools to show evidence of their growth in terms of character and work quality. Here's where I might invite data into the front seat.
An Expanded View
Expeditionary Learning schools pursue an expanded view of student success that takes into account three distinct dimensions of achievement: mastery of knowledge, student character and engagement, and quality. They achieve exemplary results in each dimension and build systems that connect all three.
Mastery of Knowledge and Skills
This dimension of achievement reflects the traditional understanding of the purpose of school. Students demonstrate understanding, proficiency, and application of a body of knowledge and skills within each discipline by solving problems, thinking critically, applying their learning to novel tasks, and communicating clearly about complex ideas. Our schools expect all their students to do high-level thinking and work daily.
One unique thing about how these schools use data to improve achievement is how students own the process of collecting data, analyzing data, and setting their own goals. In English language arts classes, students keep records of their errors in reading and writing, analyze patterns, and set goals for improvement. Similarly, in math, they track the kinds of errors they make on tests—computational mistakes, procedural errors, or errors resulting from conceptual misunderstandings. They use the data they collect to report on their progress during student-led conferences. Assessment isn't something done "to" students, but something they use to improve and demonstrate their own performance.
Character and Engagement
This dimension relates to noncognitive skills supporting student success. Performance character includes the academic mindsets and habits of scholarship that students bring to learning (such as perseverance and organization). Relational character reflects how students work with others (for instance, whether they show respect and collaborate). Both aspects of character are essential and interconnected.
Students in Expeditionary Learning schools regularly reflect on their character, receive feedback from peers and teachers, and discuss the qualities necessary for success in school and life. They might explore together what it looks, sounds, and feels like when we show integrity or compassion. Just as Jenna's class created an instrument to measure fear, students design learning targets and rubrics to monitor their progress and report on their growth and challenges.
I recently met with 6th graders from the Genesee Community Charter School in Rochester, New York, an Expeditionary Learning mentor school. These students set goals, collect data on how they're meeting those goals, and reflect on progress weekly. Everyone calculates the percentage of homework assignments completed, both as individuals and as the aggregate class. Students set such whole-class targets as "I can use systems to organize materials" and "I can make a plan to work through something that's challenging for me."
Each student tracks his or her progress on specific character goals. Ceanna wanted to get better at asking for help, so she recorded every time she asked peers or teachers for help. Madison wanted to let others have a chance to lead in small-group work. After each group session, she rated herself on a 1–5 scale on how well she'd yielded the floor to others and got feedback from her peers.
At World of Inquiry School #58, an Expeditionary Learning mentor school in Rochester, New York, teachers saw test scores rise to district-leading levels even as poverty rates increased. Like many schools, near the beginning of the year, this school gives students a pretest of released items from previous state standardized tests and uses the results to determine what skills teachers should emphasize. But this school does more: While students take the pretest, teachers watch them closely for behaviors that might have as much bearing on their test scores as knowledge and skill—rushing, getting anxious, misrecording answers, and other unhelpful behaviors. They help students recognize those behaviors and set goals for improvement. Students record their goals and monitor their own improvement as they prepare for the real test.
At weekly grade-level data meetings, classroom teachers, specialists, and administrators go through the list of students, consider each student's profile, and design a plan to address that learner's individual needs. A reflective but shy child who scored poorly on the English language arts pretest would receive targeted help in mastering skills, but might also be invited to join the poetry club, where she might gain confidence with language. Another who had strong leadership skills but often used them in distracting ways might be asked to be the announcer at student-led community meetings. Often the issue is as much about getting students engaged in their own learning as it is about developing specific skills.
High-Quality Work
This dimension measures how expertly students transfer disciplinary knowledge and skills to authentic contexts. Can students use their knowledge to solve complex problems and create a body of work that shows craftsmanship, deep thinking, and creativity? To give evidence of competence, students in this network of schools present their work to diverse audiences and communicate their thinking about it through writing and speaking.
Expeditionary Learning is distinguished by an explicit focus on high-quality student work. One likely reason other schools avoid this topic is that it's messy; quality in student products cannot be easily defined and quantified. For years, we tried to identify criteria, but they were elusive and dependent on genre or format. Quality in a poem is different from quality in a lab report; an excellent museum exhibit differs from an excellent public service announcement. Ron Berger, chief academic officer for Expeditionary Learning, and Steve Seidel, director of the arts in education program at Harvard Graduate School of Education, have worked with colleagues and schools for 25 years to collect, analyze, and archive exemplary student work and then use that work to improve teaching and learning (see http://centerforstudentwork.elschools.org). They articulated three attributes of high-quality work:
- Complexity. When educators examine student work, they look for examples of higher-order thinking connected to big concepts that undergird or unite academic disciplines. They should see evidence that learners can transfer understanding to new contexts and see an issue from multiple perspectives. Complex work demands skill with complex texts and with evidence-based writing and speaking.
- Craftsmanship. Well-crafted work is done with care and precision. Educators analyze attention to accuracy, detail, and beauty in conception and execution.
- Authenticity. Authentic work demonstrates original thinking. It uses formats, standards, and sometimes audiences from the professional world, rather than school formats—and connects academic standards with real-world issues. Authentic work matters to students.
Our schools use these criteria to collect data about student work and inspire deep conversations among faculty about what high-quality work looks like and how to help kids produce it. They regularly examine student work schoolwide and create a body of evidence that describes how it has improved over time.
For example, Genesee Charter School conducted a quality work protocol. They analyzed a year's worth of student products for complexity, craftsmanship, and authenticity. There was much to be proud of, but they identified areas in which to grow. One area was individual student accountability; educators questioned how they could be sure that every student mastered intended learning targets when some work was produced collaboratively. Another area for growth was creating products that have an authentic audience beyond the school community.
Education as a Thermostat
Our national institutions are increasingly obsessed with data. Neil Postman, one of my inspirations for becoming a teacher, offers a relevant model for a data-driven culture. He compares education's role in society to a thermostat. When it gets too hot, the thermostat clicks on cool air; when it gets too cold, it triggers warm air. Postman believes schools should provide a counterexample to the prevailing cultural tendencies of any given time:
Education tries to conserve tradition when the rest of the environment is innovative. Or it is innovative when the rest of the society is tradition-bound. … Where a culture is stressing … individuality, education should stress cooperation and social cohesion. Where a culture is stressing conformity, education should stress individuality.
When I began coaching teachers in the early 1990s, just before the impact of No Child Left Behind, I saw many teachers assign "projects" to engage students in learning. Students may have been engaged in building pyramids with sugar cubes, but the learning was questionable. At that time, our thermostatic role was to push teachers to articulate the intended learning and collect data to determine whether students achieved it.
Now I see students practicing discrete skills, taking tests, and disengaging. So teachers' role is to design relevant curriculum, engaging instruction, and structures that help students assess their own progress.
I welcome data into the front seat when it measures qualities that connect to these worthy goals. But I'll still have love for learning and passion for excellence take turns driving.
