A few years ago, if you had asked the 3rd, 4th, and 5th grade teachers at Mason Elementary School in Boston about their students' written responses to literature, they would have told you proudly that their students wrote about books all the time. In fact, it was common practice to have students write reading-response letters to the teachers reflecting on books they were reading independently.
Imagine the teachers' surprise, then, when the annual state test revealed that their students' core weakness lay in writing about what they read. "Our kids should do really well on this part of the test," mused data coordinator Hilary Shea.
When Hilary and her colleagues brought their students' reading-response letters to a staff meeting and looked at the letters as a team, they began to uncover the roots of the problem. Sure, students were writing about books, but many of the letters offered brief summaries instead of reflection. And teachers noticed that many of the 5th graders' letters were no more sophisticated than those of the 3rd and 4th graders.
As the teachers discussed the letters, they came to another troubling realization: They didn't agree among themselves about the traits of a strong reading-response letter. Some teachers prioritized accurate writing mechanics; others believed that mechanics were unimportant as long as a student demonstrated comprehension. Still others insisted that comprehension was not enough; students needed to draw inferences or make explicit connections to texts or their own lives.
It dawned on Mason's teachers that the reason students struggled to write about texts was that they were receiving mixed, vague signals about how to do so. In subsequent meetings, the teachers developed a rubric for teaching and evaluating reading-response letters at each grade level and set concrete goals for improved student performance.
Becoming Data Wise
Mason Elementary School's experience highlights the value of using data collaboratively to drive instructional change. Analyzing data from the state assessment, teachers discovered a troubling area of student underperformance. Instead of stopping at the state test results, they sought to understand the problem. By digging further into the data and examining student work, they identified an explanation. Then they developed a collective solution.
Mason's teachers undertook these steps as part of the Data Wise improvement process, an approach to schoolwide instructional improvement developed by a team of educators in the Boston Public Schools and researchers at the Harvard Graduate School of Education (Boudett, City, & Murnane, 2005). Mason is one of eight schools around the United States (all of which are public schools and four of which serve primarily low-income students) that we profiled with a group of our colleagues in Data Wise in Action (Boudett & Steele, 2007).
One theme cut across all eight schools as they worked on school improvement—they all used data collaboratively. The collaborative approach to data use yielded at least three major benefits for these schools: organizational learning, improved internal accountability, and a safety net for professional growth.
Organizational Learning
David Garvin (1993), a leading organizational scholar, defined a learning organization as one "skilled at creating, acquiring, and transferring knowledge, and at modifying its behavior to reflect new knowledge and insights" (p. 80). At Mason Elementary, teachers' collaborative analyses of student work contributed to organizational learning by yielding new insights about student understanding. Had teachers not taken time to examine classroom data collaboratively, it's unlikely that they would have found a shared instructional solution, and their students' ability to write about texts might have continued to languish. To improve teaching and learning at Mason, teacher collaboration was not just a luxury—it was essential.
In other schools, the data source that fostered organizational learning was classroom observations. At Murphy K–8 School in Boston, teams of teachers regularly observed one another's teaching, provided structured feedback, adapted the lessons they'd observed to their own classrooms, and reported back to their teams about how the lessons went. This routine helped teachers build a collective instructional repertoire and gave them tremendous insight into schoolwide instructional practices. New teachers may have initially been intimidated by the culture of shared practice, but they quickly found that Murphy was a place where, in the words of teacher Holly Concannon, "even the newest team member has something to contribute to the effectiveness of the whole team."
Internal Accountability
Internal accountability, or staff members' shared sense of responsibility to one another, is a well-established trait of effective schools (Abelmann, Elmore, Even, Kenyon, & Marshall, 1999; Elmore, 2003). A collaborative approach to data promotes that sense of shared responsibility by helping teachers see their instruction as part of a larger effort to serve students more effectively.
For example, at Two Rivers Public Charter School in Washington, D.C., the staff used a cyclical process of lesson planning and evaluating student work to pursue ongoing instructional improvement As 1st grade teacher Laura Marsh explained, the collective process held everyone to a higher standard of excellence:When you're working on a strategy by yourself, you can fudge it, but when you're working on a strategy as a whole faculty, you have the social accountability.
At West Hillsborough Elementary School in Hillsborough, California, 3rd grade teacher Kristi Chenette also found that reviewing students' class work and test results with her colleagues contributed to a strong sense of shared responsibility:Our 3rd grade team feels that we are accountable to the 4th grade team. We want to make sure that our students are set up with the higher-level skills they need to build upon the following year in order to be successful.
A Safety Net for Professional Growth
Teachers also reported that collaborative data use gave them a safety net for taking risks and improving their craft. At Community Academy, an alternative high school in Boston, math teacher Kennedy Omolo received support from his colleagues on the data team in developing a schoolwide homework policy. He reported that this support emboldened his own pedagogy:If it were not for the team, I would not have started something like [my classroom emphasis on homework], with the organization, and the portfolios, and students keeping track of their own work. They made me see that this can be done.
Research Supports Collaboration
Research has identified many other schools that have found advantages in a collaborative approach. For example, Huffman and Kalnin (2003) studied eight district teams engaged in a yearlong data-based inquiry process and found that team members reported growth in their districts' curricular coherence and their own professional knowledge. In a study of four low-income school districts, Wayman, Midgley, and Stringfield (2007) found that data use was most effective in schools where staff had access to usable data and then worked together to calibrate expectations, review concrete evidence of student learning, and participate in instructional decision making. And Lee and Smith (1996) found that teachers' cooperation and collective responsibility for student achievement were associated with higher student achievement gains on the National Longitudinal Study of 1988.
Unfortunately, despite its benefits, collaboration is not always a central feature of evidence-based reforms. In a synthesis of four large-scale studies of district reform, Marsh, Pane, and Hamilton (2006) found that responsibility for data use often fell to district and school administrators; the teachers who used data tended to focus on implications for their own classrooms. Similarly, Supovitz and Klein's (2003) study of five America's Choice schools showed that although some teachers used data to inform their own instruction, there were few schoolwide efforts to use data.
Creating the Right Conditions
It's easy to understand why data initiatives may underemphasize collaboration. Lack of time is almost always a barrier to schools' data use (Ingram, Louis, & Schroeder, 2004; Marsh et al., 2006), and collaborative problem-solving can be especially difficult and time-consuming. Even at Mason Elementary, the process of developing a reading-response rubric took several weeks and involved extensive negotiation among teachers. That effort was followed by an even longer period of testing and refining the rubrics, defining achievement goals, and measuring student progress. Reflecting on the process, data coordinator Hilary Shea said, "This focus on reading-response letters is so nitty-gritty, and look how many hours it took! But if you want improvement, this is what you have to do."
Leaders of effective schools recognize that collaborative data use doesn't just materialize out of the ether; it must be actively cultivated. The experience of the eight profiled schools suggests three concrete strategies that create supportive conditions.
Allocate Time for Collaboration
It may seem obvious, but if teachers are going to use data collaboratively, they need routine meeting times to examine the data and plan for instructional improvement. The schools we profiled devised a number of creative solutions to build time into their schedules. School leaders at Two Rivers Public Charter School used weekly afternoon staff meetings as opportunities for both wholeschool planning and grade-level team meetings. Meetings often began with schoolwide professional development activities; teachers would then break into grade-level teams to develop lessons, analyze student work, or discuss peer observations they had conducted the previous week.
Other schools supplemented regular staff meetings with special time for collective planning. The staff of Community Academy attended a weekend retreat in the countryside to put the finishing touches on an instructional action plan for improving student achievement the following year. Away from the frenetic pace of the school day, the teachers nailed down the details of a schoolwide homework policy, including the steps they would take to evaluate its success.
Perhaps the most difficult time to schedule collaborative meetings is during the school day, when students are present. However, midday meetings are sometimes necessary to enable groups of teachers to observe one another's classroom teaching. To make routine peer observations possible, Murphy School principal Mary Russo trained a group of skilled substitutes who could take over for teachers without missing an instructional beat. As math coach Angel Petrie explained, such logistical support made a big difference: "We feel well supported when Mary takes care of those seemingly small things so we can focus on the business of improving teaching and learning."
Delegate Data Management
Collaborative data use requires organized, accessible data and well-planned, smoothly facilitated meetings. School principals are usually too busy to orchestrate these efforts themselves. In the schools we profiled, principals laid the groundwork by appointing data coordinators or teams to manage logistics and plan meetings.
At Mason Elementary, principal Janet Palmer-Owens set the stage for collaborative data use when she enrolled in a data course with teacher Hilary Shea and then released Hilary from some of her teaching duties to serve as data coordinator. This move enabled Hilary to assume such tasks as creating data binders for each teacher, planning and facilitating team meetings, and developing data presentations to jump-start conversation and analysis. By nurturing one teacher's interest in data, Janet created the conditions that made ongoing collaboration so effective.
West Hillsborough Elementary had a similar arrangement: Third grade teacher Judy Pappas served part-time as the school's data specialist to streamline data management and create discussion-friendly templates and charts. Judy explained,Teachers need the data, but we don't want them spending all of their noninstructional time entering data; we want them to have that time for planning their instruction to address the problems that are indicated by the data.
Other schools designated teams of data leaders. At Pond Cove Elementary in Cape Elizabeth, Maine, principal Tom Eismeier asked the school's media specialist to help him assemble a data team that would tap expertise from across the district. The final team included two literacy specialists who could tie data analysis to the school's ongoing literacy reform efforts, as well as a district technology director who could help develop an online data system for teachers. Tom's ongoing support lent legitimacy and momentum to the team's work.
Establish Norms That Foster Trust
For data analysis to lead to real instructional improvement, even having common meeting time and great data managers may not be enough. Examining data collaboratively means treading a fine line: Teachers must take responsibility for their students' learning, but they must also have the latitude to refine and develop their craft. To build a collaborative culture and promote instructional improvement, the data-use process needs to emphasize solving problems, not passing judgment.
Several schools that we profiled took care to establish norms that fostered trust. Principal Anthony Ranii described a norm of positive intentionality that teachers had developed at West Hillsborough Elementary: "We assume that everyone wants what is best for the students and that the students' interests are at the forefront of everyone's mind."
At Murphy K–8 School, the norm oftransparency made data use—especially peer observation—less threatening. The school developed a highly prescribed peer-observation process: Before delivering a model lesson, teachers could anticipate (and even specify) what their peer observers would focus on. A few teachers' initial willingness to deliver model lessons made their colleagues more comfortable with the process. As Angel Petrie recalled, "Once these highly respected teachers made mistakes, encouraged others to point them out, and a showed a level of comfort doing that, everyone else fell in line."
Finally, at McKay K–8 School in Boston, the norm of objectivitycontributed to productive data discussions. Before teachers at McKay became accustomed to examining test scores collaboratively, some teachers reacted to disappointing test results with shock and frustration. But strong norms for talking about data helped channel staff members' analytic skills. At each grade-level staff meeting, the data team would present a few graphs about achievement in a particular content standard. Teachers would make observations about the data in each graph, explain how they'd arrived at those observations, and then describe what else they noticed. This approach depersonalized the test results and promoted objective inquiry, leading the staff toward deeper questions and shared instructional solutions.
A Culture of Improvement
When school leaders face time pressure to improve achievement, they may be tempted to analyze data themselves and respond with instructional directives. Collaborative data use, admittedly, takes longer and requires hard work. Although it can be difficult to create the conditions that support collaborative data use—including routine meeting times, skilled data management, and group norms that foster trust—the eight schools described here remind us that these conditions are attainable. Moreover, the potential payoff for organizational learning, and ultimately for student learning, is too great to ignore.
In helping schools use achievement data to improve instruction, we have observed again and again that the real power of data is unlocked when, like the teachers at Mason Elementary, staff members grapple as a team with information from a wide range of sources. Schools that explore data and take action collaboratively provide the most fertile soil in which a culture of improvement can take root and flourish.
