Analyzing data not only helps inform decisions and challenge assumptions, but also helps teachers view their instructional and collaborative practices with a new perspective (Feldman & Tung, 2001). Rallis and MacMullen (2000) describe the inquiry-minded school as one that “recognize(s) that improving teaching and learning is an intentional and ongoing process.” Given the challenges inherent in creating and sustaining an inquiry-minded approach in an individual school, it is crucial that school districts create a culture that recognizes and supports inquiry.
During the 2001–2002 school year, North Adams Public Schools—a small urban district in western Massachusetts—organized a districtwide professional development program for its elementary and middle schools using a data-based inquiry and decision making process to develop a culture of inquiry. Consisting of three elementary schools, one middle school, and one high school, the district serves 2,260 students—of whom 93 percent are white and 39 percent eligible for free or reduced-price lunch.
The Coalition of Essential Schools, a national network of schools that are personalized, democratic, and reflective, worked with the district to plan and implement this program. Members of the Coalition trained district administrators on the data-based inquiry and decision making process first, and then the district leaders helped train the teachers. The authors of this article—two from the district, two from the Coalition's Regional Center—led the effort.
The Data-Based Inquiry and Decision Making Process
- Step 1: Set a vision. School staff create a vision for their school.
- Step 2: Collect and analyze data. Each person writes three to five “I see” statements that describe what he or she sees in the data. This process forces each person to look at the data objectively. Everyone shares his or her statements with the group, and the group members add ideas.
- Step 3: Determine strengths and challenge areas. On the basis of the data, teachers list areas in which they find strengths and challenges—it is important to list both to highlight effective practices—then they vote on the one challenge that they believe is the most significant barrier to student achievement. This becomes the priority challenge.
- Step 4: Plan action. Participants brainstorm why they believe this barrier exists and identify other necessary data to collect to determine whether the hypothetical causes are in fact the real causes of the problem. If the hypothetical cause is proven to be the barrier on the basis of the new data, the group develops solutions, implementation plans, and a process for assessing the plan's impact on the challenge.
- Step 5: Assess annually. The school cites evidence around its annual measurable goals.
What Happened in Each School
School-based data leadership teams, consisting of the principal and the trained teachers in each school, met with the Coalition of Essential Schools staff to plan their data-based inquiry. The district administration wanted every school to focus its inquiry on students' low math achievement, but each school examined a different likely cause. One school chose to develop grade-level math assessments to examine students' math understanding, whereas another felt the greatest challenge was to meet the needs of special education students more effectively, and the third school identified discipline as its biggest challenge.
Math Comprehension at Sullivan Elementary School
- How effective was this assessment in helping us understand students' strengths and challenges?
- In what ways is our curriculum meeting students' needs? Which gaps in our curriculum do we need to address?
Inclusion at Greylock Elementary School
On the basis of the data analysis, the Greylock data team hypothesized that its school's current inclusion model contributed to the low math scores for both regular and special education students. The data team brought this idea to the school's leadership team, which agreed to use the data-based inquiry and decision making process to explore the issue. The leadership and data teams then collected additional data on student achievement to determine whether their hypothesis was correct. They used standardized tests and report card grades; measures of engagement (such as attendance); teacher perceptions of parent involvement; students' learning styles; professional development and common planning time for teachers; and the amount of pull-out time for all services given to students by staff other than the regular classroom teacher, such as special education instructors, speech therapists, occupational therapists, Title I teachers, and so on.
- Sixty-nine percent of individualized education plan students were below grade level in math and reading achievement;
- Seventy-four percent of regular education students were at grade level in math and reading achievement;
- Half of the individualized education plan students had multiple classroom pull-outs; and
- Eighty-three percent of individualized education plan students were classified as visual and kinesthetic learners, whereas 79 percent of regular education students were rated as visual learners and 39 percent as kinesthetic learners.
Greylock teachers noted that although regular education students achieved at higher levels than students with individualized education plans, teachers' subject matter expertise in math and science affected both inclusion students' and regular education students' performance. The data showed that teachers had different subject area strengths and had difficulty meeting the needs of all learners in their weaker areas.
As a result of this process, faculty decided to restructure the school and combine grades K–2 and 3–5 to allow teachers to team-teach. That way teachers could teach to their strengths, develop a collaborative school culture, and better meet the needs of inclusion students. Even though faculty developed this idea with the support of the principal, there was still a process of give and take before all teachers accepted the change.
Discipline at Brayton Elementary School
The data team at Brayton Elementary School believed that frequent discipline problems interrupting instructional time caused students to score low on math assessments. Because the school already had a discipline committee, the data team planned to use the data-based inquiry and decision making approach with the committee's assistance to determine whether discipline really was a factor. Together, the team decided to collect a variety of additional data, including measures of student achievement (standardized test and report card grades), engagement (such as attendance and discipline), and teacher perceptions of parent involvement. In addition, teachers documented interruptions in their classes over three days, noting whenever a disruptive behavior occurred, who it affected (a small group, the whole class), and who handled the issue (no one, the classroom teacher, the principal).
- Classroom teachers handled 99 percent of the interruptions; and
- Less than 1 percent of teachers asked other teachers to assist with interruptions.
With this information, teachers hypothesized that classroom discipline problems were indeed disrupting instruction, and that teachers perceived that they alone had to handle all the problems in their classrooms. One reason for this was that teachers were not fully informed about schoolwide rules, procedures for when and why they would seek other support, or detailed consequences for rule violation, including when to communicate with parents. Consequently, teachers felt alone in trying to handle discipline.
The staff believed that, as a first step, the school needed to create a discipline policy. Over the course of two full faculty meetings and many team meetings, the school developed a policy based on the overarching idea that “We want our school community to be a safe place where everyone can learn.” The policy lists responsibilities for students, teachers, and parents; school rules; and a series of consequences for breaking those rules. This policy emphasizes constant communication with parents.
But just because the school developed a plan does not mean that discipline issues were solved and instructional time increased. So far, implementation of the new policy has been mixed, and different teachers have different views about how well the policy is working. Momentum is building toward the idea that the policy alone is not enough and that staff need to discuss the larger issue of instructional practice and school culture as initially planned. Because they have a process, teachers are more comfortable collaborating about their practice, both with and without data.
Lessons Learned
Building a districtwide culture of inquiry is often a messy and nonlinear process, but three lessons we learned have implications for other districts.
Data-Based Inquiry Is a Powerful Tool
The data-based inquiry model gave schools a tool to address areas of concern and helped teachers reflect on their practice. Teachers who engaged in an inquiry approach were more reflective about their practice and more willing to push and probe to find answers, basing their questions on data rather than assumptions.
A common criticism of using data is that data limit dialogue. To the contrary, the district data team found that using data was a very open-ended experience. All three school-based data teams began the process solely by looking at Massachusetts Comprehensive Assessment System (MCAS) math data. Using this data led to wide-ranging conversations about barriers to student learning. Only one school, in fact, continued examining math content. In the other schools, the process led teachers away from math and toward examining other barriers to student learning.
Inquiry Should Be Localized
Although we worked districtwide, we recognized that inquiry should be localized. The process used, the questions asked, and the data collected and discussed need to address each individual school's most compelling concerns. Our process did start with the district—training district administration and using districtwide data—but it did so for three reasons. First, it built buy-in and understanding of the process. Second, it gave everyone the power to shape the next steps of the work. Third, it reminded everyone that, while the work was localized, important lessons could be shared across the district.
The District's Role Is Crucial
Schools and districts must forge a new relationship that is mutually supportive, beneficial, and focused on students' needs. The district must move from mandating and monitoring the implementation of reforms to providing services and support enabling each school to achieve its goals and mission. First, the district must have a process in place to organize data and get it to teachers to help them make decisions. Second, the district needs to provide time and other resources to help schools develop and sustain inquiry approaches. And third, the district must support the sharing of information across all district schools.
