It's time to get serious about data. Most state departments of education and school districts will say they've been serious about data for the past 15 years, especially since the introduction of the No Child Left Behind act, which considerably upped the accountability ante. Unfortunately, the way many learning organizations in the United States got serious was to look at their high-stakes student achievement results and focus their plans on the lowest-scoring subject area or subgroups of students or on the bubble kids.
We know what happened next. The subject-area scores seesawed, depending on the focus. Social studies and science were neglected in elementary schools because they weren't assessed. Many students weren't challenged because the focus was on getting as many students "proficient" in the easiest way. And if a student was already proficient, no problem. There was no need to focus any extra attention on him or her.
More recently, we've come to the realization that we must focus on improvement strategies that will have a positive effect on all students and teachers. To do this, schools must gather and analyze data that will help them understand where they are now as a system; why they're getting the results they're getting; and, if they're not happy with current results, how to get better results for everyone.
Enter the Continuous School Improvement Framework
Schools need a framework to guide the improvement of teaching and learning for each teacher and student. Schools are learning that if they don't analyze and change inefficient or ineffective processes, they'll keep getting the same results.
To that end, three years ago, the Hawaii Title I office shifted its focus from compliance to improvement. The Continuous School Improvement Framework that Hawaii used to achieve this consists of five questions:
Where are we now?
- How did we get here?
- Where do we want to be?
- How are we going to get there?
- Is what we're doing making a difference?
Hawaii's Story
Approximately six years ago, the Hawaii Department of Education had created a longitudinal data system to enable schools to access all their data through one tool. A few years later, data system staff engineered reports that helped create a data profile for each school in the state. These reports provide the biggest support for schools in their shift to comprehensive data analysis and continuous school improvement. Many schools in other states never get all their data organized to review.
The data profile, available with a couple of button pushes, consists of longitudinal demographic data, from general to more specific. These data include overall enrollment as well as enrollment disaggregated by grade level, gender, ethnicity, English language learner status, and other indicators. The system also stores student, staff, and parent questionnaire results required by the state. Also included are longitudinal summative student learning data disaggregated by grade level, student subgroup, and classroom, along with student growth data. Each school also has to provide an overview of their school processes.
The analysis starts with a list of instructional, administrative, and organizational processes and programs that the school uses. The second part of the analysis entails determining how well the processes and programs are implemented, as well as implications for improvement.
The Process in Action
In 2013, the K–6 Oahu Elementary School decided to commit to a continuous school improvement process. The school serves a diverse population of 610 students—approximately 35 percent Pacific Islander, 28 percent Asian, 19 percent from multiple ethnic groups, 13 percent Hispanic, 5 percent white, and less than 1 percent black. Ten percent of students are English language learners, and 35 percent qualify for free or reduced-price lunch.
Although the school had its strengths, it was experiencing a number of challenges it wanted to address—such as low teacher morale, lack of a shared vision, growing achievement gaps between high-needs and non-high-needs students, and decreased math proficiency for students from low socioeconomic backgrounds.
Under the leadership of principal Kimo, staff members came together in a three-hour session to review their schoolwide data, which mostly came from the state longitudinal data system, to better understand how they were getting their current results. Staff members were seated in mixed-grade and subject-area groups and tackled data that targeted four areas: demographics; perceptions (that is, culture, climate, values, and beliefs); student learning; and school processes.
What Demographic Data Revealed
The demographic data were handed out to each staff member, along with study questions to use in the analysis. Each group addressed three areas: strengths, challenges, and implications for the continuous school improvement plan. Participants began by individually jotting down what they saw as strengths and challenges as they reviewed the data. Then they filled in the third item—implications for the continuous school improvement plan.
After 12 minutes, each group posted its collective thinking about the demographic data on chart paper. One group then read all the strengths it had found, while the other groups' reporters crossed off similar concepts from their chart papers and shared ideas not already mentioned. The groups did the same for the challenges and implications. Through this process, the staff quickly merged what they saw in the demographic data from individual, to small-group, to whole-group thinking in under an hour.
Here's some of what they found:
- Strengths: Enrollment has been steady for the past four years; the teacher-student ratio is good (1:24); students are diverse; 13 percent of students are considered gifted; 10 percent of students qualify for special education services; attendance is good (96 percent across the school for four years); overall behavior incidents are down to 182 from a high of 279; and only one student was suspended last year.
- Challenges: Students are diverse; there's a higher percentage of boys than girls; 35 percent of students qualify for free or reduced-price lunch; 10 percent of students are English language learners; high-needs students have the lowest attendance rates; there were 182 discipline incidents during the year, mostly within classrooms; 80 students are well above grade level in grades 3–6, as noted by their enrollment in the gifted program.
In their discussion, the staff reached consensus on the following implications for the school improvement plan:
- We need professional development to better meet the needs of students who live in poverty, English language learners, boys, and gifted students.
- We need to clarify the processes and materials required to meet the needs of, and exceed expectations for, all our students
- We don't qualify for extra funding for high-needs students, so we need to learn how to adjust our teaching to better meet these students' needs.
- We need consistency in implementing positive behavior concepts.
- When we meet the learning needs and styles of students, we believe that attendance and behavior will improve.
What Perceptual Data Revealed
Staff members repeated this process, but now they used perceptual data drawn from a series of questionnaires: two given to students, one to parents, and two to teachers. Here are some highlights of their findings:
- Strengths: Students feel safe and that they belong at the school; students and parents are happy with the teachers; parents feel welcome at the school; everyone likes the campus environment; staff feel respected by the administration; staff believe that quality work is expected from them and that learning can be fun; and staff love seeing the results of their work with students.
- Challenges: Students think they have limited academic choices; some students don't feel challenged; parents have trouble understanding the report card; parents want their children challenged more; teachers feel a lack of communication with the administration; teacher morale is low; not all staff members believe all children can learn; teachers don't believe the school has a shared vision; teachers don't collaborate to make student learning consistent across grade levels; and teachers don't always know how to work effectively with students with learning disabilities and with those whose home language isn't English.
Through their discussion of these strengths and challenges, staff members zeroed in on the following implications:
- We need to continue making everyone feel safe and welcome.
- We need more effective communication between home and school, schoolwide, and across grade levels.
- We need to address teacher morale.
- We need to learn how to address the needs of all students and accelerate their learning.
- We need to clarify the school's vision.
- We need to get everyone's input in schoolwide decision making.
What Student Learning Data Revealed
Groups then looked at student learning data, which included the state summative assessment, disaggregated in numerous ways, including by student to show individual growth. As before, they looked at the data individually, then in small groups, then in the group as a whole. This is what they found:
- Strengths: Reading, math, and science proficiency rates outpace the state average in all areas; students of native Hawaiian ancestry attained an 83 percent proficiency rate for reading and a 66 percent proficiency rate for math, up from previous years; 4th grade reading proficiency increased by 7 percent, and 5th grade proficiency increased by 4 percent over the previous school year; math proficiency rates improved in 4th, 5th, and 6th grades by at least 4 percent each.
- Challenges: The achievement gap between high-needs and non-high-needs students has increased each year since 2012, particularly in math; math proficiency for students from low socioeconomic backgrounds and for students receiving special education has decreased; students receiving special education services have the lowest achievement rates (31 percent in reading and 21 percent in math); not enough students are growing academically each year.
After reviewing these strengths and challenges, staff members decided on the following implications:
- We must decrease the academic differences between high-needs and non-high-needs students in math and reading.
- We need to learn how to accelerate each student's learning.
- We need to work together within and across grade levels to create a continuum of learning that makes sense for students so each year leads to at least one year's growth for every student.
What School Processes Data Revealed
For school processes, staff members listed their current instructional, administrative, and organizational processes and programs, as well as programs related to continuous school improvement. Some of the instructional processes teachers were using included differentiated instruction, direct instruction, formative assessments, and technology integration. Organizational processes included parent involvement, policies and procedures, professional learning communities, and teacher observations. Many staff members were not aware of all the processes and programs that operated in the school.
They decided to highlight in green all the processes and programs they believed were being implemented with integrity and fidelity. These included standards integration, grade-level teams, instructional coaching, leadership team, professional learning communities, and teacher evaluation. They then highlighted processes and programs that needed professional development, consistency, or improvement. These included differentiated instruction, student choice in assignments, formative assessments, grading, and homework.
The discussion revealed the following problems:
- The school mission and vision aren't widely understood.
- There's a lack of intervention programs for students struggling in math and reading.
- There's a lack of timely review of progress-monitoring data and adjustments in instruction.
- Students are in homogenous groups for reading in grades 2 and up and for math in grades 3 and up.
- The response to intervention (RTI) system isn't widely understood.
- New teachers didn't receive the professional development for math intervention offered the previous year.
- There's no process for creating a continuum of learning across grade levels.
The discussion that followed this analysis focused on getting every teacher on the same page with a clear mission, shared understanding of the most effective approaches to improving student growth, and consistent implementation of RTI. Teachers agreed to eliminate overlapping programs and processes and to decrease the amount of direct instruction used. They discussed how homogeneous groupings should be used within interventions and as a way to differentiate instruction, not as their primary structure for improvement. Staff agreed that their current mission statement was too long and that they needed a new one that drove them.
From Implications to Action
Staff members then lined up the implications for all four areas they examined to look for commonalities (see fig. 1). What was working—and not working—began to emerge. Each type of data showed a related story through a different lens. With all the lenses in place, staff members could see what needed to change to get different results. They found four major commonalities:
Figure 1. Combined Findings from Discussions of Four Data Types
- We need professional learning to help us better meet the needs of students who live in poverty, English language learners, boys, and gifted students.
- Through professional learning, we need to create an RTI system that everyone understands in the same way and that everyone will implement with integrity and fidelity.
- We must revisit our mission and vision so everyone understands them in the same way.
- Each classroom teacher must use strategies that will get the best results for all students.
Staff members determined that they needed to start by learning more about how to meet the needs of all the students in the school. Not having the resources to bring in experts or devote full days to professional learning, staff members committed outside time to study new approaches in book groups, to conduct Internet searches, and to watch videos to understand what other schools do with similar populations. They then shared their learning during staff meetings.
A newly identified RTI leadership team attended an RTI training in the summer that helped them build their system for RTI. They left that training with their structure created, an implementation guide, and a software package that will help them quickly analyze their screening and progress-monitoring data and assist with monitoring RTI implementation and effectiveness. Teachers agreed to read selected articles and watch videos about differentiated instruction and ways to meet the needs of all students. The school year ended with professional development from a reading expert on reading and RTI.
This work excited the staff to create a new shared vision for the school. The old shared vision was a long statement that individuals could interpret in many different ways. The new version spelled out the components that needed to be in place in every classroom—such as differentiated instruction, formative assessments, and choices in assignments—what each will look like, and what evidence should be seen and available.
With this vision in place, it was very easy to create a plan to implement it. A leadership structure was developed to implement the vision and eliminate some of the challenges that were uncovered in the comprehensive data analysis, such as the lack of consistent communication throughout the school and the inconsistent implementation of curriculum, instruction, assessment, and learning-environment guidelines.
This is the first year of full implementation. All teachers are clear on what they need in order to implement the new vision in their classrooms and how RTI will be implemented schoolwide. The leadership structures are in place to ensure consistent implementation of the vision within every classroom, a continuum of learning for all students across grade levels, and growth for every student.
Because the state changed its assessment system to Common Core State Standards assessments last year, it is difficult to compare longitudinal summative data. But from all accounts, things are looking up. Teachers know they are going to be implementing programs and strategies that will make a difference. Because the classrooms are consistent, they feel some old frustrations will be eliminated; therefore, morale has improved.
Getting Systemwide Results
There are several benefits to this approach to school improvement. First, schools look at what all types of data are telling them. They don't just cherry-pick data and draw conclusions from a few data points without studying the linkages among the different types of data.
Second, all staff members become engaged in reviewing schoolwide data. This provides an opportunity for everyone to understand what has to change systemwide to get better results.
Third, by starting with an independent review of data followed by a small-group review, staff members can validate with others what they see in the data. This gives them the confidence to continue their analysis. By going from small-group to large-group discussions, staff members build consensus on the implications for the continuous school improvement plan. Staff see a lot independently. Together, they see much more.
Finally, with everyone committing to implementing one continuous improvement plan informed by a detailed shared vision, the focus on continuous growth for all students is clear to all.
When schools gather and analyze their data so they understand where they are now and why they're getting the results they're getting with current processes, they're able to create their own continuous school improvement plan based on a vision that everyone understands and has committed to. They're then able to put teaching and learning strategies in place that will ensure better results for all students.
Author's note: The school and principal names used in this article are pseudonyms.
