Streamlining Response to Intervention Screening

Diane Holben and Jeff Baird

As any school district knows, screening student data is a time-consuming task. It has become even more daunting with the accountability demands of NCLB and other federal initiatives. The Response to Intervention (RTI) process, for example, mandated under the Individuals with Disabilities Education Improvement Act (IDEIA), requires schools to develop a comprehensive portrait of student achievement, behavior, and instructional environment to determine whether students require intervention strategies to succeed (2004).

Traditional paper records or data stored in separate electronic applications that can't communicate with one another hamper efforts to combine multiple data for child screening purposes. Teachers, counselors, and other specialists can spend days gathering the data needed for child study conferences, and the delays in accessing data prevent schools from effectively using their data to improve student achievement (Bernhardt, 2005).

To address this concern, the Allentown School District, a midsize urban district of 18,000 students in southeastern Pennsylvania, purchased a data warehouse. The TetraData warehouse, developed by Follett Software Company, is similar to a large electronic filing cabinet; the software stores all of the achievement, staffing, and demographic data from multiple sources in one location that can be accessed through the Web. The warehouse automatically pulls new data on a nightly basis from the student information system data updated by district staff and teachers during the day. Central office staff also periodically upload district assessment data into the warehouse.

Using the data warehouse allows us to customize how we collect and organize student data into three types of reports, which teachers, administrators, and counselors told us would make monitoring of student performance more meaningful and identification of struggling students more efficient. The RTI, class roster, and student profile reports allow school staff to spend more time analyzing the data and designing interventions to assist students.

In devising all three reports, we tried to provide the maximum access to student data while still honoring the Family Educational Rights and Privacy Act (FERPA) regulations (1974). Setting security constraints prevents users from viewing restricted data; restricted fields simply appear blank in the report. In addition, users can save reports they create as an HTML, PDF, or XLS file so that they can be easily shared with others.

Response to Intervention Report

Figure 1. The response to intervention report consolidates a variety of student information and flags students at risk academically. (Click to enlarge.)

The RTI report allows us to be proactive in identifying struggling students so that we can comply with the requirements of the IDEIA legislation. Figure 1 shows a sample RTI report using scrambled student names and ID numbers to preserve confidentiality. The report consolidates various data elements and sets the target, or "cut," score for each indicator. When a student reaches a cut score, the report assigns the student points related to the level of risk associated with that score. A mathematical calculation creates a total point index across all indicators and ranks students based upon this index value. In this way, students who may be at risk are identified as early as possible so that school teams may plan and implement appropriate interventions.

Using existing research, we decided to include the following indicators in the RTI report:

Student Mobility: Students who repeatedly move from one school to another tend to fall behind in reading and other academic subjects, putting them at risk for dropping out in high school (Smith, Fien, & Paine, 2008). Also, the mobility itself might be an indicator of other issues affecting the child and a source of other consequences, such as increased student suspensions (Engec, 2006) that indicate potential dropouts.

Attendance: Absent students don't receive instruction and therefore will have lower achievement. In addition, middle school students with more than 40 absences in a year are at increased risk for dropping out in high school (American Youth Policy Forum, 2007; Balfanz & Legters, 2004).

Discipline: Students who receive multiple out-of-school suspensions are at a higher risk for academic difficulty and dropping out of school (Carpenter & Ramirez, 2007). The students miss school during the suspension, and the suspension itself often indicates other issues that could affect student achievement.

Assessments: Low student achievement on summative or formative assessments demonstrates that students have not mastered the state standards. Data from the Pennsylvania System of Students Assessment (PSSA) in Reading, Writing, and Mathematics are included, as well as district-selected assessments such as Dynamic Indicators of Basic Early Literacy Skills (DIBELS) and 4Sight Benchmark Assessments. In the near future, we plan to add the World-Class Instructional Design and Assessment (WIDA) English Language Proficiency test and the PSSA Science assessment to the report to provide a more inclusive assessment picture.

Course Grades: Any grade of D or F also indicates that a student is not proficient on the standards associated with that course. In addition, students who fail English or mathematics at the middle school level are at increased risk for dropping out in high school (American Youth Policy Forum, 2007; Balfanz & Legters, 2004).

Grade Retention: In the near future, we plan to add data about students who are retained in their grade. Retentions are a documented risk factor for dropping out in high school (Carpenter & Ramirez, 2007).

A school team analyzing report data can identify the root causes of the student achievement difficulties and design individualized interventions appropriate for the student. These interventions may be academic, attendance-related, or behavioral. The report also allows the team to monitor student progress and determine the effectiveness of the intervention without needing to collect additional data.

Class Roster Report

Figure 2. Class roster assessment reports help teachers track student performance and guide instructional choices. (Click to enlarge.)

Our class roster report lists all the students in a teacher's class with three options: assessment, demographic, or organizational data. Teachers can run these reports at any time to use in their daily planning. If teachers need more information, they can click on a student's name to access the more detailed data located in the student profile report. Figure 2 shows the assessment option on the class roster report. Teachers use the class roster report to provide them with an overview of classroom proficiency levels, which will aid them in guiding instruction for students. In addition, this report allows them to develop instructional groups and easily regroup students based on current performance levels. The demographic version allows teachers to verify all demographic data required on state testing forms. Finally, because the student rosters are updated daily, the organizational option of the report provides a current student roster with a checklist that teachers may use to manage classroom tasks such as the return of forms and homework completion.

Student Profile Report

Figure 3. The student profile report aggregrates three years of data in one place. (Click to enlarge.)

At the core of all of our reports is our student profile report, which can be created on its own or accessed through the RTI report or class roster report. Figure 3 provides a sample student profile report using scrambled but real data. This report provides the most recent three years of assessment, attendance, discipline, and demographic data for the student. Teachers, counselors, and administrators can use these reports for instructional planning, parent conferences, or instructional team meetings. The aggregation of three years of data on one report streamlines the data collection process, allowing schools to focus their time on planning and instruction rather than data collection. Both of these reports have become integral resources for us as we strive to meet individual student needs.

Because the data-on-demand feature in our data warehouse allows us to update student information nightly, the student profile report is always up-to-date. This is a critical component because the district's annual mobility rate is over 20 percent, with about half of that representing students moving from school to school within the district. The report allows school staff to access current data in a timely manner when students change schools within the district, causing less interruption to the student's academic program. To further assist schools, in the near future the report will list the interventions in which a student has already participated with the dates and completion status.

Reversing the 80–20 Rule

We believe that the data warehouse and these custom reports are changing the data analysis priorities in the district. In previous years, staff would spend 80 percent of their time and energy just simply collecting and organizing data, leaving only 20 percent of their time to analyze the results of the data and plan appropriate actions. We believe using the data warehouse and its various reports will allow us to reverse the numbers—giving us 80 percent of our time to analyze the data and formulate data-informed decisions.

As a related bonus (although it didn't seem so at the time!), this process forced us to rethink the way we organized and stored our data. We needed to standardize our data collection protocols and create long-term data monitoring schedules that would ensure clean, well-organized data for our educators to use. 

Building our data warehouse and these three custom reports seemed overwhelming at times. The support of key administrators with a strong sense of data usage and staff dedicated to the implementation of the data warehouse and devising the scope of the reports made the process manageable. And the benefits are substantial. Instead of data analysis being a superficial "post-game wrap-up," it is becoming more like the strategic conversation to make corrections in the huddle during a time-out. Schools have the data they need to make instructional adjustments in a timely fashion, which in turn improves the education of the students.

Tom Smith, a literacy coach in the district's Cleveland Elementary School, envisions the long-range impact of using data. "Not only do reports measure the impact of interventions, but they also can be used longitudinally to anticipate and prevent problems," he points out. "If a student's at-risk score rises significantly, a counselor can investigate and help connect the student with appropriate services before the student reaches a crisis stage," and avoid the trap of constantly "putting out fires." As the Allentown School District continues implementation of the RTI model, data reports such as these will facilitate individual student planning so that all students can reach their academic potential.

References

American Youth Policy Forum. (2007, March 8). Philadelphia's project u-turn: Citywide efforts to address the dropout crisis. Retrieved June 12, 2008, from www.aypf.org/forumbriefs/2007/fb030807.htm

Balfanz, R., & Legters, N. (2004). Locating the dropout crisis: Which high schools produce the nation's dropouts? Baltimore, MD: Johns Hopkins University Center for the Social Organization of Schools. 

Bernhardt, V. (2005). Using data to improve student learning in high schools. New York: Eye on Education.

Carpenter, D. M., & Ramirez, A. (2007). More than one gap: Dropout rate gaps between and among black, Hispanic, and white students. Journal of Advanced Academics, 19(1), 32–64.

Engec, N. (2006). Relationship between mobility and student performance and behavior. Journal of Educational Research, 99(3), 167–178.

Family Educational Rights and Privacy Act (FERPA), 20 USC Section 1232 (1974).

Individuals with Disabilities Education Improvement Act (IDEIA), P.L. 108-446 (2004).

Smith, J. L. M., Fien, H., & Paine, S. C. (2008). When mobility disrupts learning. Educational Leadership, 65(7), 59–63.

Diane Holben is director of accountability and assessment and Jeff Baird is instructional software project manager in the Allentown (Pa.) School District Office of Accountability and Assessment.