At this point in the pandemic, the need for clear, quantifiable data on the effect of learning disruptions has only grown stronger. A report released last week from the Harvard Center for Education Policy Research aims to make the most comprehensive assessment to date on how student achievement was affected in the first years of the COVID-19 crisis. Most notably, the report found that remote instruction had an undeniably negative impact on learning—and that high-poverty schools that disproportionately serve Black and Hispanic students faced the biggest setbacks.
The high-profile study, the first to use test-score modeling to gauge achievement both broadly and within schools, has drawn renewed attention to schools' efforts to address instructional gaps and accelerate learning for those most affected by the pandemic. It also highlights the deep financial resources schools will need to help students catch up. At the same time, the report has sparked outside concerns from some educators about emphasizing academic gains and losses without also considering the social-emotional needs and engagement of students today.
A Deep Dive on Student Performance
In its analysis, the research team drew data from the Measures of Academic Progress Growth assessments provided by the NWEA, which schools typically administer three times a year. Using the results from 2017-2019, the researchers were able to create a model for what academic outcomes might reasonably have been from 2019-2021 if the pandemic had not occurred.
For the 2020-2021 school year, students across the country performed worse than the model’s expectations, but high-poverty schools, defined by an enrollment of more than 75 percent of students qualifying for free and reduced-price lunch, suffered the most. In that year, high-poverty schools that were mostly remote lost about 50 percent more learning than low-poverty schools under the same conditions.
“On top of the gaps that are normally in a high-poverty school, there was an additional learning loss during the pandemic,” Douglas O. Staiger, an economics professor who coauthored the study, said in an interview with ASCD.
The negative impact of remote learning was especially strong for high-poverty schools because they were more likely to switch to remote instruction and remain that way for a longer period of time, the study found. The researchers, however, did not present specific reasons why these schools suffered more under remote teaching.
“The pandemic was tough on everybody, [but] when people were in person, at least it had an equal impact,” said Staiger. “When people weren't in the schools, that equalization fell apart and the folks in the high-poverty schools really fell behind.”
Significantly, the report includes a within-school analysis to track how Black and Hispanic students performed in comparison to white students within the same buildings and found no substantive differences. This suggests that expanded achievement gaps for students of color were due mainly to the fact that they are more likely to attend high-poverty schools.
Finally, the team mapped the possible financial cost of the learning loss they measured by estimating the amount of time and money a school might typically need to cover the corresponding missed instruction. High-poverty districts that were remote for most of 2020-2021, the study estimates, will need to spend an amount of money equivalent to 40 percent of their annual budget, or nearly all of their federal aid, on academic recovery to eliminate the losses their students experienced. Staiger calls this figure a “good ballpark” estimate but one that, in practice, will look slightly different in different school contexts.
The need for strong district- and state-level action in the face of these numbers is urgent and supersedes what some school leaders may have expected, said Staiger. In some cases, high-poverty districts are facing gaps in student learning that can’t be easily covered with typical catch-up techniques like tutoring.
On top of the gaps that are normally in a high-poverty school, there was an additional learning loss during the pandemic.
A Complete View of Learning Loss?
The study's isolated focus on academic learning loss, however, has led some to express caution about its potential applications.
Yong Zhao, a professor of educational leadership at the University of Kansas who is not affiliated with the Harvard study, said he isn’t surprised by the study's findings, especially that students at high-poverty schools fell further behind. However, he expressed concern about a focus on student academic achievement without an equally in-depth look at how students’ various other social and emotional needs may translate to policy.
“If you keep looking at this one thing, you will try to fix this one thing, and then you will ignore the others because there's no other information as powerful as this to show where children are,” he said in an interview with ASCD.
Zhao has argued in the past against falling into the “learning loss trap”—that is, using testing data to support grim educational prognoses that miss developments in less-quantifiable areas, such as confidence, self-determination, and entrepreneurial thinking. If students became more independently driven, or resilient in the face of at-home distractions, or passionate about individual hobbies, it wouldn’t necessarily appear in their test scores.
“We really don’t know what happened to all of these children other than their test scores,” Zhao said.
Even before this report, other instructional experts have cautioned that a narrow focus on academic learning loss, as judged by standardized test scores, could lead to deficit mindsets in schools and an over-reliance on remediation—which in turn could exacerbate gaps in engagement and opportunities for deeper learning experiences.
The Harvard study’s researchers recognize that this report doesn’t include every dimension of students’ well-being or learning, but Staiger says that the study’s new figures give added weight—and more accurate figures—to undeniably serious academic problems that educators will likely only continue to grapple with in the coming years.
“The point is, there’s an enormous challenge here. And unlike most times . . . there’s an enormous amount of resources that have been made available to schools,” said Staiger. “This is a really good time to think about how you are going to have to balance social-emotional [needs] and the learning.”
