Teachers' curriculum options were once limited to traditional textbooks, informational texts, novels, and materials provided by district-contracted education companies or passed along by others. Today, however, the online marketplace is flush with copious materials, often at little or no cost. With a few clicks of the keyboard, a teacher can find for students a worksheet to practice addition and subtraction, a frog dissection kit for a biology lab, and quizzes and answer keys for all five acts of Romeo and Juliet.
So voluminous and wide-ranging are these resources that recent studies by RAND found that nearly all teachers report using the Internet to source instructional materials, drawing from popular sites such as Share My Lesson and Teachers Pay Teachers. For example, 55 percent of English language arts teachers said they used Teachers Pay Teachers for curriculum materials at least once a week (Kaufman et al., 2018). That site reports that over one billion resources have been downloaded from it overall—a significant indicator of the shift in teachers' curriculum-building methods.
Yet we know almost nothing about the quality of such supplementary materials. Although several organizations, such as EdReports, have stepped up to offer impartial reviews of full curriculum products, to our knowledge, there's no equivalent when it comes to add-on resources. So three years ago, we, as education researchers, set out to answer a simple question: Are popular websites supplying teachers with high-quality supplemental materials?
A Focus on Quality
We narrowed our evaluation of online educational materials to ones with an ELA focus, in part because ELA teachers are less likely than math teachers to report that their instructional materials are required by their district (Opfer, Kaufman, & Thompson, 2017). One of us (Northern) commissioned and oversaw the study at the Fordham Institute (an education policy organization); the other (Polikoff) conducted the research from the University of Southern California. Experts in assessment, standards alignment, curriculum development, and ELA content helped us develop the rubric by which we evaluated the materials and completed the reviews. Our rubric captured both the overall dimensions of quality in curriculum materials—things like rigor, usability, and supports for diverse learners—and more discrete dimensions from states' ELA content standards, such as regular practice with complex texts and reading and writing tasks grounded in textual evidence.
In all, we wanted to examine the quality of the materials that teachers typically were using or looking for, so we reviewed more than 300 of the most downloaded materials found on three of the most popular supplemental websites: Teachers Pay Teachers, ReadWriteThink, and Share My Lesson.
In a nutshell, our team found more weaknesses than strengths.
First the Good News …
The basic quality of the texts that were the focus of the lessons (such as fiction, nonfiction, poetry, or informational text) was deemed good to excellent, and students were often asked to provide textual evidence when analyzing a text. Materials were also generally free from errors and well designed. On a 0–3 scale—with 0 indicating "major errors likely to affect student understanding," indicating "moderate errors that may or may not affect student understanding," and 2–3 indicating (respectively) "minor errors" or "no or very few errors"—the mean score was 2.75. ReadWriteThink had the fewest errors (mean = 2.92), while Share My Lesson had the most (mean = 2.53) and Teachers Pay Teachers was in the middle (mean = 2.79). Materials were also highly rated in terms of their organization and visual appearance.
Now for the Bad News …
When we dug deeper, we started to find the pitfalls involved in using these online materials.
Overall Quality
Overall, reviewers' capstone or holistic rating for most of the downloaded materials was "mediocre" or "probably not worth using." On a 0–3 scale, with 2 or higher corresponding to materials that reviewers thought teachers should use, the mean score for materials was 1.28, with reviewers recommending that 63 percent of materials not be used at all or deemed "probably not worth using."
None of the websites had a majority of materials earning an exceptional rating overall (see Figure 1), but ReadWriteThink received a slightly higher overall rating on average (mean = 1.41) than Share My Lesson (mean = 1.29) or Teachers Pay Teachers (mean = 1.18). A major contributing factor to the poor overall ratings was the lack of clarity of the guidance offered to teachers. For example, consider a CliffsNotes-style presentation of Kurt Vonnegut's short story, " Harrison Bergeron," which includes the story itself and a guide summarizing the plot, characters, and so on, yet offers no instructions at all for teachers about how to use it. On a 0–3 scale, with 2 intended to represent standard guidance, the mean for instructional guidance across the sites was 1.61.
Alignment to Standards
The materials were also weakly to moderately aligned to the standards to which they claimed alignment. Respondents used a 0–3 scale that ranged from not at all aligned to fully aligned. The average alignment rating was 1.35. Of all the materials, 56 percent scored a rating of 1 (see Figure 2), which technically means "lesson partly aligns to some of the listed standards or fully aligns to a few (but not the majority) of the listed standards."
These low alignment ratings occurred primarily because most materials claimed alignment to a very large number of standards (which boosts their visibility in search results). A unit on The Diary of Anne Frank on Teachers Pay Teachers, for instance, claimed to align to 22 different Common Core standards. With so many standards tagged, it's impossible for teachers to know which one or two standards the material prioritizes and therefore which resources would be most helpful.
Depth of Knowledge
Reviewers also evaluated the materials for depth of knowledge (DOK)—the cognitive demand required for students to successfully engage with the content (Aungst, 2014). Most of the content included in the main activity of each material was DOK level 1 or 2 (see Figure 3). Nearly half of the main activities had no DOK level 3 content at all, and just 6 percent scored higher than a 0 for DOK level 4.
Diversity
Another weakness? The materials did a very poor job of offering teachers support for teaching diverse learners. In fact, the level of support provided for teaching diverse learners garnered the lowest ratings among all of the evaluated dimensions. We asked how comprehensive the supports for differentiation were with regard to meeting the needs of high- or low-performing students, students with disabilities, and English-language learners. A full 86 percent of the materials scored 0 on this dimension, indicating that they offered no such support (see Figure 4).
Finally, materials tended not to reflect the cultural diversity of today's classrooms. Reviewers examined both the choice of authors and the texts themselves relative to their representation of cultural diversity, with a focus on race/ethnicity, gender, and culture/national origin. A majority of materials on all three sites did not include diverse authors or cover culturally diverse topics (see Figure 5).
On a scale of 0–3, 68 percent of materials scored 0, meaning no inclusion. Just 15 percent of materials scored 2 or 3, respectively, meaning moderate or strong inclusion of diverse perspectives, including several authors from diverse groups and/or topics of great diverse cultural importance. The overall mean on this item was 0.53, but ReadWriteThink (mean = 0.62) and Share My Lesson (mean = 0.75) scored much higher than Teachers Pay Teachers (mean = 0.30).
So, What Can We Do?
When we shared these results with the online providers, some took steps to improve. For example, as one type of quality control, Share My Lesson now provides teachers the opportunity to leave reviews of materials after they've used them. However, subsequent improvements have been minimal, and our study's findings still very much ring true. While these online supplemental materials were never intended to fully replace school-based curriculum, districts and schools must take steps on their own to help improve their use. We offer four recommendations.
1. Decide whether and how to monitor the enacted curriculum.
No doubt there are lots of mediocre materials being used in U.S. classrooms, given the poor quality overall of what we evaluated and the fact that it comprises the most popular materials from the most popular websites. School and district leaders should think seriously about how they want to handle this issue.
They could certainly take a hands-off approach, like nearly all of them now do, and continue to allow teachers to arbitrate on their own the good from the bad. Yet providing a quality curriculum is primarily a school-level function. Moreover, the popularity of sites like the ones we studied and the frequency with which they're accessed indicates that these supplementary materials may soon become part of the core curriculum—whether school leaders and department heads realize it or not.
So these leaders are advised, at the very least, to pay more attention to what's actually taught in classrooms when it comes to supplementing. What they learn could inform an array of subsequent approaches, from offering teachers training in how to evaluate and select high-quality materials to publishing a list of curated supplemental resources and explicitly discouraging the use of unacceptable materials.
2. Encourage curation of the online marketplace in your school or district.
It is incredibly difficult to navigate the plethora of supplemental materials, astutely evaluate what is out there, and ultimately make informed decisions about what to use. And yet it is critical to sift through the materials on these sites and separate the wheat from the chaff. Districts or schools could be proactive on this front by creating a committee of educators or even a new staff position tasked with curating supplemental materials and illustrating how they can complement and deepen the existing curriculum. Doing so would help to reduce the burden of searching for and evaluating available materials and improve the quality of materials ultimately selected for use.
3. Look for supplemental materials with soup-to-nuts supports, including stronger assessments and supports for diverse learners.
Our reviewers examined hundreds of materials and found very few that offered substantial supports for differentiation, particularly pertaining to high and low achievers, students with disabilities, and English-language learners. Even modest enrichment supports or scaffolding would be a useful addition to most of the materials we evaluated. The assessments included were also lacking essential content and guidance about what constitutes progress or mastery. If a material is going to serve as a stand-alone unit that teachers can take off the metaphorical shelf, it must provide teachers with greater instructional support. Otherwise, they will be using the inferior supports that come with the units or spending their time creating new supports from scratch.
Thankfully, locating these materials (once they exist!) is now getting some attention. For example, in the time since our research was completed, Share My Lesson added a "Lesson Attributes" search filter, allowing users to identify materials that are available in Spanish, as well as those addressing the needs of English language learners, gifted and talented students, and students with disabilities.
4. Develop a repository of supplemental materials focused on diverse authors and cultural pluralism.
The large majority of reviewed materials made limited to no effort to represent the cultural diversity of America's students. When fewer than 45 percent of K–12 students are white (National Center for Education Statistics, 2019), more curriculum materials—core and supplemental alike—should include materials that are written by nonwhite authors and represent the diverse cultures included in our classrooms or address culturally diverse topics. Although supplemental materials could fill this role nicely, they aren't now doing so. Simply put, there is a need for better sourcing of supplemental materials that focus on diverse authors and cultural pluralism. Websites like Teachers Pay Teachers, ReadWriteThink, and Share My Lesson could help by encouraging or developing more resources along these lines and labeling them as such so that they are easier to find.
A Clarion Call for Improvement
Teachers are on the hunt for quality materials to supplement their core curriculum. Unfortunately, the websites they are using are providing mostly subpar offerings. Given the prevalence of supplementation and the likelihood that standards-based reform will continue to guide state and national policy in the coming years, we hope that district and school leaders—as well as the content creators at these and other websites—will see these findings as a clarion call to improve their materials' quality for the betterment of teaching and learning in America's classrooms.
Why Teachers Supplement
By Amber M. Northern and Morgan Polikoff
To better understand why teachers use supplemental materials—and help districts better meet their resource needs—we conducted interviews with seven educators from four different states who were experienced in accessing online curriculum materials. Five were teachers, one was a literacy coach, and one was a technology coach.
According to our interviews, teachers turn to online supplemental materials for four main reasons:
1. To Increase student engagement
Most often, teachers report searching online sites for new classroom activities or innovative materials that will spark student engagement. Several teachers stressed that students need stimulation and varied instructional approaches in the classroom—and online materials help fill that need.
"I look for things that are more engaging so I can get my kids to buy in a little more. Textbooks are boring a lot of the time …. I'll download fun activities for the first couple weeks of school and end-of-year stuff like review materials for the end-of-year assessment. I look for hands-on, task-y stuff like scavenger hunts, which are really big right now."
—Utah teacher
"I'm looking for activities more and more because that's where teaching is going—there's a push to have kids create and a push to have kids work more in small groups. Therefore, teachers need the ability to do different things. Also, without variety, kids can decide they are bored."
—Utah teacher
2. To meet their students' diverse needs
Teachers search for materials for students who need more enrichment or more practice. Frequently, they are also looking for ways to unpack important concepts for struggling learners. Consequently, they see the opportunity to download off-grade materials—both below and above grade level—as a significant advantage of online sites. Interestingly, they don't expect a specific lesson or unit to include multiple ways to reach diverse learners; rather, they shoulder the task of locating appropriately challenging lessons by searching widely among multiple online offerings.
"I have a set of three units, and they are amazing. Even so, I still need activities to go with these units because I might need multiple ways to reach certain students … or some students may need more practice. When I have those kinds of needs, then I go to Teachers Pay Teachers."
—Vermont teacher
"I'm always looking at college-level and middle-school-level materials to reach my higher and lower students."
—Kentucky teacher
3. To fill instructional gaps
Several teachers mentioned searching for online resources before beginning a literature unit, including searching for deeper analyses and interpretations of a text.
"If I use something online, I'm asking, ‘Does it fit the skill I want to teach?' For example, if my students are having problems summarizing, will this activity reinforce that skill?"
—Texas teacher
"Generally, I look for literature and writing. I may research a particular novel or look for sources for literary analysis. I don't look online every week, maybe once a month or once a unit. Or I might say, ‘What I'm doing right now isn't working? I've exhausted all my ways of trying, so let me look for something new.'"
—Kentucky teacher
"I found some materials to help me teach academic discourse. They show how to have an academic debate, which is definitely lined up well with speaking and listening standards and also aligns with opinion/argument writing."
—Utah teacher and literacy coach
4. To save time
Finally, teachers find a lot of lessons that they too could have developed, had they enough time, but downloading readymade resources saves hours. Teachers reported feeling awkward at first—even guilty—when they started using online lessons and activities, but that feeling was short-lived once they learned how much time they were saving. Even if they have to modify the material, it reduces preparation time, and the cost is seen as well worth the expense.
"I used to think I had to do everything myself and it was wrong to use others' materials. But that becomes exhausting. It's not plagiarism; it's part of the profession."
—Kentucky teacher
"A lot of teachers say, ‘I can't believe you bought it; I can't believe you didn't create your own.' My response would be, ‘It saves me time, saves me stress, and it's about the implementation of the materials I buy.' I don't use them as is; I modify them to meet my needs."
—Utah teacher and technology coach
"Two dollars is a lot cheaper than two days of my time."
Aungst, G. (2014, September 4). Using Webb's Depth of Knowledge to increase rigor. Edutopia.
•
Kaufman, J. H., Opfer, V. D., Bongard, M., & Pane, J. D. (2018). Changes in what teachers know and do in the Common Core era: American Teacher Panel findings from 2015 to 2017. RAND.
Opfer, V. D., Kaufman, J. H., & Thompson, L. E. (2017). Implementation of K-12 state standards for mathematics and English language arts and literacy: Findings from the American Teacher Panel. RAND.
