Looking back on the 1990s, advocates of education technology can pat themselves on the back. When that decade began, computer-to-student ratios in the United States typically stood at about 1 to 20. Computers rarely appeared in the classroom; instead, they were relegated to central labs that teachers and students visited sporadically. When students did use computers, they were typically learning basic computer skills or programming. Schools rarely, if ever, made computers an integral part of the content-area curriculum.
By the end of the 1990s, however, things had changed. Nearly every U.S. school had faster and more powerful computers than those that large universities had been able to afford only 10 years before, as well as CD-ROM drives and Internet connections. The computer-to-student ratio was down to 1 to 5, and a greater proportion of computers were actually located in classrooms. More teachers had received enough training in technology to feel confident using computers in instruction.
Yet all was not well in the realm of education technology. Some researchers argued that computers had a deleterious effect on young children's social, emotional, and physical development (Alliance for Childhood, 2004; Healy, 1999). In addition to the critics of computers in the lower grades, some criticized the use of computers in education in general. Most famously, Larry Cuban argued that history was rife with examples of schools requiring teachers to use some new, unproven technology in the classroom; computers were just the latest example (1986). In case studies at various grade levels, Cuban found that advocates' claims about computers' benefits were overstated (2001). Todd Oppenheimer (2003) went further, presenting stories of simplistic, mindless assignments that would never have passed muster had they not had the window dressing of using a computer.
When push comes to shove, however, we must measure the success of instructional technology against one bottom line: Does using technology in schools raise student achievement? Until the late 1990s, comparatively little evidence was available to help us answer this question. Although many studies of so-called “computer-assisted instruction”—self-contained programs that drilled students in arithmetic, reading, and the like—had found positive effects, these effects were rarely replicable on a large scale. During the 1980s, the Apple Classrooms of Tomorrow project attempted to introduce more innovative uses of computers in a set of schools across the United States, but evaluations of the program proved inconclusive (Baker, Gearhart, & Herman, 1993). And states that were pouring large amounts of money into statewide technology plans often found that the increased expenditures did not lead to increased test scores (Virginia Department of Education, personal communication, 2000). The research base for the link between classroom computing and student achievement was shaky, at best.
The NAEP Studies
In 1998, I launched a series of studies examining the link between computer use and student test performance. These analyses used the National Assessment of Educational Progress (NAEP) database, which contains the results of NAEPs administered every year or two to nationally representative samples of 4th, 8th, and 12th graders. In addition to testing students' knowledge and skills in a variety of subjects—including mathematics, reading, science, and history—the NAEP administers surveys to the tested students at all three grade levels and to the teachers at the 4th and 8th grade levels. By looking at the relationship between NAEP scores and survey results, we can measure how various activities in the school and in the classroom—including computer access and use—correlate with student performance.
Earlier studies in this series analyzed data from the 1996 NAEP in mathematics (Wenglinsky, 1998), the 1996 NAEP in science, and the 1998 NAEP in reading (Wenglinsky, 2005) for students in 4th and 8th grade. The study reported here looks at a new area: the results for 12th graders who participated in the 2001 NAEP U.S. history assessment. This assessment measured students' knowledge of U.S. history from the Colonial period through today, as well as their ability to draw inferences and analyze primary documents.
Techno-Byte
Techno-Byte - Technology and Achievement: The Bottom Line
Fifty-seven percent of U.S. children ages 7–17 use a home computer to complete school assignments.
—Children's Partnership, 2005
Results for Younger Students
First, a brief review of the earlier studies. Results from the NAEP assessments in mathematics, science, and reading for 4th and 8th graders indicated that the quality of computer work was more important than the quantity. Students could receive a substantial benefit, no benefit, or even negative consequences from working with computers in the classroom, depending on how their teachers chose to use technology. Using computers to help students work through complex problems, thus tapping higher-order thinking skills, produced greater benefits than using computers to drill students on a set of routine tasks. The fact that computers were most effective when teachers used them to promote higher-order thinking skills is a huge argument in favor of technology; CEOs of major companies say again and again that they need workers who can come up with creative solutions to complex problems.
But these earlier analyses of NAEP survey data also confirmed an unfortunate reality: At this time, teachers were typically not using computers in the most effective ways. For example, fewer than 30 percent of students reported that their math teachers used computers to teach higher-order thinking skills. Most elementary and middle school teachers still lacked training in computer use, and they therefore frequently used computers in the simplest ways—as drilling machines rather than as catalysts of creativity. The challenge for elementary and middle school teachers, then, was to move away from using computers as a kind of modern tablet on which students can do their arithmetic and instead use computers to help students solve problems in the content areas of mathematics, science, and language arts.
Computer Use and Achievement in 12th Grade History
The new study, based on 12th grade students' performance on the 2001 NAEP U.S. history assessment, indicates that the optimal role of technology for high school students is different from that for younger students. In middle school, students profit when their teachers incorporate computers into instruction in ways that promote higher-order thinking in specific content areas. In contrast, high school students need to be able to deepen their thinking and enhance their work products through technology-driven processes that are the same in such diverse subjects as English, history, trigonometry, and physics.
Figure 1 shows how various aspects of instructional technology relate to student performance on the NAEP history assessment. For each student characteristic shown in the figure, the study used a form of regression analysis to arrive at an effect size indicating how strongly that characteristic correlated with student performance on the assessment. An effect size of 15 or above is considered substantial.
Figure 1. Which Characteristics Affect Student Achievement on NAEP History Scores?
Technology and Achievement: The Bottom Line - table
Student Characteristic | Effect Size |
|---|---|
| Socioeconomic status | 36 |
| Amount of time student has used computers in school | -15 |
| Amount of time student has used computers for schoolwork outside school | 16 |
| Extent to which student has used computers for history-specific tasks | 0 |
| Extent to which student has used computers for generic academic tasks | 6 |
0 = No significant effect; 15 or above = A substantial effect.
The author's analysis of the 2001 NAEP database shows how selected characteristics correlate with U.S. history scores.
Students' socioeconomic status was the factor most strongly related to their history achievement. Not surprisingly, students with more affluent backgrounds performed better than less affluent students on the NAEP assessment.
In terms of the quantity of time spent with computers, the results appear to be mixed. The more time students used computers for schoolwork outside school, the higher they were likely to score on the NAEP history assessment. The more time they used computers in school, however, the lower they were likely to score on the NAEP. This finding suggests that the high-quality schoolwork using computers happens outside school and that teachers can make better use of computers by having students complete such assignments at home rather than at school.
- Word processing. Students who are skillful at keyboarding can more easily express their ideas than students who scribble out their homework with a pen and paper.
- Using computers for art projects. Although there may not be any history knowledge involved in creating computer graphics, this activity provides students with a set of conceptual tools that they can apply across subject areas.
- Creating charts, tables, and graphs. These tasks help students think abstractly about economic, social, and physical phenomena.
- Using computers to complete school projects. Experience in planning, implementing, and sustaining a large project—a practice often referred to as “project-based learning”—appears to promote student achievement. An example is developing a Web site that incorporates knowledge from many different subjects, such as a project on global warming that combines economics, physics, and data analysis.
- Using computers to communicate through e-mail and chat groups. This finding may surprise us; presumably, students do not obtain an e-mail address for the express purpose of completing their history homework. But if students have the e-mail address, it gives them the opportunity to discuss readings, homework assignments, and projects in various classes.
The NAEP questionnaire results also suggest that despite all the improvements in access to technology, teachers cannot take for granted that students have sufficient skill in these generic tasks to use computers across the curriculum. Using computers for word processing and school projects was very common among 12th graders (two-thirds of students reported doing each of these to a moderate or large extent). But charting and graphing were rare (fewer than one-fourth of students reported doing this to a moderate or large extent), and 16 percent of the students actually reported not using e-mail at all. More specialized tasks, such as developing a Web page, were also outside the experience of many students.
The Best Role for Technology
Currently, many high school teachers believe that they must come up with unique ways to incorporate computers into their students' learning tasks. These teachers invest a great deal of time and creativity in developing assignments that require students to use computers.
However, the findings of this study suggest that rather than planning lessons around the computer, high school teachers should assume that students will use technology-based tools to address some of their learning tasks. Teachers should not think, “Aha! I will assign a research paper and require students to use the Internet to obtain information.” Rather, teachers should assign a research paper and take for granted that students will use computers in a variety of ways to complete the assignment. This approach mirrors the technology-rich work environment in which many students will find themselves after graduation.
If high schools adopt such a generic technology approach, however, they need to provide technology training to two groups of students outside content-area classes. One group comprises students who lack basic technology skills, such as the 16 percent of students on the NAEP survey who reported that they did not use e-mail. Schools need to offer courses in basic computer skills for these students, giving them the capacity to excel in their content-area courses as well as teaching them the technology skills they will need to obtain any white-collar job in the future.
Another group of students who need additional technology training are those seeking enrichment. For example, students who plan to go into science, mathematics, or engineering after graduation will need advanced computer skills. Thus, schools should make computer science courses available to students who want to learn networking, graphic design, database use, Web page development, and the like.
High schools have been largely successful in creating subject-specific tasks that involve digital technology. But the analysis of NAEP data described here suggests that the greatest boost to student achievement may come when schools take the role of the computer in the classroom one step further and simply ensure that students have the generic technology skills they need to apply this powerful tool to their learning across the curriculum.
