If you were one of the highest-achieving students when you were in school, taking a standardized test wasn't a painful experience. You generally looked forward to receiving your results; you tended to get almost all the questions correct, and you received accolades for your performance.
Would it surprise you to know that, just as much as students who struggled and received low scores, you were served poorly by these fixed-form tests? That's because, although your test scores may have shown that you compared well with your peers or that you had reached a predetermined "proficiency" cutoff point, they did not give an accurate picture of what you knew, what you didn't know, and what instruction you needed.
Fortunately, many schools have now begun using a different form of assessment that is more useful for students at every ability level: computerized adaptive testing.
What Is an Adaptive Test?
Computerized adaptive testing (CAT), which was first developed four decades ago (Weiss & Betz, 1973; Weiss & Kingsbury, 1984), begins with a large pool of questions and then selects individual questions for test takers, depending on their responses as they go along. As the test taker answer questions correctly, the questions become more difficult. As the test taker does less well, the questions become less difficult (see fig. 1). Computerized adaptive tests require the following components: a pool of questions to draw from, calibrated to a common measurement scale; a mechanism to select questions on the basis of the student's responses; a process to score the student's responses; a process to terminate the test; and reports that relate scores to the student's instructional needs.
Since adaptive testing was first developed, its use has expanded exponentially. It has been adopted by the armed forces, and by the licensure and certification bodies for a variety of professions. It has also become popular for many uses in K–12 education, where it offers the following advantages (Kingsbury & Houser, 1998; Thissen & Mislevy, 2000):
- In most studies, adaptive tests have been found to be as accurate as fixed-form tests that are twice as long. This enables the assessment to have fewer questions and to take less time while still providing good information about student achievement.
- Studies also reveal that adaptive tests drawing from large item pools can provide much more information, and more precise information, than fixed-form tests do about both students who are struggling and students who are excelling.
- Because they are administered on a computer, adaptive tests provide immediate feedback to students and teachers. These instantaneous results help ensure that test data can be used to adjust instruction.
Even with the increased sophistication of today's adaptive testing programs, there are some limitations. The primary one is ensuring that schools have the network infrastructure to successfully implement a web-based adaptive testing model. A related issue is providing the devices to take an adaptive test, such as laptops or tablets. It is the responsibility of both schools and test publishers to confirm that adaptive tests function comparably on multiple delivery devices.
Helping Teachers Help Students
It often feels as though our schools are designed as factories in which the goal is to create a consistent product (the successful graduate) with the same attributes time after time. And our accountability and assessment policies appear to be built to reinforce this factory-style model.
But the purpose of education isn't to create a single model of an adult. It is to help create the next generation of lawyers, mathematicians, firefighters, college professors, bricklayers, political activists, doctors, architects, writers, bakers, and sculptors. It is to foster adults who will make positive contributions to a world that doesn't exist today. (Imagine a school in the 1970s trying to teach a student to be a webpage designer!) If we envision education as meeting each student's academic needs, why wouldn't we devise an assessment system that adjusts to these same individual needs?
That is exactly what adaptive testing does. The advantage of an adaptive test is that it is customized, providing a better measure of achievement by offering questions that are specifically targeted to each student's ability level. High-performing students are not bored by breezing through items that are too easy for them, and lower-performing students are not discouraged by slogging their way through a large set of items that are too difficult. Student engagement increases because students at all ability levels encounter questions that are challenging, yet not insurmountable.
Educators know that, even when students enter kindergarten, they have a wide range of achievement levels. Some of them know how to read sentences, others can recognize a few words, some know their alphabet, and some are unfamiliar with books. We need to strengthen all students' reading skills regardless of his or her starting point. As students move up through the grades, they continue to progress at different rates and to learn some skills more quickly than they learn others. Treating students as if they don't have differences in achievement is not a rational way to help all students succeed. By providing assessment information tailored to each student, adaptive testing enables teachers to better target instructional materials and programs.
Adaptive Tests in Practice
Adaptive tests are becoming more prevalent in school districts across the United States. It is estimated that approximately 7,000 school districts representing 30,000 schools are currently using some form of computerized adaptive testing. The number of school districts using these tests will likely continue to increase as providers develop adaptive assessments that reflect the Common Core State Standards and that can be used to help identify students who are likely to have difficulty in demonstrating proficiency on the new Common Core assessments.
At Mountain View Middle School in Rio Rancho, New Mexico, educators shape teaching strategies classroom by classroom and student by student using data from the adaptive Measures of Academic Progress (MAP) assessment. MAP is administered to all students at the school in the fall, winter, and spring to assess students' understanding of math, reading, and language arts. Both the school's curriculum and its MAP assessment are aligned to existing New Mexico state standards and the Common Core standards.
Mountain View teachers use a team-based, collaborative approach to instruction. Each instructional team includes four core subject teachers and one special education instructor, and computerized adaptive assessment data, along with information from other assessments, play an integral role in their decision making. Mountain View's principal, Julie Arnold, and assistant principal, Veronica Sanders, have built time into the school's schedule to enable teachers to examine student data and collaboratively build responsive instructional strategies. The teachers group students in similar or intentionally mixed-level groupings for appropriate interventions.
Eighth grade language arts teacher Ellee Weaks says,
We look at the strands within the test to see where students are performing the best and the worst comparatively. That helps us design instruction. Over the last couple of years, many students were struggling with the critical-thinking strand. So we as a team made a decision to focus on open-ended questioning and work on student responses to those questions. As a result, student performance on critical-thinking tasks has improved in class and on the test.
In fall 2011, 82 percent of Weaks's students were proficient in critical thinking tasks as measured by district quarterly assessments and classroom final exams. Following the educators' intensive focus on open-ended questioning, the percentage of students demonstrating proficiency rose to 91 percent by spring 2012 and to 94 percent by spring 2013.
Weaks gives another example:
We typically find that higher-performing kids are scoring at levels indicating that they are ready for content they've never encountered outside of the test. This has driven us to do special projects that we wouldn't have otherwise considered and that helped those high-performing students grow further.
Weaks' instructional team gave these high-performing students the opportunity, in small groups, to self-select reading and writing projects that interested them. One group took on the challenge of creating and self-publishing a book together, pushing the boundaries of their individual abilities and practicing successful collaboration.
Similarly, 6th grade mathematics teacher Rebecca Spaeth shifts students from individual desks to groups and back to individual desks on the basis of their performance in particular areas of mathematics. Recently, Spaeth and other math educators used the adaptive tool to identify students who needed additional help with measurement tasks. They found online resources, changed classroom practices, and provided targeted instruction to boost individual and overall student achievement in that area.
Mountain View educators have also used the data from adaptive assessments to encourage students to take greater ownership of their learning. An understanding of the test data is part of the vocabulary of students, teachers, and even parents. Students know their scores—and they also quickly grasp the mechanics of an adaptive test. "When they get an easier reading question," says Weaks, "they know they may need to slow down and reread passages to improve their performance."
"As a parent," says vice principal Sanders, "it's been great to have my kids come home and say, 'Here's how I did today,' and to be able to pinpoint strengths and weaknesses."
A Question of Equity
Although the use of adaptive tests within a school such as Mountain View may seem like just a measurement issue, it is really a strategy that affects the equity of teaching and learning. Students whose achievement isn't measured well because the fixed-form test they took is not well-targeted to their ability level are put at a disadvantage when the results are used to make a variety of decisions, from promotion to inclusion in special programs. If a student's score is inaccurate, that student may be placed incorrectly and harmed as a result. A test that does not adjust to the individual student's ability level has the potential to disempower both the student who is struggling and the student who is excelling.
This brings us back to the high-achieving student who found tests to be easy and got almost all of the answers correct. From the teacher's point of view, these tests said little about what that student needed to learn next. At the same time, every student who was struggling had the same problem. They were facing tests that didn't reflect their current achievement level and thus failed to provide teachers with the information they needed to help those students improve.
As educators, we know that all children have the potential to learn and grow. We also know that students start their education at different levels of understanding, learn in different ways, and bring different levels of interest and motivation to a given task or topic. Through computerized adaptive testing, we now have the technology to change the assessment paradigm, making test results more accurate and meaningful.
Figure 1. A Test Taker's Progression in a 20-Item Computerized Adaptive Test
The sample item progression shown here illustrates how item difficulty is adapted for a student over the length of a 20-item test. The student responds to the first two items correctly. The third item, which is more difficult, is presented and responded to incorrectly. This causes a slight drop in the ability estimate, and the fourth item therefore becomes easier. This process continues until all 20 items are administered. The standard error of measurement (an indicator for measurement precision) shrinks as the test proceeds. Toward the end of the test, at approximately item 10, the test taker's ability estimate starts to stabilize.
