hat does this mean? That used to be our school board's reaction
when presented with a report of our district's performance
on standardized tests. Other than showing how our students
did compared to others across the nation, the tests held little
meaning for us. As board president Virgil Mabrey explains,
"We were having a hard time connecting [the test results]
to the curriculum or using that information to help us improve
it."
That changed when the South Madison Community Schools began
using a new type of testing, custom designed to align with
our curriculum and geared toward measuring individual student
growth. Now, with the help of technology, we will be able
to enhance our testing system so that we can get immediate
results and test students on demand.
Searching for
a better test
We started working with Northwest
Evaluation Association about four years ago. We were unhappy
with the lack of useful information from the results of the
traditional, state-mandated standardized achievement tests.
We believed our teachers could do a better job if they had better
assessment tools that showed individual student results. NWEA,
a nonprofit test developer in Portland, Ore., helped us create
an assessment plan for our district and allowed us to shape
the content and questions of the tests. 
Rather than test grade-level knowledge as traditional standardized
tests do, the new tests provided a more accurate measure of
student achievement, based on a common scale that crosses
grade levels to show student growth. After two years, we were
able to use data from these tests to measure student progress
and the effects of changes in the curriculum. And we can now
answer the school board's question, "What does this mean?"
We score the NWEA tests here in the district with the help
of computer software. This means we get the results much more
quickly than was the case with state-mandated tests and most
other standardized tests. But we wanted to take advantage
of technology to make the turnaround time even shorter. Also,
we were looking for a way to do more mid-year and on-demand
testing.
When NWEA offered the district a chance to serve as a pilot
site for the computerized version of the tests, Measures of
Academic Progress (MAP), we jumped at the opportunity. We
wanted to see if the students' scores would be any different
if they were tested by computer, rather than by paper and
pencil. It was our chance to get immediate test results, as
well as examine what the additional benefits of computerized
tests might be.
Over a two-week period in spring 2000, a dozen teachers took
about 400 first- through eighth-grade students to our in-house
computer lab for testing in reading and math. Students saw
their test "forms" on the computer screen and marked their
answers by clicking the mouse. Each test included the student's
name and coding identifying his or her classroom and program-related
information.
Scores on the computerized tests were calculated immediately
after the exam. Not only were the scores comparable to those
on the pencil-and-paper tests offered by NWEA, but the students
actually preferred taking the test by computer.
Adaptability and efficiency
One of the most important benefits of a computerized test
is its ability to adjust questions on the basis of a student's
answer during the test -- what's called adaptive testing.
How students respond to difficult questions determines what
the following item will be. If the questions are too easy,
more difficult questions are offered, and vice versa. The
test adjusts the question to keep the student appropriately
challenged.
In adaptive testing, the questions are given a numerical
value, with the less challenging questions getting lower numbers
and the more challenging ones having higher numbers. Using
this weighted grading scale makes it possible to compare student
results even though different students have taken different
tests.
With traditional standardized tests, students who aren't
challenged by the questions might be bored or frustrated,
and those who are unable to do well might be frustrated or
lost. In either case, the results really aren't an accurate
measurement of what the students have achieved. And because
the results come back so much later, teachers and schools
aren't able to make timely adjustments based on the findings.
Another important benefit of computerized tests is their
efficiency. With the state standardized tests, administrators
had to scan the test forms into the computer and then compile
the results. The computerized tests do that instantaneously.
For South Madison, a 3,500 student, fast-growing rural district
within 20 miles of Indianapolis, the computerized tests also
hold an added potential benefit: We will be able to do meaningful
assessments on new students entering the district at any time
during the year. When new students register, they can take
the test, and the results can be generated and sent to their
assigned teacher immediately. That way, teachers will know
the instructional level of each new student.
MAP will be useful, also, for assessing our special education
students. We now test these students annually, but with the
relative ease of computerized testing, we can assess their
progress more frequently and adjust our curriculum accordingly.
Critics of computerized tests say they might be a disadvantage
to students who struggle with the technology. Others say that
computerized tests create "standardized" children trained
to fit into a "point-and-click workplace where their job is
mainly to feed into a computer whatever it asks of them."
(See "Electronic Exams," by Kevin Bushweller, in the June
2000 Electronic School.) But from our perspective,
it's difficult to imagine a solution that offers us better
information more quickly to help us target instruction for
each student. Teachers can use the information from an assessment
administered today to make adjustments in the classroom tomorrow.
Computerized testing does require schools to block out time
to use their computer labs. Although this was not a problem
during our pilot program, full implementation will create
a heavier burden on the labs. Districts with fewer computers
per student should take this factor into consideration.
The real payoff of our new tests is that they allow us to
generate and track important data on student performance.
We can obtain individual student growth information from semester
to semester and from year to year. We can look at growth on
a grade-level or district basis. We can see, for example,
whether our reading program is improving student achievement.
The tests have created a different culture in our schools.
Our teachers now ask for help with what they do in the classroom
on the basis of the test results. They have become more flexible
in their teaching in order to reach students who are achieving
above or below their grade level. We are beginning to think
of children as individual learners, rather than as third-,
fifth-, or eighth-graders. And instead of looking at categories
of students, such as remedial or gifted, we look at each student's
performance level and teach to that.
As a result of the information we've gleaned from the test
data, we've started a series of five-day workshops for teachers
on differentiating instruction. Teachers are learning to develop
tiered lessons that present the same material on different
levels. After the workshops, we plan to develop support groups
among the teachers and continually update information.
The test data also allow for more meaningful accountability.
State- mandated tests and percentile scores allow for comparison
from school to school but don't necessarily focus on internal
accountability, or how much students are learning. Once, for
example, the local newspaper chose to rank the state's schools
by their scores on traditional standardized tests. A statistical
fluke showed a gap between two of our elementary schools.
The article generated a "good school, bad school" type of
thinking and upset our students, parents, and teachers, as
well as the school board. Our new computerized tests provide
a more accurate picture. We can answer questions with information
about actual student growth, not just the snapshot information
reflected in the state test results.
We used to figure that if our students scored above the 50th
percentile on the mandated tests, we were doing all right.
Now we're setting higher and more realistic standards. Of
course, we could use NWEA's database of 500,000 students as
the norm if we wanted to gauge how our students are doing
compared with other students and other schools. But we've
found that the best use of the data is to develop our own
norms. Our goal isn't to beat our peers; it's to keep improving
learning among our own students.
We are currently deciding whether to move entirely to computerized
testing. One consideration is the cost factor: The tests cost
$4.25 per student, but each student can take the test an unlimited
number of times. In addition, the MAP tests engage students
and provide comparable results, with the added benefits of
immediate, accurate results and on-the-fly customizations.
Those elements provide the potential for year-round testing
updates, plus improved capacity to track student growth and
act on test results immediately.
It seems likely that the board, community, and teachers,
all of whom have a voice in the decision, will support the
transition. If that holds true, we will be taking advantage
of the computerized tests this spring. The tests won't replace
the state-mandated paper-and-pencil achievement tests, of
course. But they will be invaluable. As board president Mabrey
puts it, "Improved tests are an exciting concept for us. ...
We're still in our infancy here, but light years ahead of
where we were five or six years ago."
James
Coyle is the assistant superintendent of South Madison
Community School Corporation in Pendleton, Ind.
|
