|
When fourth-graders at Carrie Martin Elementary School made
the second highest gains on the 1998 Colorado State Assessment
in reading and writing, state officials wondered how we could
have achieved so much so quickly. A few privately joked that we
must have cheated, but one look at the data showed our changes
were serious and real.
Eighty percent of Carrie Martin students passed the state reading
test, and 65 percent passed the writing test. This compared to
65 percent passing the reading test and 33 percent passing the
writing test the year before. And it compares to statewide averages
of 60 percent and 30 percent, respectively.
Our success is all the more remarkable because more than 25
percent of our students qualify for the free and reduced-price
lunch program, and 22 percent have special education individualized
education plans. (In Colorado, special education students are
required to attempt the state tests. If the tests are too difficult
for a student, a zero is averaged into the school's score for
that student.)
How did we raise our scores so dramatically? We used our assessment
program to measure everything that affected student performance.
Then we changed or cut anything that didn't improve achievement.
Data-driven instruction
Carrie Martin is one of 18 elementary schools in the Thompson
School District, a primarily rural district of 14,325 students
in northern Colorado. Frustrated with the limited information
that standardized tests gave us, district officials began using
Northwest Evaluation Association
(NWEA) achievement-level tests 10 years ago, but we didn't get
serious about "data-driven instruction" until four years ago.
That's when we started relying on pre-assessment and state content
standards to identify student needs and learning styles, then
using that information to plan and implement teaching strategies
appropriate for each child.
Pivotal to our assessment program are the NWEA achievement-level
tests, which have been custom-designed to align with our curriculum
and to predict how students will do on the state tests. We used
the data from the NWEA tests to measure student progress and the
effects of changes in the curriculum. The data also allowed us
to predict performance on the state tests, to encourage students
to do better, and to point out specific areas where they need
to work harder.
Even before we began using NWEA tests, we realized we had been
focusing too much on middle or average students. If we were going
to challenge all of our students appropriately, we needed to raise
our benchmarks and stop "teaching to the middle."
We now try to teach each child at his or her own achievement
level. To measure how we're doing, we test children at their achievement
level -- which isn't necessarily their grade level. An advanced
fourth-grader might take achievement tests at the sixth-grade
level, while a classmate might be tested at the third-grade level.
NWEA helped us set up this system by sending representatives
to meet with a group of teachers from our district. Together,
they drew from NWEA's bank of 15,000 field-tested items to develop
math and reading tests that aligned with our curriculum. NWEA
helped us develop short assessments, called locator tests or placement
tests, to determine at what level each student should be tested.
Charting individual students' growth on achievement-level tests
allows us to focus on each student's needs and progress. Most
students take pencil-and-paper tests in the fall and spring. At-risk
students -- including those who score below the benchmark in the
fall -- also take a computerized version of the tests as a mid-year
assessment. Most students show progress after the fall test, and
they can't wait to tell their parents and teachers about their
success. The mid-year test provides the positive feedback these
kids need, and most are never at risk again.
The achievement-level tests also help us challenge our more
advanced students. For example, when last year's fifth-grade students
broke the previous school record on NWEA math scores in 10 years,
we told them, "We think you can do better." We raised the bar
as high as we could, challenging some students to take tests at
the highest level. Our students rose to the challenge: Twenty
scored above the eighth-grade benchmarks, 17 scored above the
seventh-grade benchmarks, and the majority of our special education
students scored at or above the fifth- and sixth-grade benchmarks.
Crunching the numbers
NWEA provides ongoing help with test administration, scoring,
and data interpretation, which helps us use the test data appropriately
to improve learning. Data are collected and analyzed for the whole
district, for specific schools, for different grade levels, and
for each student.
Detailed test data for each student -- showing the student's
test scores and how they compare to what's expected -- are used
during parent-teacher conferences. For every single student, we
set goals that include what the parents will do, what the student
will do, and what the teacher will do. If a student is below the
benchmark, the student, the parent, and the teacher develop a
personal education plan. Together, they review state assessment
scores, achievement-level test scores, classroom activities, and
a variety of other factors. Then they decide on a plan that might
include tutoring, summer school, or other actions to help the
student succeed.
Data from the achievement-level tests also enable us to provide
accountability information to our broader school community. Every
year, we compile a school profile and an annual report for the
district, the state, and our accountability committee. This report
includes an action plan, an outline of our goals, and a report
on our measured growth. Among other things, it also includes graphs
of our test scores, along with breakouts of the data, such as
how girls scored versus boys.
Members of the accountability committee -- which includes staff,
parents, student council members, and community representatives
-- use these results to evaluate what's working and to recommend
changes. The committee also uses the information to develop surveys
that are sent to parents and teachers for more input.
We analyze the information gathered from these surveys to make
changes at the classroom level. Based on our survey results, teachers
determined what fifth-graders needed to exit our elementary school.
We worked backward, so each grade level was a stepping stone to
the exit requirements. For example, we changed our spelling practices
to improve daily writing. Each grade level was given about 20
words that are considered "no-excuse words." The weekly spelling
test was no longer the only criterion for the spelling grade;
if a no-excuse word was misspelled in a writing assignment, the
student's spelling grade could slip from an "A" to a "C" or lower.
The no-excuse words are cumulative, so students must be able to
spell words required in previous grades, as well as their own.
Once students understood the importance of the no-excuse words,
most learned them well.
Making time for tests
No change comes without problems. Initially, some teachers thought
the achievement-level tests were just another assessment -- and
a big waste of time. Many said, "We're not going to have time
to teach if we have to administer all these assessments."
To address these concerns, the district's assessment director
and NWEA representatives explained how achievement-level tests
are different -- how they would show students' progress over time.
We would be able to see whether our students progressed as much
in grade five, for instance, as they did in grades three or four.
Hearing this caused some additional anxiety among teachers,
who feared they would receive poor evaluations if their students
didn't progress. However, as teachers implemented the tests and
saw for themselves how information from the tests could be used
to improve student learning, the teachers became less fearful.
Still, there's no denying the fact that the achievement-level
tests -- two one-hour sessions in the fall and another two in
the spring -- do take some time away from other activities.
To make time for the tests (and academics), we have made some
sacrifices. For example, we rarely schedule schoolwide assemblies
or activities that pull students out of the regular classroom
and away from core curriculum. Instead, we focus on what we need
to teach to meet our goals. It turns out that this has not been
much of a hardship. Our surveys show that students, parents, and
teachers all want to stay focused on academics.
We decided to give up something else last year -- the Iowa Test
of Basic Skills. Initially, the school board thought we needed
the Iowa test to measure how schools were doing. But as board
members saw how rich the achievement-level test data could be,
they realized that we didn't need the Iowa test and that the time
would be better spent working on areas identified by the NWEA
tests.
Teachers weren't the only ones who were nervous about the tests.
Some parents and students also worried, especially when they realized
that the district tied these test results to high school graduation
requirements. We addressed this concern by educating parents about
the tests and the data when their children enter third grade.
Most parents are amazed to see that we can predict as early as
third grade whether their child is on track for high school graduation.
While this idea scares some parents at first, it also prompts
them to help their child grow academically.
We try to limit anxiety during testing times. We reassure the
students that there's no time limit; we just want to see how much
they've grown since the last test. Students who have had experience
with these achievement-level tests typically look forward to each
testing time. They want to be able to prove what they've learned
and what they know.
Overall, the test administration is not as grueling for a child
as other tests can be. A student can usually complete a test in
45 to 60 minutes. Because reading and math tests are given on
separate days, testing takes two days. Contrast this to the six
days of state testing that leave students mentally and emotionally
drained.
Beyond testing
Of course, no single test can guarantee success. But we've used
the achievement-level test results to work with our entire school
community -- students, parents, classroom teachers, district administrators,
and others -- to measure the effects of different strategies.
By measuring before-and-after test results, for instance, we
found that a strict discipline program, coupled with incentives,
led to higher student achievement. Students who come to class
ill-prepared, for example, or who talk without raising their hands
or don't stay on task, get a check. Students who have fewer than
three checks each quarter are rewarded through recognition, additional
recess time, and other bonuses.
Test scores also improved after we increased homework for all
students -- even for kindergartners. Every night, our students
are expected to write a paragraph and read for 30 minutes. Homework
also includes activities such as going to the grocery store to
estimate how much selected items will cost and to compare that
estimate to the total.
As these activities increased parent involvement, parents have
requested guidance in monitoring their children's efforts. So
we developed a system in which teachers send home a weekly sheet
that tells what each child is doing, gives a status on assignments,
and notes any problems, discipline or otherwise. Parents don't
have to wait until the end of the quarter to know how their child
is doing.
Achievement-level testing allows us to measure the success of
every initiative. These tests keep us on track and allow us to
create higher standards for our students. And, we've found, when
you have higher standards, students rise to meet them.
Keith
Liddle is principal of the Carrie Martin
Elementary School in Loveland, Colo.
(Editor's Note: For a discussion of the technology
of data mining, see "Smart
Data: Mining the School District Data Warehouse," September
1999.)
Assessment experts are just beginning to tap the potential for
achievement-level testing. The next step is to leave paper and
pencil behind and move on to computerized adaptive tests that
measure each individual student's achievement in less time and
with more reliability than anything we've seen so far.
The Northwest Evaluation Association
(NWEA), a nonprofit assessment organization that serves more than
300 member school districts around the country, is in the final
stages of developing an Internet-enabled assessment system that
adapts questions to the performance of each student. When a student
answers questions correctly, the questions become more and more
difficult; incorrect answers lead to easier questions.
The idea is to help students avoid the frustration caused by
too-difficult questions or the boredom resulting from questions
that are too easy. These tests can be shorter -- and take less
class time -- while still providing a highly reliable estimate
of each student's achievement level. Research shows that scores
from an adaptive test are as valid as those from a traditional
test of twice the length.
As with NWEA's paper-and-pencil achievement-level tests, the
computerized tests can be customized according to a school district's
curriculum and state standards. Each test draws from a large,
calibrated pool of questions that vary according to each student's
answers. No test items will be repeated for a student who takes
the test more than once.
These adaptive tests can be designed for both PC-based and Mac-based
networks, which enables schools to give tests to whole classes
of students and transmit results for scoring and analysis. Typically,
these computerized adaptive tests cost less to administer than
conventional standardized tests and eliminate the cost of test
booklets and materials handling. Because test administrators can
connect to a testing service and download appropriate testing
information for each student as needed, tests can be kept secure.
The new computerized system, now being tested in five school
districts, will soon be available nationwide. For more information,
check out NWEA's web site.
Allan Olson
is executive director of the Northwest Evaluation Association,
a nonprofit assessment organization in Portland, Ore., that serves
more than 300 school districts nationwide.
|
