A new and improved way to fail

Why the standardized tests of the future won't be quite so standard anymore

Does the term "standardized test" remind you of those dreadful times of scrambling for number-two-pencils and filling endless bubbles? Well, soon, you may dread these mind-numbing tests for an entirely different reason-the frustration of facing the blank computer screen.

JUSTIN CHEN/YH

In this age when "Microsoft-imperialism" and "http://www.anythinginimaginable.com," it's little surprise that computers are breaking up the pencil-and-paper monopoly on standardized tests. The GRE, the crucial stepping-stone to graduate school, has been offered on the computer since 1993. The paper-and-pencil GRE is in fact already obsolete—the computer-based version is the only format Educational Testing Services (ETS) offers anywhere in the U.S.

The computer-based test, however, is not just an onscreen version of the paper exam. You'll still be frustrated over multiple choices and secretly hoping you could toss your lucky penny, but the way you'll be scored is completely different.

The new system, adaptive testing, adapts to the abilities of each test-taker rather than being a uniform, one-size-fits-all examination. You are given one problem at a time. You choose the answer and it is final. There is no room for correcting. You get immediate feedback. If you answered it correctly, a new problem of a higher difficulty level is given. If the answer is incorrect, the next question is easier.

Everyone starts out with a question of medium difficulty. But after that, the series of questions for each person is different. Since most people would answer some questions right and others wrong, the level of difficulty follows a zig-zag pattern. Each right answer lifts up the difficulty and each wrong one bumps the level down.

To most of us, who grew up with more "archaic" technology, adaptive testing may be confusing. What's more complicated, however, is how this test is graded. In fact, the raw grade is completely meaningless. Theoretically, everyone, regardless of ability, will get 50% of the questions right. The difficulty level is adjusted each time such that the test-taker has half the chances of answering the next question correctly.

With all of its complications, not surprisingly, when the computer-test was first introduced, most remained wary of the sweeping change and loyal to the old-fashioned exam. Many expected a significant drop in their scores because of the every-answer-is-final-policy and the dread that the immediate feedback may be discouraging. But studies have shown the scores on computer-based, adaptive tests, have not decreased. In fact, in recent years, GMAT scores—also offered solely in a computer-version— have been rising.

Why the increase? The most likely reason is that college students in general are acultured to thinking and analyzing in front of the computer screen. Most of us have never suffered pains of writing a paper with the clunky typewriter. With emails, instant messaging services, and the enormous numbers of hours spent online to killing typical college boredom, we find it natural to work in front of the computer screen. Some studies actually show that low-tech, hand-written exams actually underestimate the abilities of the computer-dependent generation of students (Haney; Russell; The Christian Science Monitor).

Many students, however, still hold reservations towards the high-tech conversion of exams. "I would think twice before agreeing to take a computer-based exam," said Michael Andrew, Dartmouth, '01, "we have been trained through 12 years of school to take tests with paper and pencil. Taking the test on a computer seems almost unreal—like playing an academic-video game."

Critics of computer-testing argue that the technology itself is a built-in source of discrimination. If computer-adapted students benefit from the format, then students with little background in computer usage would suffer from the transition. Minority students, students from lower income families, older test-takers, and particularly international students—who have less exposure computers—may expected a wider gap between the scores of the two formats of exams. Elizabeth Mead, the Assistant Director at the Office of International Students & Scholars, agrees that computer-based testing is advantageous for those who grew up in the digital age. "But a lot of international student I advise have spent their undergraduate years in the U.S. They are been submerged into the American technology culture. The computer-based tests are not discriminatory towards them," said Mead.

Qinan Tang, BR '03, an international student from China, explains that in her home country, the low-tech, paper exam is still the most prevalent format. Those international students who spent their college years in their home nations still have the option to take the traditional test.

So what's next? Will the humdrum of computers soon replace the scratching of pencils when taking the SAT, ACT, LSAT, and MCAT? For the SAT and ACT, the obstacle in the high-tech conversion is the sheer number of test-takers each year and the still limited computer accesses in many secondary schools across the nation. Anita M. Sharif, the center manager of the New Haven branch of Kaplan, believes that the MCAT may not be as suitable for computer-testing as the GRE or GMAT. For the essay section of the test, students with poorer typing skills may be at a disadvantage. Also MCAT questions often involve more scratch work, which is most convenient in paper exams.

Sharif explained that the student feedback at the Kaplan center regarding the computer-based exam have been positive. "With just one question on the screen, you focus your attention on one problem at a time," said Sharif, "In paper exams, some test-takers get unnecessary and damaging anxiety from the sheer number of questions on the page or from the difficulty level of those questions towards in end of the test."

Some common complaints focus on the fact that the new digital test costs nearly twice as much as its predecessor. When the computer-based test was first introduced, long waiting lists and over filled computing centers caused headaches to both the test-takers and administrators. Confusion over how the paper and computer-based test scores should be compared is also a major concern. Despite all the complicating factors, ETS maintains that adaptive testing is more precise in measuring individual achievement.

What are some sure-win testing strategies? Anita Sharif from Kaplan explains that "first and foremost, learn to manipulate the computer." Even if computer access is unavailable to you, you may make appointments for free practice session at many ETS computer testing-centers.

Never skip questions. A skipped question is considered to be a wrong answer, which will bump you down to a lower difficulty level question. This may bring down your overall score even if you answer the subsequent questions correctly. Know which part of the question to focus on. Ricken Patel, a graduate student in Harvard University's public-policy program, explains despite having taken the paper-version of the test before, he ran out of time when taking the computer-based exam (Guerncy, The New York Times). This is the case for many test-takers. Wasting time scrolling up and down to look for the crucial parts of a question is definitely not good testing strategy.

Finally, pacing and prioritizing is the key to a high score. Since a few initial wrong answers would lead to a train of easier questions, one should pay special attention to the first ten or twenty questions. Otherwise, you may find yourself answering easy questions that may inaccurately reflect your abilities.

Back to News...