April 2004

Teaching to the Test

By Lloyd Bond

A recurring criticism of tests used in high-stakes decision making is that they distort instruction and force teachers to "teach to the test." The criticism is not without merit. The public pressure on students, teachers, principals, and school superintendents to raise scores on high-stakes tests is tremendous, and the temptation to tailor and restrict instruction to only that which will be tested is almost irresistible.

Although many view teaching to the test as an all or none issue, in practice it is actually a continuum. At one extreme, some teachers examine the achievement objectives as described in their state's curriculum and then design instructional activities around those objectives. This is done without regard to a particular test. At the other extreme is the unsavory and simply dishonest practice of drilling students on the actual items that will appear on the tests.

In addition to offending our moral sense, teaching the actual items on a test (what James Popham calls "item teaching") is counter-productive for the very practical reason that it makes valid inferences about student achievement almost impossible. There is nothing special about the set of words that happens to appear on a given vocabulary test. We assume that the words are a sample from a larger population of words, and we want to infer something about the students' knowledge of this larger set, their general vocabulary. In like manner, we want to infer that students can solve not only the particular set of math problems on a test, but that they can solve an entire class of problems. Drilling students on a specific set of test items destroys our ability to generalize to this larger domain.

But is teaching to the test all bad? Emphatically not. Consider the coach who drills young athletes on the very skills they will perform in competition, or the typing instructor who teaches students precisely the finger arrangements and keystrokes that will be used in typing. These practices are not seen as unethical or unsavory for the simple reason that in these two domains instruction and assessment merge into a single activity. Indeed, instructing students on anything other than the actual test itself seems illogical.

The above two examples are so obvious as to be trivial. But more significant illustrations of the issues are easy to find. In the ambitious New Standards Project, a national initiative that regularly brought teachers together from around the country to learn techniques for integrating instruction and assessment, participating teachers learned to literally merge these two activities in such a way that they were indistinguishable. Lauren Resnick of the University of Pittsburgh, one of the visionaries behind the project, noted that rather than bemoan the inclination to teach to the test, we should take advantage of it. We should make exercises so compelling, and so powerful as exemplars of a domain, that honing one's ability to solve them represents generalizable learning and achievement. Viewed in this light, teaching to the test is no longer vaguely disreputable because the skills and knowledge are themselves general and are the very things we wish students to acquire.

In his senior level psychology course on learning at the University of Nebraska, professor Dan Bernstein (now at the University of Kansas) was disappointed in the level of understanding of key concepts that his students displayed. He decided that the fault might not lie entirely in his students, but in the way he approached both instruction and assessment. Over the next few years, he changed his assessment from short abstract essay questions to problems that asked students to apply concepts in new contexts; added out-of-class questions about the readings to free up class time for discussion; and provided web-based examples of responses to test problems, so that students could learn to identify what makes some answers better than others. In short, Professor Bernstein merged instruction and assessment in such a way that "teaching to the test" became an integral part of his craft. The reader is invited to examine his approach in detail at his online teaching portfolio.

In its program of advanced teacher certification, The National Board for Professional Teaching Standards encourages certification candidates to practice putting together portfolios. They urge candidates to get suggestions and critical feedback from their colleagues and from others who have gone through the process. Candidates are encouraged to study excellent teachers and how they think, write about, and reflect upon their work. The National Board advises candidates to take several videotapes of their own teaching, to think about and write critically and reflectively about what they see. Teachers are encouraged to anticipate the difficulties students will have with various concepts and how to structure and sequence instruction to minimize these difficulties. In essence, the National Board encourages teachers to practice and hone the very things they will be tested on.

There is a lesson here for teachers and assessment specialists alike. The tension between the instructional and assessment communities, as well the pejorative connotations that "teaching to the test" entails, will continue unabated so long as testing and assessment are seen as something quite apart from instruction and learning, rather than an integrated reflection of what was intentionally taught. To paraphrase A. G. Rud of Purdue University, what is needed is a deliberate attempt on the part of all parties to link curriculum, instruction, assessment, and standards in a more generative and even transparent way.