Thursday, February 12, 2015

The prevailing wind that the only thing worse than having standardized tests is not having them

I ran into an Ed Week editorial this morning (click here) which again exemplifies how few people actually know what a standardized test is, what it was designed to do, and as a result what the limits are regarding possible interpretations. The editorial starts this way: "What's worse than annual standardized testing? Not having it at all."

In response I posted the following:

What would truly be helpful is for folks to understand what a standardized test is, what it was designed to do, and what the limits are to the interpretive range of a test score. That understanding would quash most of the argument here.

A standardized test is designed to offer a rank ordering of students that can then be used for comparative purposes. In order to provide a consistent rank ordering the design of the test requires that we sacrifice any ability to answer questions regarding why a student lands at a particular score.

Thus a standardized test score, by design, is incapable of offering any information as to what caused it to come to be. Good teaching? A high or low SES? A rock solid curriculum? Quality of any kind? Or a lack thereof? The test scores from any of the standardized tests offered by any of the fifty states and used for accountability are designed to shrug when such questions are asked, point to their offer of a comparative lens, and be forever silent on everything else.

And lest we split hairs here every state utilizes the same test-building methodologies used in the standardized tests prior to state standards, regardless of whether they name it a criterion-referenced exam or a standards-based  exam or whatever. Every state offers a technical manual on their instruments and the statistical underpinnings by which they are judged are based upon the methodology we use to produce a rank ordering, which by design renders them useless for any other purpose beyond presenting a comparative lens.

Pretending that such tests can transmogrify into something that was never a part of their design is just plain dangerous. For as long as I can remember no one seems to much care about that fact and thinks these things can say anything and everything we might ever want them to say. In the meantime, large numbers of policy makers and even a good number of educators continue to be blind as to how we might actually make things better, thinking they have answers when in fact they don't.