Please reject the Children at Risk school rankings
Unfortunately for all of us that care about public schools, Children at Risk is back to releasing more of their junk science in the form of school rankings. They use test scores and graduation rates and a few other variables to create what they claim is a more accurate picture of how schools compare than what the state does. Newspapers all over the state just parroted their nonsense.
And here’s why its nonsense: they don’t—and this is unfathomable to me given their position as a research and advocacy organization—have a clue what a standardized test is, what the scores mean, or the limitations on their use. They, like so many others, think that once a score exists it is free to be interpreted in any manner convenient to the interpreter. That is bogus.
A standardized test score allows a researcher to analyze the differences in total literacy and numeracy attainment between children. However, absent other research, it does not allow for an analysis of what caused that literacy or numeracy attainment. Good educational researchers are aware of this and thus would never—and I mean never—use standardized test data to make inferences that could not be supported by a body of evidence. They would certainty never offer up a crude interpretation that drew lines in the sand and assumed definitive meaning on one side or the other. And they would never make the mistake of presuming that a high or low score on its own could signal anything.
All those mistakes were made by Children at Risk. They seem to have presumed that because you can perform math on the numbers the results are meaningful.
Here’s the thing: a test score has never been an automatic signal of anything. All standardized test scores signal a result that has multiple effects or causes. School will always be one of them. So will the world outside schools (as but one other example), and the non-school effects are often as powerful if not more powerful than what happens in a school. If you try and pass judgment on the signal without knowing the cause, you risk thinking you are judging one when you are in fact judging the other. That renders the judgments invalid. I can’t assign all the credit for an effect to one cause just because its convenient to do so when I know there is more than one.
So, ignore the Children at Risk rankings. As well intentioned as they claim to be and perhaps are, their work does not contain the truth about our schools. Furthermore, they contain nothing that anyone should act on. And that is sad given the size of the effort. We need the truth about our schools. We need to understand where they are effective and where they are not, and we should champion systems that help us with those understandings. This is not one of them. Please reject it.
And here’s why its nonsense: they don’t—and this is unfathomable to me given their position as a research and advocacy organization—have a clue what a standardized test is, what the scores mean, or the limitations on their use. They, like so many others, think that once a score exists it is free to be interpreted in any manner convenient to the interpreter. That is bogus.
A standardized test score allows a researcher to analyze the differences in total literacy and numeracy attainment between children. However, absent other research, it does not allow for an analysis of what caused that literacy or numeracy attainment. Good educational researchers are aware of this and thus would never—and I mean never—use standardized test data to make inferences that could not be supported by a body of evidence. They would certainty never offer up a crude interpretation that drew lines in the sand and assumed definitive meaning on one side or the other. And they would never make the mistake of presuming that a high or low score on its own could signal anything.
All those mistakes were made by Children at Risk. They seem to have presumed that because you can perform math on the numbers the results are meaningful.
Here’s the thing: a test score has never been an automatic signal of anything. All standardized test scores signal a result that has multiple effects or causes. School will always be one of them. So will the world outside schools (as but one other example), and the non-school effects are often as powerful if not more powerful than what happens in a school. If you try and pass judgment on the signal without knowing the cause, you risk thinking you are judging one when you are in fact judging the other. That renders the judgments invalid. I can’t assign all the credit for an effect to one cause just because its convenient to do so when I know there is more than one.
So, ignore the Children at Risk rankings. As well intentioned as they claim to be and perhaps are, their work does not contain the truth about our schools. Furthermore, they contain nothing that anyone should act on. And that is sad given the size of the effort. We need the truth about our schools. We need to understand where they are effective and where they are not, and we should champion systems that help us with those understandings. This is not one of them. Please reject it.
Comments
Post a Comment