What our response to NAEP scores should be (and it isn't pearl clutching)
The response to a decline in NAEP scores was as predictable as the fact that I will be a year older one year from today: the sky is falling, learning loss is real, we should have kept kids in the classroom (or insert the second guess of your choice instead), oh want are we to do? I could go on, but I’d rather point out what that response is really saying: someone designed an educational system that isn't ready to serve kids in whatever circumstance we find them.
In most organizations, disruptions are treated as a natural part of existence, something that will create a new reality against which the organization must constantly reinvent and update itself to succeed, or, in a great many situations, continue to exist into the future. As the adage goes, adapt, or die. The leaders of healthy organizations know this and go to great lengths to keep their organizations as nimble as possible so that the inevitable disruptions—even large ones—are just moments to rethink what the organization must be and do to remain relevant.
Schools we’re not designed to be nimble organizations. Educational reformers (a phrase I cannot stand in the least) alongside policy makers have create a standardized world designed to lock-step students into a thirteen year regimented sequence of events. Policy makers did this as a measure of control, wanting to standardize a profession they did not trust or believe in but needed to rely on (and it was a lot cheaper than the more effective solution of elevating education to the high professionalism it deserves). Reformers did it in the name of equity, thinking that a standardized education delivered to all students would equate with a sense of fairness that had long been lacking in the system.
Both were dead wrong, even as it produced a meeting of the minds across policy and political lines.
That standardization created a system rife with averages: average curriculum, the assumption of an average student, average test scores, average movement of test scores from one year to the next, average resources, etc. There is nothing wrong with average as the central tendency of a data set, but it is a terrible tool for making decisions or running a system. There is no such thing as average. No average person, average dog, average house.
The Air Force once tried to build a cockpit for the average pilot. They measured all the pilots, took the average of those measurements, built a cockpit, and then watched pilot after pilot die in that cockpit. Why? Exactly zero actual pilots matched the the resulting cockpit, which meant they built a cockpit that fit no one by trying to build something that would fit everyone. They built a system that couldn’t handle even the slightest anomaly.
Which is what policy makers and reformers have done to education. We have a system that wasn’t designed for most (any?) of the students that walk in our doors. We have a system that may have been built with good intentions (reformers) or to simply control what educators do (policy makers), but regardless, it was designed generically to apply to everyone, and therefore risks fitting no one.
Sometimes that can be hard to see. The pilots who were at least close to resembling the “average pilot” probably had better outcomes than those who were a good way from it, and students who seem to be closer to the hypothetical “average student” will have better outcomes than those who are far from it. But limited success with a design isn’t evidence of a good design. That’s easy to see if you imagine pegging the design on a position somewhat above or somewhat below average, which would produce a parallel result but with different people living or dying or learning or not learning. It would be just as bad, but for someone else.
Such systems are intolerant to anomalies while pretending otherwise by having “counted” the anomalies in their designs. But the proof is in the results. Oddly shaped pilots who were included in the calculations of average died, and students who are outside the standardized sense of average but were considered in determining what normal looked like will fail for falling into the margins and not the mainstream.
Although these generic systems don’t produce great results for lots of folks, generic systems, by nature of being universal in their application, strongly oppose, even fight change. When a thing can be said to have been designed for everyone change risks detracting from the universality and thereby excluding someone, which would be unacceptable. Universal things thus present themselves as unavailable for change or even criticism given that they are universal. Even when “universal” is defined as “average” or “generic,” and is standardized and thus is anything but universal.
And that is what brings me back to what I find so frustrating about those responses to NAEP scores.
Those response are an admission that those who see the only approach to public education as a standardized system have rendered themselves incapable of having a clue as to what to do. There’s an ugly truth in there: if you are truly committed to a standardized, genericized, system then as the pundits suggest our kids are probably screwed. If all you can imagine is a system that standardizes an education at each of grades 1-12 and then standardizes even the instruction within a grade, then you lack the necessary tools to deal with even the slightest of crises, let alone a two year pandemic.
If that standardized approach is all you know, and that standardized approach is interrupted for any reason, which it obviously was, what appears to remain ahead of you is to squeeze two years of missed standardized work into the time that remains with a student, but as next year and the year after and the year after that are already planned out and standardized, finding opportunities to squeeze in even a little extra, let alone two years worth of stuff, will appear impossible as the schedule is as full as this sentence is long.
The pandemic, however, affected everyone, including those who have historically succeeded in the standardized system, which is helping it get lots of attention. If this only affected the students impacted by prior disruptions, I doubt we’d be having this conversation.
But we are having this conversation so we should take advantage of that fact and finally just say it: the design of our approach to public education, based as it is in standards and standardization and averages will always only work for a few, but because it can be declared as universal, any failure is not the fault of the system, but the educators who work there. False though that is, it represents a surprising amount of thinking of both the policy maker and the reformer.
What I hope comes out of this is that we can finally acknowledge that we created a bad system and desperately need a new one. That standardizing education, whether in the name of control or equity, creates a system that is designed for no one, and that a better way has to be out there.
Let me propose a thought for that better way. Imagine that rather than treat literacy (or any other critical skill) as a series of lock-step lessons dictated by a standards document and end of year testing (standards standardize by the way—that is what they are for), we treated it like a benefit of education. It would be our job as educators to ensure that benefit for every student, no matter where they lived or how many disruptions occurred in their lives.
Our lives as educators would consist of a constant set of adjustments and innovations as we worked to respond to the ever changing nature of the world and the constant effort to keep up with student need, so that addressing disruptions wasn’t an occasional thing when it comes to our job of creating literate students, but a daily one.
If that was where we were as a system and another pandemic came along, we would have the tools to change the system in response to it, to redesign, rethink, innovate, get clever and creative, and figure what we would need to do to deliver the same amount—or more—of the benefit going forward into the new world as we did in the old. We wouldn’t have to wring our hands or clutch at pearls and no one need coin self-serving phrases such as “learning loss.” Rather, we would already be accustomed to the fact that part of our daily routine consisted of inventing anew what we do to satisfy the benefit it is always our job to deliver in an ever changing world.
Another pandemic would of course put that redesign effort on steroids and task all of us with challenges that would at times seem insurmountable, but in the end we wouldn’t sit around wondering how we can squeeze two years of standardized instruction we were unable to deliver into the time we have left with a student. We’d instead spend our time focused on the students and their need for a benefit it is our job to deliver.
In most organizations, disruptions are treated as a natural part of existence, something that will create a new reality against which the organization must constantly reinvent and update itself to succeed, or, in a great many situations, continue to exist into the future. As the adage goes, adapt, or die. The leaders of healthy organizations know this and go to great lengths to keep their organizations as nimble as possible so that the inevitable disruptions—even large ones—are just moments to rethink what the organization must be and do to remain relevant.
Schools we’re not designed to be nimble organizations. Educational reformers (a phrase I cannot stand in the least) alongside policy makers have create a standardized world designed to lock-step students into a thirteen year regimented sequence of events. Policy makers did this as a measure of control, wanting to standardize a profession they did not trust or believe in but needed to rely on (and it was a lot cheaper than the more effective solution of elevating education to the high professionalism it deserves). Reformers did it in the name of equity, thinking that a standardized education delivered to all students would equate with a sense of fairness that had long been lacking in the system.
Both were dead wrong, even as it produced a meeting of the minds across policy and political lines.
That standardization created a system rife with averages: average curriculum, the assumption of an average student, average test scores, average movement of test scores from one year to the next, average resources, etc. There is nothing wrong with average as the central tendency of a data set, but it is a terrible tool for making decisions or running a system. There is no such thing as average. No average person, average dog, average house.
The Air Force once tried to build a cockpit for the average pilot. They measured all the pilots, took the average of those measurements, built a cockpit, and then watched pilot after pilot die in that cockpit. Why? Exactly zero actual pilots matched the the resulting cockpit, which meant they built a cockpit that fit no one by trying to build something that would fit everyone. They built a system that couldn’t handle even the slightest anomaly.
Which is what policy makers and reformers have done to education. We have a system that wasn’t designed for most (any?) of the students that walk in our doors. We have a system that may have been built with good intentions (reformers) or to simply control what educators do (policy makers), but regardless, it was designed generically to apply to everyone, and therefore risks fitting no one.
Sometimes that can be hard to see. The pilots who were at least close to resembling the “average pilot” probably had better outcomes than those who were a good way from it, and students who seem to be closer to the hypothetical “average student” will have better outcomes than those who are far from it. But limited success with a design isn’t evidence of a good design. That’s easy to see if you imagine pegging the design on a position somewhat above or somewhat below average, which would produce a parallel result but with different people living or dying or learning or not learning. It would be just as bad, but for someone else.
Such systems are intolerant to anomalies while pretending otherwise by having “counted” the anomalies in their designs. But the proof is in the results. Oddly shaped pilots who were included in the calculations of average died, and students who are outside the standardized sense of average but were considered in determining what normal looked like will fail for falling into the margins and not the mainstream.
Although these generic systems don’t produce great results for lots of folks, generic systems, by nature of being universal in their application, strongly oppose, even fight change. When a thing can be said to have been designed for everyone change risks detracting from the universality and thereby excluding someone, which would be unacceptable. Universal things thus present themselves as unavailable for change or even criticism given that they are universal. Even when “universal” is defined as “average” or “generic,” and is standardized and thus is anything but universal.
And that is what brings me back to what I find so frustrating about those responses to NAEP scores.
Those response are an admission that those who see the only approach to public education as a standardized system have rendered themselves incapable of having a clue as to what to do. There’s an ugly truth in there: if you are truly committed to a standardized, genericized, system then as the pundits suggest our kids are probably screwed. If all you can imagine is a system that standardizes an education at each of grades 1-12 and then standardizes even the instruction within a grade, then you lack the necessary tools to deal with even the slightest of crises, let alone a two year pandemic.
If that standardized approach is all you know, and that standardized approach is interrupted for any reason, which it obviously was, what appears to remain ahead of you is to squeeze two years of missed standardized work into the time that remains with a student, but as next year and the year after and the year after that are already planned out and standardized, finding opportunities to squeeze in even a little extra, let alone two years worth of stuff, will appear impossible as the schedule is as full as this sentence is long.
The only possible outcome in such a scenario is captured in the phrase “learning loss” (which I suspect was coined by a testing company), which put more simply is yet another admission that we lost some standardized time and don’t know how to get it back because we can’t see past the standardization (or in the case of the testing company, they don’t want to, because it is only within a standardized world that their product is sellable).
Here is arguably the simplest assertion I have ever made: students aren’t going to learn as much during times of disruptions as they are in times that are more normal. Here’s something else: the pandemic—which is pretty sure to be the biggest disruption we’ll see in our lifetimes—is still only one of the disruptions these kids will experience that the standardized system is poorly equipped to respond to. Every disruption of the current standardized system pushes positive outcomes for a student further and further away because the standardization is interupted.
Here is arguably the simplest assertion I have ever made: students aren’t going to learn as much during times of disruptions as they are in times that are more normal. Here’s something else: the pandemic—which is pretty sure to be the biggest disruption we’ll see in our lifetimes—is still only one of the disruptions these kids will experience that the standardized system is poorly equipped to respond to. Every disruption of the current standardized system pushes positive outcomes for a student further and further away because the standardization is interupted.
Historically, and sadly, most of those disruptions prior to the pandemic happened mostly with kids who were largely south of “average” and were thus easy to dismiss or ignore as their failure was predictable and expected and the effects of their disruptions could be dismissed as unrelated to schooling.
The pandemic, however, affected everyone, including those who have historically succeeded in the standardized system, which is helping it get lots of attention. If this only affected the students impacted by prior disruptions, I doubt we’d be having this conversation.
But we are having this conversation so we should take advantage of that fact and finally just say it: the design of our approach to public education, based as it is in standards and standardization and averages will always only work for a few, but because it can be declared as universal, any failure is not the fault of the system, but the educators who work there. False though that is, it represents a surprising amount of thinking of both the policy maker and the reformer.
What I hope comes out of this is that we can finally acknowledge that we created a bad system and desperately need a new one. That standardizing education, whether in the name of control or equity, creates a system that is designed for no one, and that a better way has to be out there.
Let me propose a thought for that better way. Imagine that rather than treat literacy (or any other critical skill) as a series of lock-step lessons dictated by a standards document and end of year testing (standards standardize by the way—that is what they are for), we treated it like a benefit of education. It would be our job as educators to ensure that benefit for every student, no matter where they lived or how many disruptions occurred in their lives.
Our lives as educators would consist of a constant set of adjustments and innovations as we worked to respond to the ever changing nature of the world and the constant effort to keep up with student need, so that addressing disruptions wasn’t an occasional thing when it comes to our job of creating literate students, but a daily one.
If that was where we were as a system and another pandemic came along, we would have the tools to change the system in response to it, to redesign, rethink, innovate, get clever and creative, and figure what we would need to do to deliver the same amount—or more—of the benefit going forward into the new world as we did in the old. We wouldn’t have to wring our hands or clutch at pearls and no one need coin self-serving phrases such as “learning loss.” Rather, we would already be accustomed to the fact that part of our daily routine consisted of inventing anew what we do to satisfy the benefit it is always our job to deliver in an ever changing world.
Another pandemic would of course put that redesign effort on steroids and task all of us with challenges that would at times seem insurmountable, but in the end we wouldn’t sit around wondering how we can squeeze two years of standardized instruction we were unable to deliver into the time we have left with a student. We’d instead spend our time focused on the students and their need for a benefit it is our job to deliver.
Given the choice, that’s what I would choose.
Comments
Post a Comment