An Educational Contrarian

Metrics-driven accountability systems absent context will always fail. Big statement, I know, but there’s lots of evidence for this fact, including the history of how school accountability has been done for the past 20-30 years. The fact is that metrics are technical tools that require expert interpreters and context, and that crude interpretations absent expertise is a recipe for invalid conclusions. Let me show you why. Take one of the simplest metrics I know: graduation rates. Which school is better at graduating kids: a high school that graduates 99% of its kids, or one that graduates 65%? The prevailing wisdom says that goes to the school that graduates 99% of its students. But is that actually true? The answer: you can’t know absent expertise and context. And if you think you can you are flat out wrong. Take the 99% school. Imagine what that school probably looks like: middle or upper middle-class incomes, parents who are highly educated, minimal violence on the streets, most kid...

Eddie Dean, a restaurateur and philosopher in Dallas, occasionally sends me notes that make me think. This morning it was about open verses closed systems. He’s sent me similar notes before since this is a topic we’ve frequently discussed. And over the past few weeks I’ve spent considerable time with my friend, George Thompson, talking about the difference between a learning organization, which is an open system, and bureaucracies, which are closed systems. It's always interesting to me how different conversations with very different people will coalesce like that. Of course, my own work is about accountability, which has a considerable opinion on these topics as well and is the lens I choose. Accountability in a closed system is about the rules. Increasing accountability means finding new ways to hold those within the organization to account. We have two massive closed systems that society is asking some pretty serious questions about at this very moment: policing and attitu...

We are all of us cousins. Every human being on the planet. It may not be through a long-lost aunt or a great grandfather, but it’s probably not much more than a great great great grandparent. That’s remarkable. If you believe in science, we all come from a common ancestor from 200,000 years ago and our ancestors’ paths have probably crossed multiple times since. If you believe the world came into being six or seven thousand years ago—I don’t but I’ll grant it for the moment—then our common ancestor is even more recent, and we’re more cousins than ever. We need to acknowledge this, that we’re all related, all one family. Millions of our cousins around the world are sick. Millions more are impoverished and living in slums. Millions are governed by cousin tyrants who don’t seem to care about their extended family, or forgot they are part of one. And many millions more continue to wonder why bombs and militaries are more important to some than food, healthcare, education, and children. A f...

Being accountable to any test result is being accountable to the wrong thing. Right now, the most important test in the world is for the Coronavirus. The information it provides is immensely useful, and yet to treat that information as more than information about the presence or absence of the virus is a mistake. Neither outcome tells us anything about a person’s overall health. Neither outcome signals anything about what has happened or what will happen. And both outcomes come with a caveat—there is a small possibility of the result being wrong, of suggesting you have it when you don’t, or that you don’t when you do. To treat either outcome as more than it is absent contexts, details, and a whole lot of additional information renders any next step invalid, likely to be unhelpful, or even harmful. All tests suffer from this limitation. It is a consequence of trying to squeeze as much precision as possible out of a single result, and the necessary price we pay for needing and trying to ...

The headline in my email this morning from Ed Week asked whether it was appropriate to do teacher evaluations in light of the Coronavirus. I wish they would ask the more honest question: is it appropriate to beat up on teachers during the Coronavirus, or should we give it a rest for a year? If the evaluation systems were based on a true accountability, this question wouldn’t exist. The fact that it does, that accountability and teacher evaluation in schools are in fact being put on hold—means that we don’t have anything even close to an effective accountability or evaluation environment. I continue to argue that if it can be put it on hold you have to stop calling it accountability because it isn’t. I would argue the same for evaluations. Education's myopic autopsy-based approach to everything inserts a punishment and punishment avoidance mentality into the process, not a how can we be great mindset that is at the heart of great organizations. As I study accountability in those or...

In this age of the Coronavirus and its overwhelming impact on literally everything, a bright spot in an otherwise ominous cloud is the way we are thinking differently about old problems, rethinking our relationships with each other, and reflecting on what is actually important. We should do the same with educational accountability. And we have a window in which to do it. Of all the problems to rethink, educational accountability should be at the top. For the past two decades (longer in some places) educational accountability has followed the "better autopsy" method for improvement, which will always fail. At the end of a school year the state performs an autopsy (and a partial one at that) and then forces schools to ask, "what could we have done last year to have had a better autopsy last year?" and then whatever the response do that this year. The better autopsy accountability is nonsensical for lots of reasons, but none more so than it will force schools n...

For a word used with ease in educational policy circles, accountability is a term that is surprisingly misunderstood and misused. Seeing this is relatively simple. Ask an audience to brainstorm a list of terms they associate with accountability and a pattern will quickly emerge. Many of the words will be positive such as: Transparency Effectiveness Responsibility Outcomes And many of the words and phrases will be negative, such as: Feet to the fire Testing due to lack of trust Blame Shame If you list these words in two columns on a sheet of paper what you will be observing are the two sides to accountability. The negative terms represent what happens when an organization refuses to be accountable and/or is perceived as failing. In that case, accountability is something imposed on that organization by outside stakeholders for the purpose of bringing the organization in line. Such an accountability focuses the organization on failure prevention at the expense of everything el...

Standardized test is the name most people assign to the tests used in state accountability systems, commercially available norm-referenced tests, and college admittance tests such as the ACT and SAT. I have long encouraged folks to drop the term “standardized,” since that merely refers to the conditions under which tests can be administered, rather than what this narrow family of tests are and do. Instead, I prefer to call them predictive tests. This describes what they are intended to do. I have also strongly encouraged a more critical use of vocabulary regarding predictive testing. This is because of the massive confusion that results from the plethora of terms now applied to testing that don’t mean what most people think, such as standards-based, or criterion-referenced. What sets a predictive test apart from all other forms of testing is its ability to produce predictive scores. Simply (and crudely) put, if I am slightly above average this year you can predict that I will probably ...