Enabling Change

Hunting Shipwrecks: Why Valuing Error Improves Outcomes

With the world in the state that it is, I’m looking for all the guidance I can get about how we get through this, get stronger, and evolve. So this summer I’ve been re-reading Nassim Nicholas Taleb’s Antifragile: Things That Gain from Disorder.

One of Taleb’s many insights relates to optionality; specifically, it is better to have the sense to recognize a good outcome when it’s staring you in the face than to have a theory about what produces said outcome. Also the inverse: it is necessary to recognize a bad outcome and to stop doing the thing that is producing it.

These are old and uncomplicated ideas. And yet in education we struggle mightily to hold onto them. We confuse things we can measure (test scores) for things that are worth measuring (real life results). We presume causal connections where none exist. (If we could just get all of our 8^th graders to pass Algebra I, everyone would graduate from college!) We invest so heavily in our theories that we ignore evidence that we could be wrong, or worse yet, confuse adherence to theory with achievement of the outcome itself. (On that last point, I am telling on myself.)

A corollary to this observation is that trial and error only works when we treat error as information. We tend to focus (at least rhetorically) on figuring out “what works” and how to scale it. But in a challenging, human-centered field like education, the reality is that much of what we try doesn’t work. Even when we acknowledge that reality, how well do we really learn from failure?

Taleb relates an anecdote about a friend who hunts valuable shipwrecks on the ocean floor. His process is to 1.) identify (using research) the places where he is mostly likely to find success, 2.) search there until he can conclusively determine that the wreck is not there, and then 3.) move onto the area with the next highest probability of success and not return to the previous one.

As educators, we have deeply held beliefs and convictions (hopefully informed by evidence) about how we should do school. We build learning experiences, develop curriculum, or design schools (or school systems) around those convictions. That’s our step one in the shipwreck-hunting process. But how often, or how well, do we engage in steps two and three?

One of the challenges we face is that the outcomes we care most about (real life) are far removed (distal, in evaluation parlance) from the day-to-day work of school. At the Workshop School in Philadelphia, we think about long-term outcomes in terms of security (personal, social, financial), connection (meaningful relationships), and purpose (investing time and energy in things that are personally meaningful). None of those outcomes materializes overnight, so how can we know whether we’re helping, hurting, or just treading water?

That question is a lot easier to answer if the outcome is something like mastering fractions, rather than the outcome of students navigating their way through a minefield of racism and trauma into a life that feels like it’s their own. In both cases, however, making trial and error work relies on defining short-term (proximal, in geek speak) outcomes that you can observe quickly. (There is an interesting parallel here to something that I’ve learned about wayfinding: that focusing on the next step and then taking stock of where you are is much more important that having a ten-year plan.)

This too is not a new idea. In education, it is at the heart of formative assessment. And yet, in my past life as a researcher and my current one as a school leader, I am continually reminded of how bad we are (and sometimes, how bad I am) at learning from error. We can see and acknowledge that something is not working readily enough, but we fail to fully exploit that knowledge by figuring out what we should stop doing. As with shipwreck-hunting, ruling out wrong answers increases the probability of finding right ones.

As we embark on a school year of nearly unprecedented uncertainty, this is one of my new year’s resolutions: to value the “error” part of trial and error as much as I value the “trial” part. To work as hard to subtract as I do to add. And above all, to make sure I follow all of the steps of the shipwreck hunt, making sure that each new tool or strategy builds on our past mistakes.

Photo at top courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action.