When It Comes to Educational Data, It’s Purposeful or It’s So What

Back in the late 1990s, when I was a newly minted teacher, one of the buzziest concepts in education was data-driven decision making. As Susan Carroll and David Carroll write in 2002’s Statistics Made Simple for School Leaders: Data-Driven Decision Making, data “has the potential to influence public education beyond any public school reform initiative that has yet been designed.”

What was true then is doubly so now. In 2016, we don’t even need to make statistics simple. It’s all mercifully done for us using algorithms more complex and data sets larger than we could have previously imagined. And not just schools. Everyone and everything. We’re all residents of a quantified world built by wearables, internet searches, social media interactions, geo-tagging, mobile devices, and all the other ephemera of our modern lives.

The So What Principle

Ostensibly we get something out of all this. I do love my data-powered Spotify Discover Weekly, and it can be interesting (or depressing) to see how my day’s physical activity compares to last week’s or last month’s. But despite its quantity and precision, most of the data in both our personal and professional lives is little more than bar trivia — interesting but really rather pointless. Or, as Dan & Chip Heath relate in Switch, it’s “true but useless” (an aphorism attributed to Jerry Sternin).

People sometimes describe this as being data rich but information poor. I prefer a different name - the So What Problem. Thanks to data, I can know so much about my life and work — how many times a piece of content gets shared, what type of person read this or that article, how many times I streamed Steely Dan’s “Only a Fool Would Say That” — but unless the data relates to, sheds light on, or otherwise informs a meaningful, practical problem, then so what.

Finding the Purpose

The cure for the so what problem is to ensure that we always begin with a purpose. The purpose drives all else. It progresses the dialogue from data-driven decision making to purpose-driven decision making.

Here's a strategy that works for me. Whenever looking at a piece of data, interrogate it with three questions:

  • Does it matter? Just because something’s true doesn’t make it important to the task at hand. There’s lots of data in the world, some of it fascinating. Sometimes, it may even seem important, but is it really? How? To whom? What meaningful problem does it clarify? If you don’t know, look again.
  • What can I do about it? Choosing to do hard things is good; choosing to do impossible things is not. If we focus on this goal, how will we work on it? Do we have the time, resources, money, support? If not, don’t stop. Break the goal down and make those prerequisites the goals.
  • How will I know if I succeeded? This is data’s sweet spot. If we pick something that matters and have broken it into achievable goals, how will we measure our progress? Think holistically, not just one or two data points. What data, when combined, reveals the answer?

Don’t shortcut this process. Take the time needed to work the problem. When you come up with something, interrogate it with these three questions again. Dig and dig until you get to something that holds up. It’s a long path to meaningful outcomes. Better to ask “so what” at the beginning of the journey rather than the end.