• March 20, 2018
  • 5 minutes
  • 0

Innovation labs measure success to justify themselves—but it can’t be done

The head of MindLab on why labs should give up on their Holy Grail: impact measurement

MindLab is the world’s oldest government innovation lab and has spawned imitations all over the globe. It stands at the forefront, both in theory and practice, of efforts to make government cleverer, faster and more inventive. Here, its director Thomas Prehn shares his experiences and ideas in a regular update.

Apolitical has previously interviewed Thomas here. He can be contacted on the Apolitical network here. The views expressed here are Thomas’s own and not representative of any public body. This piece appears in our feed on government innovation.

How do you measure success – or failure – as a public sector innovation lab? (20.March.2018)

Well, we’ve seen a rise of behavioural insight methodology. And we all work with data and statistics. Both approaches nurture the urge for evidence-based policymaking: in an A/B test, one will always prove better than the other; in diagrams, data curves always change over time. But are these tools really adequate for making policy amid messy politics where the qualitative indicators outnumber the quantitative?

At MindLab, after working with the Danish Agency for the Labour Market, can we rightfully claim that the thousands of previously unemployed people now in jobs owe them solely to our intervention and not to everything else happening around the world: Brexit, Trump’s presidency, the economic recovery of Poland.

Yet we constantly have to justify our existence and effort by making it possible to see what value we’ve created. In order to survive. Proof of value is truly the Holy Grail for MindLab and labs like it: impossible to find, but drinking from it will grant you eternal life.

“We constantly have to justify our existence and effort by making it possible to see what value we’ve created”

For three years we have been searching. We developed and tried various approaches to measure impact: specifying early signs of success, returning to old projects to gather intel about outcome effects, identifying and claiming responsibility for the conditions that enabled projects with a feel-good vibe to succeed.

But without success. When we came closest, we came off as arrogant and complacent to our colleagues, taking credit for what had been created in collaboration. And, still, we were not able to prove a causal link to real measurable effects.

Now, we’re trying something new. We’ve identified eight indicators tied to MindLab’s strategy — not the policymaking we work with — but how we bring value to the core tasks of our colleagues. MindLab does not own the intention behind these projects, nor are we responsible for the corresponding solutions. Our colleagues are. But we’re a part of their ecosystem, in a symbiosis where they benefit from our efforts. That’s the only reason we exist. We’re making room for indicators like: strategy execution, inducing smart work, and fostering mindset and leadership shift.

It’s an experiment in the continued quest for the Holy Grail. While it might be a legend, the search itself has proven valuable to labs like MindLab: if not for our continued survival, then for the opportunity to reflect and evolve.

(Picture credit: MindLab)

Thomas Prehn

Read other instalments of this series:
Thomas Prehn’s innovation diary
Design sinking
How to scale up public policy
A backlog of curiosity
Innovation is not a room
Bring me problems, not solutions
How to build a team of innovators


Leave a Reply

to leave a comment.
Master the skills you need for the public service.

Discover inspiring resources, tools and policies.