How to design for scale: lessons for ambitious new interventions

This piece is part of Apolitical’s spotlight series on scaling social impact, in partnership with the Bernard Van Leer Foundation. It also appears in our government innovation newsfeed.

In many respects, the so-called Graduation Program has been a roaring success: it’s the only intervention proven to lift people out of extreme poverty.

With food parcels, life skills classes and two years of asset transfers, it’s a complete package. Almost 100 countries have experimented with it, providing a rock-solid evidence base. So why isn’t it national policy across the world?

The biggest determinant of whether an intervention will succeed at a huge scale, changing the lives of millions rather than dozens or hundreds or even thousands, is not whether it works, but whether it was designed in a way that made it suitable for scale in the first place.

The path from developing a program that works to helping huge numbers of people is a long one, often taking 10 to 15 years. Organisations that don’t anticipate the difficulties will develop programs with features that make them inherently difficult or even impossible to scale up.

The Graduation Program, for example, is expensive. And now governments that want to implement it are trying to cut costs — but they don’t know what to cut and what to keep without losing the effects.

For reasons like this, success on a small scale is no indication, let alone a guarantee, of success on the big stage. Only interventions that are relatively simple, clearly better than the alternatives and not reliant on unique conditions are likely to scale well.

But it can be done. The secret is to plan for that long journey right at the beginning and to design something that will not just survive, but thrive as it gets bigger. So how do you do that?

The secret sauce

The first thing to do is to understand the problem being addressed, and how the intervention solves it.

"If you don’t know what that secret sauce is, you really don’t have a chance of scaling it"

“Sometimes an innovation catches on, but we don’t actually know why or what is making it work,” said Whitney Pyles Adams, who runs the NGO CARE’s Scale X Design accelerator. “And if you don’t know what that secret sauce is, you really don’t have a chance of scaling it elsewhere.”

“We have a program in Egypt working with men who are street harassers and turning them into allies who go out and become champions for women,” added Adams. “And one question we had for the team was: why do these men come in the first place? What brings them in the door? And they didn’t know. So they need to figure that out.”

Understanding why something works makes it clear what is necessary for the intervention. And that in turn it provides a big hint about whether the intervention is replicable and scalable. Perhaps those men in Egypt only came because they were unemployed and bored — in that case the intervention might not work somewhere with higher employment.

Even the creators of an intervention don’t always understand how it works right away — it’s something that emerges in the early stages of altering it for scale.

Make it scalable

The simpler and cheaper an intervention, the more scalable it is. And if you understand why something works, then you know what’s required — and what can be disposed of.

“It’s a game of subtraction, not a game of addition"

“When you design for scale, you need to pull everything out which is not absolutely essential,” said Larry Cooley, a global scaling expert. “It’s a game of subtraction, not a game of addition. Think of it like a game of Jenga: if I pull this out, will it still stand up? And now — will it still stand up?”

In 2014, the NGO Room to Read did just that. It works in low-income countries to support literacy and gender equality in education. Previously, its program had a lot of frills for its classes: flash cards, literacy wheels, dice with words on them and writing books for every pupil. Teachers loved the resources because they broke up the monotony of the day.

But they were expensive. So they cut all of them, replacing them with a single reusable student book — and it was still effective. Now the intervention is closer to a price point that can be absorbed by ministry budgets in countries like Tanzania and Vietnam.

However, a lot of what makes many programs work is intangible: how to teach, how to learn, how to interact with clients. When talking about the material elements of a program, these can be overlooked. But they are often just as important.

Codifying this tacit knowledge is challenging but essential. Otherwise, programs remain reliant on the original team. That can be a fundamental bottleneck for scaling — particularly if scaling in new sites abroad.

The early childhood intervention Reach Up program recently confronted this problem. Reach Up has been led by a small team of academics in Jamaica for almost 50 years. But now a dozen countries want to adopt it, from China to Colombia. So the team created digital manuals that detailed the curriculum, how to train people to deliver it, and how to adapt the materials to another culture. The program is now national policy in one country, soon to be three. Read the case study here.

Simplifying, cost-cutting and codifying: altogether, this is the shapeshifting stage for the intervention. Some changes will improve things, and others won’t — but it’s better to fail on a small scale, when it costs less time and money.

Imagine it at scale — and work backwards

With a working model in mind, it’s time to imagine what it would look like at scale. Three related questions need to be asked — and the answers will often send the model back to the drawing board.

What is the size of the need and demand?

This tells you what scale to aim for. The distinction between need and demand is important: just because you think something is important doesn’t mean the beneficiaries will agree. Take the case of open defecation in India: NGOs have often assumed that any toilet is better than no toilet — which from a public health perspective may be true, but from the clients perspective often isn’t. Unpleasant toilets go unused.

Who would pay for it — and is it affordable?

There are two vehicles for sustainable scale: government or markets. Those are the budgets and business models to work to when designing an intervention. It helps to be able to demonstrate a cost-benefit ratio, and to be transparent about the underlying costs.

It’s dangerous to assume there will be economies of scale. Often the opposite is true: sometimes big upfront investments are required to boost capacity, or labour supply constraints can kick in when hiring, for example, legions of trained nurses.

Who would do it?

There are huge differences in the agility and operational capacity of NGOs and government ministries, and each is suited to different kinds of interventions.

"Often you need to design an intervention with government adoption in mind"

“Often you need to design an intervention with government adoption in mind,” said Karen Levy, Director of Global Innovation at the NGO Evidence Action. “Theoretically, you could design a deworming program that was more cost-effective by having it roll out in waves across the country. Then you could use the same people over and over again as it moves from one place to the next. That’s fine — but you would need to have ongoing procurement, ongoing disbursements of operational funds. There’s a reason why lots of government campaigns are annual — the systems are much better suited to that.”

Hence Evidence Action’s program relies on the annual mass administration of deworming pills — and in 2016 it treated almost 200 million children. Even though doing it annually is not the most cost-effective option, it’s the only one that could achieve that kind of scale.

Test time

With a honed intervention and a vision of scale in mind, it’s time to put it to the test. That means randomised evaluations to demonstrate impact.

In recent years the public and non-profit sectors have embraced evidence-based decision making. Broadly speaking, this is a good thing — but pilots must be designed with scale in mind. Often they aren’t, and the world is littered with pilots that amounted to nothing.

The most important thing is for the pilot to resemble how the program will be run at scale.

People often get this wrong: they want their pilots to succeed, and they end up running them in the most responsive communities with the most supportive leadership. They also use the best frontline workers they have, and there is more attention paid to what is happening, so standards are kept high.

These conditions do not reflect what will happen at scale, and therefore a positive result may be misleading. A pilot should test what would be the steady-state conditions at scale.

The US early childhood development program Head Start ran into this problem with their pilot. When they tested it, the results were great: it reduced high school dropout and incarceration rates, and boosted employment. At scale, the effects were less impressive. That’s because the pilots were staffed by highly trained specialists; at scale, there simply weren’t enough of these people to hire. So they ended up recruiting from a pool of punters looking for a job — not professionals committed to helping poor children.

So: run the pilot as it would be run at scale. But also make sure it actually tests how the program or intervention will be used at scale.

The New Coke debacle is a good example of this. When Coca Cola were testing the new recipe, they did what was then the largest market research study in history. In fact, they did it several times. The results were good, so they took the plunge.

But the test didn’t match up to life: people hadn’t realised that the new product cost them the option of drinking the old one. And there’s a difference between drinking a sample from a tray in a shopping mall and buying cans to drink at home. Coca Cola threw away a huge amount of resources by not testing how the product would actually be used.

The long road to scale

The timeframe to reach scale up is generally 10 to 15 years — but the most important moment is the very beginning.

That’s when mistakes are minor and sunk costs are small. It’s the moment to shape the intervention through rapid testing and feedback, to create the simplest and cheapest model that retains effectiveness.

The trick is to plan for scale while never committing to it. To imagine what an intervention would look like at scale — how big it needs to be, who will pay for it and who will do it — while being ready to row back or scrap it if it tests poorly.

Then you still have the whole road ahead of you — but at least you can be confident it’s the right one.

(Picture credit: Flickr/Henning Supertramp)

Tom Graham