This article is written by Emilie Gravel, Ph.D., Senior Research Adviser, Employment and Social Development Canada, and Hasti Rahbar, Head of Multidisciplinary Projects, Innovation LAB at Employment and Social Development Canada.

When Emilie and I first met, one of the things that drew us close together, besides our sarcastic humour and unconditional love for cats, was our commitment to evidence-based decision making in the public sector.

We often find ourselves in passionate discussions about the importance of using evidence-based approaches to deliver public policies, programs and services that have demonstrated their worth. To us, evidence-based public service is not a passing fad, it is the wave of the future. And with the Covid-19 outbreak, investing in decision-making that is supported by more than tradition, opinion, or partisanship has never been so crucial for governments everywhere to respond effectively to increasingly uncertain and fast-changing circumstances.

With emergency income supplementation melting budgets more rapidly than climate change melting polar ice caps, we simply cannot afford to make decisions without considering evidence surrounding it.

This is easier said than done. We work in challenging environments where demands, timeframes, mindsets, and skills, are often misaligned with evidence-based approaches. Our goal with this article is to empower you to use evidence-based approaches. We share our personal experiences on the subject, and hope they can offer a bright spot when the going gets tough. And importantly, we recommend some practical solutions to help you overcome some of the most common hurdles you may encounter when trying to integrate evidence in your work.

What is evidence anyway?

First, while what counts as evidence is debated, there is more consensus about what counts as scientific evidence. Scientific evidence is the result of observations acquired by using the scientific method. What makes this method so powerful is a reliance on observable data collected and tested using a systematic, rigorous, and replicable methodology.

That said, the problems we tackle in the public sector are increasingly complex and scientific evidence by itself cannot paint a full picture. Other types of evidence such as those provided by administrative data, program evaluations and insights from citizens and stakeholders are necessary for effective and ethical evidence-based policy making.

One-size fits all approach to evidence-based policy making not only runs the risk of being ineffective because of inherent differences between contexts, rendering the whole approach counterproductive, it can also contribute to further marginalising populations in vulnerable situations. When gathering evidence, make sure it includes the voice of those targeted by your policy. Now let’s turn to some practical advice.

1. Give your scientific literacy a boost

If you want to better integrate evidence in your work, developing a basic understanding of research methods should be your logical starting point. This will take time and effort, but the pay-off is immense.

Discuss with your supervisor the possibility of integrating this in your learning plan objectives. This could take the form of a self-directed, high-quality online courses to taking an in-person semester long course at a university. Choose a course covering both quantitative and qualitative methods. A good consumer of scientific evidence should be comfortable with both types.

Having sat at some decision-making tables, when someone asks a question, the answer is seldom “I don’t know”. But that may be a very appropriate answer in circumstances where there is not enough evidence to support a conclusive answer

Please do not skimp on your statistical literacy! For many of us, the very thought of mathematics is anxiety provoking, but give yourself a chance. There has never been more quality and accessible statistics learning material. At the very least, get a good grasp of the concepts of effect size and statistical significance, the latter being the most misunderstood and misused concept in quantitative research. Knowing what information statistical significance does and does not convey is a cornerstone of appropriate interpretation of scientific evidence.

2. Tackle lack of time by working smart

The latter point brings us to another major hurdle to integrating evidence in our work: lack of time.

Developing a decent understanding of a body of literature is often incompatible with the tight deadlines under which most of us work. To get a quick lay of the land on a particular subject, we first try to find review papers, ideally meta-analyses for quantitative research and meta-syntheses for qualitative research, which provide the highest quality of evidence in the category of research syntheses.

If we can’t locate any, then we search for systematic reviews and then narrative reviews. Asking a few core questions regarding the quality of the methodology is another valuable strategy that can help you narrow down a large body of literature to the best available evidence. Finally, consider hiring a graduate student who specialises in your policy area on an internship and assign them a comprehensive review of the issue you are tackling.

Senior students especially are budding experts, which means they bring a good grasp of their field and experience in literature review. On the flip side, by providing them an opportunity to work in the public sector, they gain experiences and develop skills that can be difficult to acquire in academic settings, which in turns can increase their hiring potential post-graduation, hopefully in the public sector. It’s a win-win situation.

3. Be aware of thinking styles and a lifetime of behaviours

Public service organisations are characterised by formal hierarchical processes with little room for emotion or independent thinking.

The system is set up this way for a reason. Without it, there would be chaos. But this very same protective mechanism that prevents things from spiralling out of control has become a barrier for introducing new methods and tools into the public service.

Asking “what if” is hard in large hierarchical organisations because it signals uncertainty and risk, but curious minds can’t help but ask. The result: both sides end up feeling frustrated.

To bridge the evidence-policy gap, we are utilising data visualisation and storytelling techniques to convey our message in a simple and user-friendly way

The strategist Jeanne Liedtka has summarised this tension and dynamic between mindsets into the George and Geoffrey metaphor. They are archetypes representing two very different behaviour patterns found in organisations: George is analytical and risk averse whereas Geoffrey is more creative and comfortable in uncertainty. Neither one is better than the other, but they have different needs when working together.

Challenges arise when they can’t communicate, but when they work together they can propel the organisation forward. In our experience, those working on new innovative and experimental approaches in the public sector tend to be closer to Geoffrey, pushing against a system that likes the certainty of the status quo. Not surprisingly, decision makers in bureaucratic environments tend to be Georges who understand and manage risk through careful analysis. Luckily, small experiments offer a great way to start a conversation about assumptions and evidence through progressive action and goals without disruptive change. They provide low-risk, small-scale and fast-paced opportunities to learn what works and what doesn’t.

4. Foster a culture of curiosity and humility

As Emilie puts it, “when I completed my undergrad degree, I thought I knew everything. When I completed my Ph.D., I realised I knew nothing, and that was for the better.”

Speaking of mindsets, curiosity and intellectual humility go hand in hand in our search for evidence. As good researchers and public servants, we need to be actively curious and know our blind spots. We often talk about being open-minded while simultaneously viewing people who are unsure about their beliefs as lacking confidence or being poor performers.

Having sat at some decision-making tables, when someone asks a question, the answer is seldom “I don’t know”. But that may be a very appropriate answer in circumstances where there is not enough evidence to support a conclusive answer. Maybe “I don’t know yet, but here is how we can find out” is a better answer. Intellectual humility can take us into an uncomfortable space where, as researchers and decision-makers, we need to admit to being wrong or not knowing. This mindset starts from the self through the recognition that we are more imperfect and imprecise than we’d like to think.

5. Speaking the same language: Hard data meets human communication

We need to become better at communicating. Increasingly, storytelling and data visualisation techniques are becoming popular and for good reason.

No matter how interesting your data is, there are far more people who lack the time and knowledge to interact with your findings than there are those who will eat your 20-page technical report for breakfast and ask for seconds. If your goal is to reach a wide audience, then you need to speak a common language.

Right now, our team is in the midst of a pandemic-related project where data collection and analysis are happening in real time and needs to be disseminated to decision-makers quickly. As you can imagine, the sheer amount of complex and interrelated evidence is too much for any senior decision-maker to wrap their head around, considering how many other high priority files they have to keep up with. But our evidence is important and we wholeheartedly believe in our work. To bridge the evidence-policy gap, we are utilising data visualisation and storytelling techniques to convey our message in a simple and user-friendly way. For example, by visually representing an interaction between two key players in a system, including where things are going well and where they are not, all based on our research.

This is our advice for integrating evidence in your work. We know it’s hard, and we know it takes time. But it’s also exciting and can open the door to so many new opportunities. If there are days where it feels like rolling a boulder uphill, know that you are not alone and that, with the right team, you truly can make a positive difference. — Emilie Eve Gravel and Hasti Rahbar

About us

Hasti Rhabar has a background in Psychology and Public Administration. She has been a public servant for 10 years having worked in policy, social research, and currently in a Human Centred Design Lab specialising in behavioural insights and design thinking methods. She loves working on multidisciplinary projects with horizontal teams and diverse perspectives.

Emilie Eve Gravel is an experimental psychologist who now works as a federal government research adviser. She is a social and behavioural sciences junkie who gets a kick out of finding new ways to inject scientific research into public service and to make it more accessible to public servants.

(Picture credit: Unsplash)

Make sure to share your own thoughts with the author by leaving a comment below