• Analysis
  • November 23, 2018
  • 7 minutes
  • 1

From research labs to data training, here’s how to bring evidence into policy

Canadian policymakers rely on Google rather than policy data, claims a new report

Canadian policymakers are relying on Google to find out which policies work due to a lack of evidence-building infrastructure in government, a new report claims.

Evidence That Works, released last week by public policy thinktank the Mowat Centre, reports that Canada lags far behind other developed countries in its use of evidence-informed policymaking.

This, author Anne White writes, is due to a dearth of trust and transparency between the policy and research communities. White, a senior advisor in the Privy Council Office’s Results and Delivery Unit, writes that many other governments face the same problem. She recommends six steps Canada and other countries should take to bolster evidence-based policymaking in their countries.

Where Canada is going wrong

In Canada, policymakers don’t have consistent access to reliable and relevant research.

The crux of the issue, writes White, is is the strained relationships and communication between policymakers, researchers and the social sector, all of whom work with different incentives, needs and time frames. This cuts into opportunities for data- and evidence-sharing and collaboration.

Policymakers interviewed by White said they start their search for evidence with Google, which means they’re directed to websites, projects and policies selected by the tech giant’s algorithm.

Policymakers also turn to trusted advisors, universities, other countries’ and international organisations’ databases and academic literature to find and filter evidence — sources which can’t provide policy- and context-specific evidence. “They simply do not know how to find evidence of what works,” White writes.

While the information they find can be useful for finding people to connect with, alone it’s “insufficient — and often dangerous — to use it as best practice”, the report claims.

The need for evidence

In an ideal world, government officials would have evidence at all stages of the policymaking process in one easily accessible database. They would use it to define the problem, assess the legal, political and research contexts and identify possible solutions.

But in reality, policy evaluations are often done as audits of a program: a check on their progress and effectiveness, typically measured through survey results. These audits are often haphazard and lack the thoroughness of randomised controlled trials.

Governments at both the federal and provincial level in Canada have taken steps to improve evidence use in policymaking. These include the creation of the Social Research and Demonstration Corporation, an independent non-profit research organisation, as well as a number of innovation units and teams dedicated to better service design and evidence use. There are also dozens of research partnerships between government departments and social research organisations in place.

But, White writes, these are only the first steps to integrating evidence use into policymaking consistently. If governments want to make smarter decisions about which policies they should and shouldn’t use, she recommends the following steps:

1) Governments should invest in their own social innovation evidence lab. It would work with researchers, social purpose organisations, funders and investors to test best practices, measure impact and serve as a forum for discussion and debate on competing approaches — similar to the What Works Centres in the UK.

2) They should also have an independent knowledge repository. The centralised reference point would function like the US’ Results First Clearinghouse Database, which is a one-stop shop for resources on social policy programs’ effectiveness. It should function as an easy-to-use, transparent online database where public servants can find out which policies are working, which aren’t and who is leading the work.

3) Public servants need better data literacy and analysis skills. It’s often difficult for policymakers to understand program-level data, national surveys and administrative information due to a lack of training, which prevents governments from maximising the data they do have. All civil servants working on policy should be able to use and understand the data that underpins their work. And, White says, they also need to get more comfortable giving policy advice that isn’t aligned with — or even directly contradicts — what ministers want.

4) Governments need to fund more research. Greater investment in technical support for research and development is key to more rigorous experimentation and evaluation.

5) We need established standards for evidence. To ensure evidence is of high quality, government needs benchmarks for all evaluations it funds. It must ensure evaluators are trustworthy, transparent about their research methods, politically independent and provide step-by-step monitoring and evaluation strategies.

6) Governments must build a foundation of trust with their partners. White’s interviewees said the biggest challenge they face is that different incentives drive government, researchers and the social sector. Misalignments in goals, focus, approach and even personality make these partnerships challenging. All interviewees said that the most effective way to improve these relationships is by installing trusted leadership. —Jennifer Guay

(Picture credit: Unsplash)

Discussion

Leave a Reply

to leave a comment.
Master the skills you need for the public service.

Discover inspiring resources, tools and policies.