Regulatory Evaluation Platform

This information was sourced from "Regulatory Evaluation Platform" by Observatory of Public Sector Innovation (OECD), https://oecd-opsi.org/innovations/regulatory-evaluation-platform/. Licensed under Creative Commons Attribution-ShareAlike 3.0 IGO (CC BY-SA 3.0 IGO)

Innovation Overview

What is problem the innovation solves? The complete lack of a timely metric about regulatory red tape and the cost (on a GDP basis) of complying with regulation. What is the innovation? Using NLP (natural language processing) - a branch of AI - to automatically calculate regulatory burden metrics from the text of regulations, and correlate the evolution of regulatory burdens over time with GDP statistics in order to measure how increased regulatory burdens impact the industry. Individual regulatory provisions are also matched to industry sectors (using NAICS - North American Industry Classification System) so that users can monitor the evolution of regulatory burdens for specific industry sectors. Finally, it also tracks and counts "incorporations by reference" namely regulations that refer to third-party standards (eg ISO norms), thereby including them by reference in the regulatory corpus.

The objectives or goals of the innovation are to significantly speed up a previously manual process that took so much time that it could not provide with accurate and timely measure of the evolution of regulatory burden.

The beneficiaries of the innovation are:

• The government, in that it (1) speeds up an impossibly tedious tasks (2) allows for the process to be done weekly as opposed to yearly, thereby providing a much more contemporaneous and vivid visibility over the evolution of regulatory burden across ministries.
• Industry: In that the government now has a timely tool to monitor how regulation can negatively impact industry performance.

How is the innovation envisioned for the future? For example, how will it be institutionalised in its current context? How will it scale even bigger? We aim for broader adoption of the metrics generated by the platform in the measurement to monitor regulatory burdens. We aim to ingest regulations from other jurisdictions for benchmarking purposes. We want the platform to support bills and draft regulations so that the evolution of regulatory burden can be anticipated before formal adoption and that the metrics generated by the platform be part of the broader regulatory impact analysis process.

What Makes Your Project Innovative?

Regulation is the code that runs society. Similarities between computer code and legal code have been pointed out by academics before (eg. Larry Lessig's "code is law"). Yet, one type of code is developed using agile methodologies (interative roll out, prompt feedback, quick adjustments, maximise value creation) and the other one relies on an arcane process where almost nothing is measure and feedback takes a long time to come in and only through untimely and complicated processes (court decision interpreting the text of regulations, industry consultations, lobby, elections) that amplify the voice of the biggest firms and silences small entreprises and individuals for lack of resources to contribute to consultations. The REP is part of much broader journey towards the agile regulator and provides a first timely signal about how regulation (based on metrics inferred from their text) is likely to impact the economic performance (based, for now, on GDP metrics).

What is the current status of your innovation?

Working product available on a live website.

Collaborations & Partnerships

• KPMG => technical expertise (AI, NLP, full stack dev), subject matter expertise in legal information and legal technology
• Transport Canada => subject matter expertise in the development of metrics, validation, development of use cases and personas
• Statistic Canada => Supply of the economic model that correlates regulatory burden with GDP

Users, Stakeholders & Beneficiaries

• Citizens / Citizens: Their governments have a tool to measure regulatory burden and ensure that, for instance, the regulatory stock doesn't inflate to a point where Canada's economic competitiveness is impacted.
• Government officials: Speeds up significantly the job of inferring regulatory burden from text.

Results, Outcomes & Impacts

What results and impacts have been observed from the innovation so far? Increased discussion over regulatory burden measurement, appetite for tracking of incorporation by reference and measurement of other types of metrics and signals.

How have the results and impacts been measured (e.g., methodologies used)? Validation with SMEs in the government. Statistics Canada premilinary (to be published) research that validated some hypotheses (eg that increased regulatory burdens correlates with lower GDP, among other stats).

What results and impacts do you expect in the future? Accross the board adoption adoption of the metrics generated by the platform for regulatory measurement.

To the extent possible, please indicate the tangible or numeric results. Provides 50 times more data about regulatory burdens (ie weekly refreshes v. yearly refreshes). Automates a process that took two weeks before for more than 1 FTE, so at least a millionth increase in efficiency (from 2 weeks to a second)

Challenges and Failures

What challenges have been encountered?

• Industry matches depend on the text of the provision, which can't always be mapped to a specific industry. There is no training set that can be used to train an AI algorithm with supervised learning techniques.
• Initially difficult to validate given that this is new data that can't be benchmarked (resolved with significant SME involvement and GPD correlations to validate hypotheses).

Conditions for Success

Strong support from government sponsors who believe in the necessity to innovate in the space and are not only willing to fund development, but also to allocate a lot of their time to validation, interation and design.

Replication

We are currently in the process of ingesting regulations from the Quebec government. This is a first steps towards creating algorithms and matchers that are more agnostic and can apply to other languages than English and to a different legal system (Quebec is a civil law province and while its statutory law acts is theoritically no different from statutory law in common law provinces, legislative drafting techniques are different and influenced by civil law drafting techniques).

Lessons Learned

• Need for a strong commitment and vision from government sponsors;
• There is a lot of room for improvement in regulatory drafting, measurement of regulatory efficiency and burden (we discovered that there's a lot more to do);
• The potential for improving society through a more agile regulatory process is immense.