The Importance of Policy Evaluation
The development of a policy often involves significant resources and effort, yet many policies are treated as final products rather than living documents. In health care, it is commonly stated that "discharge planning starts on admission." Similarly, policy evaluation should be embedded from the inception of policy creation to ensure continuous improvement and relevance.
Policy evaluation plays a vital role in measuring compliance and effectiveness, helping organizations determine whether policies are achieving their intended objectives. Without a structured evaluation process, it becomes difficult to identify strengths and weaknesses, undermining both public and private sector governance. Effective evaluation strategies contribute to sound governance and better policy design and implementation, as emphasized by the OECD.
Defining Policy Compliance and Effectiveness
"Policy compliance" refers to the conformity with regulatory requirements, including laws, standards, and regulations, as defined by the Government of Canada. "Policy effectiveness," on the other hand, describes the extent to which a policy achieves its intended benefits (Nagel, 1986). Though related, these concepts serve distinct roles and often require different methods for assessment.
Strategies for Ensuring Policy Compliance
Research highlights several strategies for regulatory compliance (Gunningham, 2015):
- Advice and Persuasion: Focusing on education and negotiation.
- Deterrence: Emphasizing penalties for violations.
- Responsive Regulation: Adjusting measures based on compliance behavior.
- Risk-Based Regulation: Prioritizing areas of higher risk.
- Smart Regulation: Using multiple tools for broader impact.
- Meta-Regulation: Requiring internal control systems.
- Criteria Strategies: Providing decision-making criteria for regulators.
Factors Influencing Compliance
The OECD (2000) identified three primary factors influencing compliance:
Awareness and understanding of the rules.
Willingness to comply, influenced by incentives and enforcement.
Capacity to comply with policy requirements.
Core Characteristics of Policy Evaluation Frameworks
Policy evaluation frameworks generally share three essential characteristics:
- Systematic: Evaluations should follow a planned and organized procedure (Gasper, 2018).
- Rigorous: Methods should be well-designed and tailored to the questions being addressed (Commission on Evidence-Based Policymaking, 2017).
- Objective: Evaluations must be impartial, free from personal or political influence (Parkhurst, 2017).
Tools for Policy Evaluation
Countries often use the following tools to assess policy performance and effectiveness:
- Spending Reviews: Analyzing policy priorities in relation to financial sustainability.
- Performance Management: Engaging employees to enhance organizational effectiveness.
- Audits: Assessing the reliability of information on policy effectiveness and compliance.
However, policy evaluation frameworks should be adaptable to the specific context of each organization or system.
Toward Inclusive Policy Evaluation
Policy frameworks must consider historical harms and systemic barriers faced by Black, Indigenous, People of Colour (BIPOC), LGBTQ+, and other equity-deserving groups. Dedicated, skilled human resources for policy-making are essential for creating inclusive and equitable policies. Organizations must commit to mechanisms that promote diversity, equity, and better outcomes for marginalized communities.
By embracing these principles, organizations can create policies that are both effective and just, fostering a more equitable society.
Towards Meaningful Policy Evaluation
Policy evaluation is crucial for ensuring that policies achieve their intended goals effectively and equitably. By incorporating systematic, rigorous, and objective evaluation practices, organizations can identify areas of success and improvement. Key tools like spending reviews, performance management, and audits, along with adaptive compliance strategies, play a significant role in policy evaluation. Moreover, considering historical harms and systemic barriers can help create inclusive policies that promote diversity and equity. A comprehensive approach to policy evaluation contributes to sound governance and better societal outcomes overall.
Content adapted from “Enhancing Policy Evaluation at PHSA” report by Adam Bowbrick, Amanda Engel, Zahraa Hawili, Kevin Hemmat, & Sasheeka Udawatte, School of Public Policy, Simon Fraser University, prepared for the Provincial Health Services Authority of British Columbia.
I acknowledge my place of work is within the homelands of xʷməθkwəy̓əm (Musqueam), Skwxwú7mesh Úxwumixw (Squamish Nation), and səlil̓ilw̓ətaʔɬ /Selilwitulh (Tsleil-Waututh) Nations. The Provincial Health Services Authority is responsible for services throughout what is colonially known as British Columbia which includes the homelands of over 200 First Nations, 7 major language families and 30 different dialects.
References:
Alberta Health Services. 2019. “Evaluation Planning: A Resource for Addition and Mental Health Researchers.” Last modified February, 2019. https://www.albertahealthservices.ca/assets/info/amh/if-amh-ke-evaluation-planning.pdf
Bovens, Mark, Paul Hart and Sanneke Kuipers. 2009. “The Politics of Policy Evaluation.” The Oxford Handbook of Public Policy. DOI: 10.1093/oxfordhb/9780199548453.003.0015
Government of Canada. n.d. “Generic Valuation Tool: Regulatory Compliance and Enforcement.” Accessed March 20, 2022. https://www.bac-lac.gc.ca/eng/services/government-information-resources/guidelines/generic-valuation-tools/Documents/GVT_RegulatoryComplianceAndEnforcement.pdf
Gunningham, Neil. 2015. “Compliance, Enforcement and Regulatory Excellence.” Penn Program on Regulation. https://www.law.upenn.edu/live/files/4717-gunningham-ppr-bicregulatordiscussionpaper-06
McConnel, Allan. 2010. “Policy Success, Policy Failure and Grey Areas In-Between.” Journal of Public Policy 30 (3). https://doi.org/10.1017/S0143814X10000152
Nagel, Stuart. 1986. “Efficiency, Effectiveness and Equity in Public Policy Evaluation.” Policy Studies Review 6(1). https://doi.org/10.1111/j.1541-1338.1986.tb00651.x
OECD. N.d. “Chapter 1. Towards a Systemic Approach to Policy Evaluation.” Accessed March 20, 2022. https://www.oecd-ilibrary.org/sites/9fa07ac8-en/index.html?itemId=/content/component/9fa07ac8-en
OECD. 2000. “Reducing the Risk of Policy Failure: Challenges for Regulatory Compliance.” Accessed March, 22, 2022. https://www.oecd.org/regreform/regulatory-policy/1910833.pdf
OECD. 2009. “Evaluation policy and guidelines for evaluation.” Last modified October 2009. https://www.oecd.org/dac/evaluation/iob-evaluation-policy-and-guidelines-for-evaluations.pdf
OECD .2011. “Typology and implementation of spending reviews”, OECD SBO Meeting on Performance and Results. Accessed April 5, 2022 http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=GOV/PGC/SBO(2011)9&doclanguage
Wallner, Jennifer. 2008. “Legitimacy and Public Policy: Seeing Beyond Effectiveness, Efficiency and Performance.” The Policy Studies Journal 36 (3). https://doi.org/10.1111/j.1541-0072.2008.00275.x
Log in or sign up to continue the conversation