This post is written by Marta Olazabal, Basque Centre for Climate Change, BC3, and William Lewis, Lund University.

  • The context: It is time to act on adaptation.

  • The problem: We are still figuring out ways to assess whether adaptation is working or not.

  • Why it matters: Evaluation is essential to enable learning and successful adaptation.

Many regions worldwide are already highly impacted by climate change and others are identifying important risks and vulnerabilities to droughts, fires, rainfalls, storm surges, extreme temperatures and sea-level rise. To prepare for this, thousands of public governments worldwide are working on strategies and plans to adapt to climate change. For these actions, adequate monitoring, evaluation, reporting and learning (MERL) systems are crucial to know how effective and efficient they are, to correctly allocate public resources, improve actions or to compare and learn from others implementing similar measures.

In 2021, the COP26 Glasgow Climate Pact recently launched the two-year Glasgow–Sharm el-Sheikh work programme on the global goal of adaptation. This work programme aims to enable enhanced planning, implementation, and monitoring and evaluation systems for adaptation by better understanding methods to measure progress and metrics. It is clear that we have a long way to go to achieve agreements on how to globally assess adaptation, but even at the local level, the question still remains: How to measure climate adaptation success?

Measuring progress on adaptation is not an easy task. First, adaptation is a very ambiguous concept that has strong connections with sustainability, resilience and disaster risk management. It is difficult to decide whether something is strictly adaptation, or, at the very least, it is highly subjective. On the contrary, mitigation is quite straightforward. The accountability of greenhouse emissions is the universal metric for mitigation, easy to aggregate, universally applicable. However, when it comes to adaptation, there is not a universal metric or set of metrics. Adaptation is highly qualitative and relative to the context. As a result, there is a general lack of guidance on how to measure it. Second, finance for adaptation and adequate technical and local knowledge to identify risks and vulnerabilities is still generally missing. There is a lack of experience when it comes to adaptation policymaking and, thus, to evaluating adaptation. The field is still embryonic and only now we are witnessing second generation plans in a few countries, in a few cities, in a few regions. There is indeed momentum in adaptation policy and the opportunity to generate an effective culture of adaptation evaluation is giant.

An understanding of the wider (quantitative and qualitative) costs and benefits of adaptation actions is crucial for learning. Globally, understanding the progress of adaptation would help to better allocate international and national resources and fine-tune adaptation finance mechanisms as well as loss and damage implications. At the local level, reflective learning processes are needed within organisations to ensure that investment is actually reducing vulnerability and increasing climate resilience. It is not only the problem of making the policy more effective in general, i.e. actions reduce vulnerabilities and increase the capacities of systems, sectors, and societies to adapt. It is also the challenge of guaranteeing a sustainable and just adaptation. One that understands adaptation needs and beneficiaries, cares for the most marginalised and vulnerable, and provides a pathway to adapt that involves mitigation and sustainability goals.

What we found by interviewing cities about their MERL practices

We spoke to 6 worldwide cities (Athens, Auckland, Barcelona, Glasgow, Lima, Montreal), all early adopters of MERL systems and adaptation metrics, to learn about their experience and identify best practices. We found that, through participatory approaches, a strong focus on vulnerability and multi-loop learning, cities are right now developing organic ways to learn and understand the wider impacts of their climate adaptation actions. In general, the focus tends to be on output-style indicators: i.e. metrics that focus on processes rather than on outcomes and impact, which is the most challenging part.

Recent iterations of climate change adaptation plans have signified a move to simplify indicators and streamline the collection of data. For example, to supplement their output indicators, the Municipality of Lima has set up a Metropolitan Technical Group on Climate Change and Water Resources. The group comprises different representatives from across the municipality and civil society and allows Lima to ensure that wider benefits are being achieved. Similarly, the city of Barcelona has a highly participatory process with members of the public engaged at key stages of the Climate Plan’s lifecycle. Barcelona has also developed a number of specific research projects to investigate the outcomes for selected adaptation projects, then used to pilot the collection of outcome data.

Other cities have chosen not to develop a set of indicators for their climate plans. Glasgow has instead invested in understanding the evolution of vulnerability of the city’s residents. Similar studies have also been undertaken by Lima and Montreal, and underline one approach to assessing the outcomes of adaptation actions outside of traditional indicators: repeated vulnerability assessments.

In Montreal, we have witnessed a move from complex and large sets of metrics to simplified sets that focus on vulnerability. Indeed, monitoring and evaluating a plan with over 940 commitments is a complex process especially in an agglomerated city like Montreal with multiple different administrative bodies. To overcome complexity in data collection, many cities use a platform to centralise data collection.

However, prior to the implementation of a platform, a culture of learning and buy-in to MERL across the city is vital. The city of Athens has worked closely with different departments to break down silos and ensure that data on climate adaptation is collected at every possible opportunity. On the other hand, Auckland, while in a pre-implementation stage, is taking the opportunity to survey the whole of Auckland Council to understand all forms of monitoring and evaluation currently in place. The aim of this is to allow synergies and opportunities for external MERL partnerships to be identified.

Looking ahead

Through participation, a focus on vulnerability and multi-loop learning, worldwide cities seem to be right now paving the way towards effective adaptation of MERL. These were just a few examples that show the evolution of MERL approaches in different large cities facing similar challenges. One common aspect emerges from these experiences: long-term collaborative approaches to MERL are key to the transition to a learning organisation in the field of adaptation to climate change. More is to come while cities are experimenting. Following up their different adaptation implementation and evaluation journeys will be key to developing reference frameworks for context-specific sustainable and transformative long-term climate adaptation strategies.

Read the full Policy Brief: Lewis, W., & Olazabal, M. (2021). Learning how to learn: The experiences of 6 global cities in tracking urban adaptation. BC3 Policy Briefs (PB 2021-05). BC3 Basque Centre for Climate Change. Leioa, Spain.

👋 You can create a post like this one! Share your thoughts with a community of public servants. Learn more

(Image Credit: Unsplash)