Kaufman's Model of learning evaluation: Evaluating training effectiveness

How to use this guide

This guide is based on Kaufman's Learning Evaluation Model and is tailored to address the unique challenges and responsibilities of public service roles.

This document provides a blueprint for how your organisation could benefit from using the Kaufman Evaluation Model to assess how training programmes lead to better policy decisions.

Figure 1: Kaufman’s Learning Evaluation Model.

By using this guide, you can conduct a thorough evaluation that not only identifies areas for improvement but also demonstrates the value and impact of training initiatives. This ensures that the training is not just a one-off event but a catalyst for meaningful change in practices and policies.

Please refer to each step below for a comprehensive guide on how to proceed with the evaluation.

For more information or examples of questions you could ask participants, please click on the links where indicated.

Step 1: Input

Step 1 focuses on assessing the quality of the training itself and how it could be improved to ensure its effectiveness.

Pre-Training Survey

Before the training begins, distribute surveys to gauge participants' existing knowledge and expectations. This can help tailor the materials to the trainees’ needs. It also gives you ‘before’ data to compare to ‘after’ the training.

Feedback on trainees' knowledge pre-intervention can be followed up later on in your evaluation process — see Step 5, post-training survey.

For examples of the questions you could ask, download our pre-training survey template.

Carrying out a survey is a time-efficient method of gauging trainees’ current understanding of your training topics and their expectations for future training initiatives. Other, more in-depth methods could also be used in this stage, including:

• Focus groups: Conduct focus group discussions with a diverse subset of potential participants to review draft materials.
• Pilot testing: Involve a small group of participants in a pilot run of the training. Collect feedback on the clarity, relevance, and comprehensiveness of the materials.
• Open review: Allow participants to comment on draft materials through a shared online platform, offering them the opportunity to suggest additional resources or case studies.

Step 2: Getting feedback

Outcome is important: we need to know if we achieved what we set out to. However, if we want to continuously improve, we also need to know _how _we achieved it.

Feedback survey and/or focus groups

Surveys and questionnaires are an important tool in checking how well a training session is being received.

For multiple training sessions, it’s a good idea to ask for feedback a few times during the training:** at the end of each session to hear people's first thoughts, and then again halfway through the whole training to understand their broader experience. **

Where the resources are available, feedback focus or discussion groups can also be a great way to dig deeper into the impact of a training initiative. In these groups, everyone gets to share what they think about the training in a respectful, inclusive, and safe space

For examples of feedback questions you can ask via surveys or focus groups, download our feedback question template.

After getting feedback, the next step is to look at all the answers carefully to find out the strong points of the training and the parts that need to improve. You can use this information to make the training more helpful and effective, improving and adapting to what trainees need.

Step 3: Skill Check & Lessons Learned (output)

Now you’ve completed step 2, which focuses on participants’ experience of training — Step 3 focuses on if that learning stuck, once the course has finished.

**Learning pulse check **

Feedback forms are a useful tool where participants can express what they have learned, and any areas they feel they need further information on. It’s also a good starting point to get participants to think about how the learning acquired can be applied in participants day-to-day roles — this is covered in more depth in step 4.

Download our ‘learning pulse check’ question template for question examples.

Knowledge tests

A knowledge test aims to assess what knowledge the participant has acquired during the training. In a knowledge test,** the questions you ask should be specific to the training that you have conducted.**

As this is specific to your training, you should ask your trainer to produce a knowledge test based on the training content.

Step 4: Learning in action review (output)

In step 4, the emphasis shifts to understanding how participants can apply the knowledge acquired in real-world settings.

Activities carried out in step 4 should be conducted approximately 3 months following the completion of the training, leaving enough time for participants to apply the learning within their job roles.

Case studies

In this activity, trainees should be presented with relevant case studies based on real public service scenarios. You can then follow up with reflection questions, encouraging participants to apply the knowledge they have acquired when analysing case studies and propose solutions.

Download our case study reflection question template for question examples.

**You should ask your training provider to provide some case study examples if they have not already. **Take a look at our interactive maps for some examples of case studies in the following areas:

• Gender Equity: Learn from your neighbours
• Data & Digital: Learn from your neighbours

Role-Playing Exercises

Role-playing exercises can be a great way to simulate real-world scenarios. Participants can be assigned different roles within a public service context, such as a policymaker, a community representative, or a collaborating partner. These exercises should be designed to mimic challenges that they might face in the participant’s job roles. This method not only tests the application of the learned material but also enhances problem-solving and decision-making skills in a controlled environment.

Project Assignments

If appropriate, assigning projects that require participants to apply their training in their current job roles can be another effective method in encouraging applied learning. These projects should be relevant to their work and encourage them to integrate new strategies and approaches learned during the training. The completion of these projects can be followed by a presentation or a report submission, where participants showcase their application of the training and the outcomes achieved.

**Feedback Sessions / peer-to-peer learning **

After case study, role playing, or project-assignments, take some time to conduct feedback sessions. These sessions should be structured to allow participants to reflect on their performance, discuss the challenges they faced, and receive constructive feedback from peers and facilitators. This process not only helps in cementing their learning but also promotes a culture of continuous learning and improvement.

Incorporating peer-to-peer learning into these feedback sessions can be advantageous. Participants are encouraged to share their practical experiences of applying the training in their job roles, discuss challenges, and exchange ideas. This not only strengthens their learning but also cultivates a collaborative learning environment.

Action Plans

Asking trainees to develop action plans is a proactive step towards ensuring the application of the knowledge acquired during the training.

Developing an action plan encourages trainees to think critically about how they can apply the learnings from the training in their respective roles, considering various aspects including stakeholder engagement, measurable outcomes, and continuous learning.

Download our simple learning action plan template.

It aims to guide participants in creating action plans that are both practical and aligned with the goals of their organisations. See figure 2 for some key considerations (or questions) to keep in mind when developing your action plan.

Figure 2: Key questions to consider when developing an action plan.

Step 5: Outcome Evaluation

Step 5 focuses on the long-term impacts that a training intervention has had on trainees’ behaviour, which means assessing how the training has influenced their understanding, attitudes, and actions.

This step and associated evaluation activities should be completed 6 months to 1 year after the training has been completed.

Post-training survey

Pre- and post-training surveys are valuable tools for measuring trainees' knowledge gain, behavioural shifts, and attitude changes. By comparing the two sets of results, you can evaluate the effectiveness of your training programme and identify areas for future initiatives.

**You should ensure that any post-training questions align with those in the pre-training survey (in order to measure progress / changes to results). **

For examples of the questions you could ask, download our post-training survey template.

Revisiting Action Plans

As detailed in step 4, ‘Action Plans’ serve as a roadmap by which the trainee intends to apply the learnings from their training.

In step 5, these action plans are revisited to assess the long term impact of training in relation to actions taken.

Step 6: Impact Evaluation

The sixth and final step is centred around evaluating the outcomes of your training intervention at a wider organisational level and, ultimately, a societal level. The introduction of new policies, or improvements to existing ones is an example of this.

This is the most challenging and time-consuming step. However, it's also the most important to understand whether quality training in the public service leads to better policy making — when working with external evaluation partners, ensure this step is considered in the proposed evaluation approach.

Step 6 should be considered on a longitudinal basis - i.e. Annual evaluation activities to track organisational metrics and outcomes related to the goals and objectives of the training intervention. Circulating attitudinal surveys can be an effective method to capture this longitudinal data, without investing too much time or resource.

Measuring this final step can be difficult, as the training intervention offered to participants may not be the only causal factor in knowledge acquisition, application and behaviour changes. It’s also worth bearing in mind that the further you attempt to measure the impact from the date of the training itself, the greater the difficulty in attributing cause and effect.

Table 1: Ahead of designing and carrying out any specific evaluation activities with stakeholders, the evaluator should ask a set of specific questions.

Attitudinal surveys

Annual surveys are an effective method to capture longitudinal data on individual knowledge, attitudes and behaviours within your organisation. Similar to the questions asked ‘post-training’ in step 5, these questions can also be used to help you plan further training initiatives based on where your employees think the need is greatest.

For examples of the questions you could ask, download our attitudinal survey template.

Analysis of policy development

This activity seeks to analyse any changes in organisational or governmental policies that have been influenced by the training intervention.

Figure 3: Key steps in the analysis process.

👋 We would love to know what you think! Leave us a comment below to share your thoughts.

(Image Credit: Unsplash)