Evaluation

Why evaluate your project?

“An effective charity considers the impact that it wants to have and actually has on the people who benefit from it…” Charity Commission (2004)

Most organisations are required to evaluate their projects, usually for reporting purposes as a lot of funders require certain objectives to be met and reported on. This often involves reporting on the number of service users seen, workshops and one to one’s held, and the results of any feedback forms. But what if you want to go beyond this, to capture the real impact made on service users and the communities that you work with?

In his original work, Donald Kirkpatrick (1959) made recommendations for evaluation that have laid the basis for a lot of thinking in the subject ever since . He argues that the evaluation of programmes should concentrate on four levels:

• Level 1: Reaction – evaluating the reaction of participants to the intervention

• Level 2: Learning – measuring the knowledge, skills and attitudes gained from the intervention

• Level 3: Behaviour – measuring the changes in behaviour that resulted from the intervention

• Level 4: Results – relating the results of the intervention to wider organizational objectives

This framework is most often applied to the evaluation of training programmes but can be usefully applied to many areas of work in the financial inclusion and financial capability sector. A lot of evaluations that take place within the sector only capture reaction and often with good reason: it’s the easiest to capture, it helps organisations to improve their programmes and it can be easily analysed right away. However, to capture real change and attribute them to interventions, it’s necessary to go beyond this level, to look at changes in learning and behaviour and map these back to the goals of the organisation and even the sector as a whole.

Capturing levels 2 – 4 can be a real challenge, especially as they often involve long time frames and a lot of ‘soft indicators’ or indicators that are difficult to measure. However, models of best practice do exist and as the sector makes progress it’s important to share these models and build on good practice in order to develop standards across the sector. Please view the RBS Innovate page to see what we are looking to accomplish through RBS Innovate project in 2010 and also the case studies page to see examples of two projects which have completed successful evaluation. 

Resources

• Charities Evaluation Services http://www.ces-vol.org.uk