By Dr Natalia Lewis, Senior Research Fellow in Primary Care, Centre for Academic Primary Care, University of Bristol and Researcher in Residence, Bristol, North Somerset and South Gloucestershire Integrated Care Board.
Back in 2022, I started my secondment as a researcher in residence at the Bristol, North Somerset and South Gloucestershire Integrated Care Board (BNSSG ICB). I was tasked with supporting the development of an evaluation framework for two Locality Partnerships.
My first blog and podcast in December 2023, described the process of co-developing theories of change for the Partnership Working, Ageing Well and Dying Well workstreams with stakeholders from the NHS, local authorities and voluntary and voluntary, community and social enterprise sector from the two Locality Partnerships. Co-developed from bottom up, these theories of change incorporated local knowledge and priorities which were linked with the BNSSG ICB population health outcomes. In this blog I discuss the project final stage and output.
Report and guidance
In January-March 2024, we asked stakeholders to apply the co-developed theories of change to evaluation, planning and reporting purposes, documented their examples, and prepared an end of project report: Developing an evaluation framework for Locality Partnerships: project report and guidance. The report describes the three theories of change and explains how to use them in practice. It provides the practical tool for building an evaluation plan around theory of change that is tailored to a project or service.
Some key points for those using the report to note are:
Adapting theories of change
You can adapt the three theories of change to your project or service aim and objectives to justify a business case or funding proposal, build an evaluation plan, and develop quality improvement projects. By using our report and guidance, you can capitalise upon co-developed knowledge instead of starting from scratch. This approach saves time and resources – in the ICB context characterised by short turnaround time and limited evaluation capacity. If applied across multiple services and projects, it can bring synergy and improve coordination across evaluations.
A working document
The report is a working document targeted at people who plan, fund, deliver and evaluate projects or services. We understand that every project or service needs a specific evaluation plan that is tailored to its aims, objectives, evaluation needs, and capacities. However, despite seeming differences across projects and services both within and between Locality Partnerships, there are a lot of similarities and overlaps in the problems, inputs, activities, and outcomes.
Toolkit for evaluation
The report contains a toolkit for developing evaluation plans specific to each project or service. You do not have to start from scratch every time. You can adapt our theories of change and application examples to your evaluation needs. The report is available in a PDF and Word version with editable infographics to make the adaptation process and presentation of results easier.
Step-by-step guidance
The report presents the relationships between the three theories of change and describes each one in detail. Each theory of change is reported as a narrative describing the problem, resources, activities, reach, mechanisms of change, outcomes and enablers linked in a causal pathway. Each theory of change is illustrated with simple infographics.
There is a chapter providing step-by-step guidance on how to build an evaluation framework around your theory of change. For example, if you want to evaluate a new project or service, you can use one or two theories of change which are closest to that project or service.
Working with relevant stakeholders, you can map project/service-specific problem, resources, activities, mechanisms, and outcomes on the relevant categories from our exemplar theories of change to explain the pathway to intended outcomes. Then you can map the pathway on the theory of change infographics, identify relevant measures and plan the evaluation.
Next steps
With BNSSG ICB managers, we have developed a communication plan to inform stakeholders about the report and embed it into relevant policies and processes. The report is now in the hands of the ICB knowledge brokers who will enable translation of the co-developed knowledge into policy and practice.
For more information about this project, contact: Dr Natalia Lewis.