By Dr Natalia Lewis, Senior Research Fellow in Primary Care, Centre for Academic Primary Care, University of Bristol and Researcher in Residence, Bristol, North Somerset and South Gloucestershire Integrated Care Board.
This blog is in two parts. Read the second blog on the project final stage and report.
Since December 2022, I have been working part-time as a researcher in residence at the Bristol, North Somerset and South Gloucestershire Integrated Care Board (BNSSG ICB).
The Researcher-in-Residence (otherwise known as Embedded Researcher) model involves co-locating researchers within non-academic organisations to enhance the role of research evidence in informing decision-making. I applied for this job because I wanted to understand how decision-making in the real-world health system happens and how my research skills and expertise can be used to make local policies and practices evidence informed.
My post was established to support development of an evaluation framework for the Woodspring and One Weston Locality Partnerships within BNSSG ICB. These Locality Partnerships are made up of NHS providers, the local authority, and voluntary and community sector organisations, serving a population of over 225,000. The Partnerships run tailored programmes and initiatives delivering proactive and joined up holistic care to the population.
Scoping exercise
I started with a scoping exercise to understand the Partnerships context and identify evaluation approaches that are feasible and acceptable in their context.
Very supportive, experienced, and knowledgeable colleagues in the ICB Clinical Effectiveness Group became an essential part of my team. Delivery Director from the One Weston Locality Partnership David Moss and Clinical Effectiveness Programme Managers Ruth Whateley and Kate Strong worked as knowledge brokers facilitating relationships, interactions, and engagement with ICB stakeholders.
I spent the first two months reading internal documents and meeting with Partnerships stakeholders to understand how they evaluate their work. I learned that there is some research capacity within ICB and in Local Partnerships, but research and evaluation processes work differently compared to universities. The former has limited evaluation capacity and short turnarounds.
Locality Partnerships – complex systems
From a researcher perspective, Locality Partnerships are complex systems. Partners from NHS, local authorities and voluntary and community sector organisations come with their own ethos, philosophy and language. They represent differing organisational cultures nested within wider social and political environments.
The partners share strategy, some structures and resources. They have developed connections and built trust and relationships. I observed partners’ willingness and readiness to work together towards agreed outcomes. These enablers of integrated working can be challenged by differing cultures, political agendas, competing priorities, fast changing environment, short turnarounds, multiple initiatives happening in parallel, and limited research and evaluation capacity and capability.
It became clear that the proposed evaluation approach should build on the existing relationships and trust, agreed outcomes and approaches, engage Partnerships stakeholders in the co-development process, and use existing evaluation capacities and capabilities.
Presenting the findings
I presented findings from the scoping exercise to the Partnership Board, Delivery Group, and Evidence into Practice Advisory Group and proposed to take a theory of change approach which fits the Partnerships context. In February 2023, the Partnerships agreed to take the theory of change approach with the intention to:
- develop a theory of change which tells a story about how the two Local Partnerships work to produce intended outcomes;
- draw a logic model which represents the theory of change (a logic model is a graphic depiction (road map) that presents the relationships among the resources, activities, outputs and intended outcomes for the Partnerships work);
- map an evaluation framework on the logic model (to help prioritise data collection and analysis in exploring the main aspects of the Partnerships’ work).
The theory of change approach recommends working with stakeholders to agree and prioritise the inputs, activities, outputs, mechanisms of change and outcomes. Working in collaboration with the Dartington Service Design Lab, experts in complex systems, Head of One Weston Locality David Moss, and ICB Clinical Effectiveness Programme Managers Ruth Whateley and Kate Strong, we prepared and facilitated four workshops in May-October 2023.
Stakeholder workshops
Workshop 1 focused on the practices of the two Partnerships, local system dynamics and resources which enable and constrain the pathway to change.
Workshop 2 targeted definitions of outcomes at individual, service, locality, and ICB levels.
Workshop 3 focused on mechanisms of change that connect the resources and practices of partners (as defined in workshop 1) with the different levels of outcomes (defined in workshop 2).
Workshop 4 aimed to test the logic of the theory of change and set up the next phase which will develop an evaluation framework for the Partnerships.
Professional stakeholders from the NHS, North Somerset Council, and the voluntary and community sector organisations, attended the workshops. Participants co-developed:
- asset maps
- problem trees
- persona profiles
- mind maps on the barriers and facilitators to integrated working
- summaries of local work activities
- outcomes and mechanisms of change for partnership working, Ageing Well and Dying Well workstreams
- draft theories of change for partnership working and Ageing Well and Dying Well workstreams.
Stakeholders valued the time to engage and develop new connections with local partners. They appreciated the opportunity to co-develop shared outcomes and processes.
Next steps
Our team are analysing the workshops’ outputs to finalise the three theories of change. The next step will be to map an evaluation framework on the theories of change, which will allow the Local Partnerships and system to evaluate and mutually recognise the impact of their integrated work.
Listen to the podcast where David Moss, Natalia Lewis and Ruth Whateley talk about this collaborative project, which brought together academics, policy makers, and service providers.
For more information about this project, contact: Dr Natalia Lewis.
See Part 2 of this blog: Co-developing a theory of change to evaluate integrated working in two Locality Partnerships – Part 2