Exploring alignment between QuIP and SROI principles
| 22 November 2021 | Latest QuIP News
This blog celebrates the publication of a joint Social Value UK/Bath SDR paper comparing the key similarities and differences between QuIP and Social Return on Investment, an approach used widely across the world.
Anyone who works in monitoring & evaluation knows that this is a world peppered with methods, approaches and acronyms. With QuIP we have, of course, added to the ever-growing list of approaches an evaluator should consider when planning their work – for which we may not always be thanked! (Do check out Better Evaluation if you want help cutting through the maze). I was recently invited to participate in two very interesting seminars organized by the Walton Family Foundation who brought together a truly international group of academics and practitioners who have a shared interest in exploring causality amid complexity. WFF have a particular interest in ‘causal’ approaches – adaptations to traditional quasi-experimental design approaches and specific methods like QuIP, Outcome Harvesting, Most Significant Change, Contribution Analysis, Qualitative Comparative Analysis, etc. More on the WFF work to follow…
The consensus at these seminars was that this range of approaches offers a great deal to the world of evaluation but that we have not always made it easy for practitioners to understand what differentiates the different methods and help them to understand which approach to use. We attempted to do this in a limited way in our briefing paper, but more seriously in Chapter Two of Attributing Development Impact which compares QuIP with 30 different approaches to development impact evaluation, including SROI. This new paper goes beyond the two sentences in this chapter, and is devoted in its entirety to considering each step of SROI and how QuIP compares – with a particular focus on how the approaches can be complementary.
I first presented QuIP to a Social Value UK conference in 2019 where it received a great deal of interest from SROI (Social Return on Investment) practitioners who were interested in how it could boost the qualitative aspects of an SROI study. Since then, I have been working with the team, particularly Craig Foden and Nicola Lynch, on how aspects of QuIP may be useful for SROI practitioners. A global pandemic somewhat got in the way of things for a year or so, but we are really pleased to have finally published some thoughts which we hope are useful to users of both QuIP and SROI.
This was complemented by a workshop earlier this month where Nicola shared her experiences of using aspects of QuIP in her work, and we discussed the potential for using QuIP’s exploratory/ goal-free approach to interviews and focus groups in an SROI study, as well as using causal mapping in the analysis of the qualitative data. QuIP is firmly a qualitative approach (the clue is in the name!), whereas SROI aims to quantify or value impact – in that the two approaches firmly differ. However, there are activities within an SROI study which require tracing and understanding impact in more detail, and this is a fertile area of joint development.
The paper presents both approaches and then considers QuIP’s response to each Social Value Principle in turn. A brief summary of some of the points considered in last week’s workshop is below, but please do see the paper for more information.
Principle 1: Stakeholder involvement
Both approaches have a strong focus on giving all stakeholders a voice, but at different points
- SROI involves all stakeholders in design of an assessment, including those with little or no power
- This does not occur in QuIP as questions are framed in advance around expected changes, but the interview is exploratory and therefore open to unexpected drivers and changes – it is led by the respondent. Inclusion of stakeholders in discussion about findings follows analysis of initial data.
Principle 2: Understand changes
Both approaches have a strong interest in causal mechanisms
- SROI involves stakeholders in defining all the outcomes experienced or expected to be experienced. The analyst decides which outcome is the most important to measure.
- QuIP collects all reports of change; all are included in causal maps, but filtering these maps to answer key questions could be seen as a similar process. The extent to which change can be attributed to the intervention is assessed by the analyst during the coding process.
Principle 3: Value the outcomes
In SROI the process of valuing outcomes is directed by stakeholders; this is not possible in QuIP
- SROI involves stakeholders in ranking/valuing the outcomes they identify.
- Due to the exploratory way in which stories are collected, stakeholders cannot be asked to place a relative value on changes. Instead, analysis in Causal Map helps to identify where important patterns, clusters or outliers are.
Principle 4: Only include what is material
What to include are limited by relevance and significance in SROI – similarly with QuIP but less systematically?
- SROI follows key tests for materiality in inclusion of outcomes
- These tests for relevance and significance could be similar to the decision over how to filter causal maps and use and attribution coding at the point of data analysis in QuIP
Principle 5: Do not over-claim
SROI and QuIP share a focus on reporting attributable changes
Principles 6 & 7: Be transparent & verify
SROI and QuIP are transparent about how findings are reached and reviewed
- QuIP has a focus on transparency particularly through the analysis of data in Causal Map, but there is no systematic verification process
There is more work to do together, and we have plans – watch this space!
Comments are closed here.