A common issue with qualitative impact assessment is how to organise and make sense of large quantities of textual data, and to do so in a way that is transparent, so that generalisations drawn from it can be peer reviewed. Various software packages are available on the market to assist with the task, and most can be used with the QuIP coding system, with some preparation to set the systems up appropriately. At Bath SDR we have invested in our own bespoke data analysis solution which is designed to speed up and standardise the coding and analysis process for our reporting purposes, but this does not preclude others from using the QuIP coding approach in other qualitative analysis software.
We use a thematic analysis inspired approach to coding, employing standard routines to aid speed and transparency. Coding is predominantly inductive, identifying specific causes or drivers of positive and negative change.
Once complete this allows for detailed analysis of different cause-outcome configurations which can be clustered together into thematic areas of interest.
The more deductive part of the coding relates to attribution. Unlike the field researchers, QuIP data analysts need to be fully briefed about details of the project that they can assess how the responses relate to the project’s theory of change. Cause-and-effect statements are coded according to whether they (a) explicitly attribute impact to project activities, (b) make statements that are implicitly consistent with the project’s theory of change, (c) refer to drivers of change that are incidental to project activities. These statements are also classified according to whether respondents described their effects as positive or negative.
Once coding is complete it is possible to query the data set, looking for causal relationships and patterns which can be visualised in network diagrams – causal maps. This allows analysts to see causal relationships between drivers and multiple related outcomes. In order to manage, analyse and present this data in a meaningful way, that ensure the respondent voice is kept at the heart of the process, Bath SDR uses an analytics platform called Microstrategy.
The interactive dashboard allows us to:
- analyse causal connections at aggregate or granular levels
- filter and compare by attribution, variables and respondent characteristics
- visualise data to help report on complexity
- connect immediately to the coded data behind the visualisations
To read more about QuIP visualisations and coding, please see our brief How QuIP Data is Analysed.