New journal article! Causal Mapping for Evaluators
| 24 October 2023 | News
Steve Powell shares news of the publication of our article on causal mapping in the journal Evaluation. We have come to think of causal mapping as (part of) the logic of QuIP analysis, but until now had not set this in the context of an existing body of work which has contributed to the development of an approach particularly suited to evaluation. This article is now published online and will feature in a special conference edition of the journal Evaluation forthcoming in January.
We are very happy that our article Causal mapping for evaluators has finally been published! We’ve had a shared folder called “Paper 1” almost since the three of us (Fiona, James and I) first met five years ago and started working on what became the Causal Map app. James and Fiona were interested in an app for identifying causal claims within QuIP Interviews, and I was interested in synthesizing evidence from other evaluations and across evaluations.
In the meantime, other publications came and went but Paper 1 all about causal mapping (with a small ‘c’ and ‘m’) remained the paper for a Big Think, always waiting for an elusive away day or the ever-ambitious summer holiday plans. Work went very slowly… We were describing, or attempting to describe, a practice that we were only gradually developing and didn’t fully understand ourselves. In a delightful and unplanned process, the three of us gradually worked towards a common ground through the paper. I was starting very much from a position of someone trained in statistics, and had been working in a broadly quantitative way “with a generous dash of qual” for many years. I’ve woken up in a whole new world of qualitative delights through the encounters and conversations elicited through writing this paper.
I met Gary Goertz through James, and this connection opened up the quite extensive causal mapping literature – beginning with Robert Axelrod which I, like most evaluators, had not been familiar with. I was able to persuade Fiona and James that causal mapping as described by others is a pretty exact fit for what we were already doing. Evaluators are much more familiar with schools of thought with names like “Systems Mapping” which are very similar in some ways but differ in other important ways. That quickly gave us the title and the target audience of Paper 1: Causal mapping for evaluators.
The article which has resulted from this work aims to share our experiences of using the approach of causal mapping within the sphere of evaluation and to outline the scope for evaluators to use the approach more systematically and widely. This article starts with a brief history of causal mapping and clarifies some definitions. It then reviews causal mapping within evaluative practice, distinguishing the tasks of gathering data, coding causal claims and answering evaluative questions. The final section concludes with reflections on the strengths, weaknesses and future potential of causal mapping in evaluation.
Example causal map of factors reported as leading to more varied food consumption
One issue which prompted much discussion during the writing was the question of whether what we call causal maps can really be understood as models of actual causal relationships in the world? Can we do “causal inference” from a map to predict or explain what happens out there in the world? Causal maps are usually understood as explicitly recording what people believe, not necessarily what is true in any objective sense. But if we treat them only as models of people’s beliefs about causal pathways, there may be a perception that they are less interesting and valuable – merely of psychological interest. Our response is to see causal maps as repositories of evidence for those pathways, often evidence which can help to directly answer interesting questions such as “How much evidence is there for a path from X to Z via Y?” and “Does the amount of evidence for that path differ between different groups of respondents/types of evidence?“. Thinking about how to tackle these sorts of evaluative questions has led to rapid advances in the analysis capability developed within the Causal Map software to allow users to pose those crucial questions of qualitative datasets, but it also requires an acknowledgement that what people believe causes change is indeed important evidence worth considering when making evaluative judgements.
There is much to discover and enjoy in the world of causal mapping – we hope you enjoy digging into this long awaited article! A pre-publication version can be accessed here.
Comments are closed here.