The importance of closing the feedback loop: a perspective from Uganda
10 May 2021 | QuIP Articles|
Since 2016, when QuIP was first introduced in Uganda, seven studies have been conducted here to date, across diverse sectors such as:
- Sexual Reproductive Health and Rights (SRHR)
The QuIP process was designed to include an unblindfolding/validation workshop with respondents towards the end of each study. This is for two main reasons:
- In an effort to reduce bias, the QuIP methodology typically involves ‘blindfolding’ the researchers and respondents to some extent. This approach therefore places an ethical duty on the commissioner to conduct a comprehensive debrief to disclose the purpose and the findings of the study once the data collection is complete.
- More importantly, as QuIP focuses on the voices and experiences of intended beneficiaries, it is best practice to verify initial analyses and interpretations with participants.
These workshops usually involve a presentation of the findings, followed by a focus-group-style discussion. The aim of these sessions is to ensure greater transparency, generate additional insights, and confirm/adjust data interpretation. They also help to facilitate deeper reflection on the results leading to recommendations and actions.
In Uganda our research team have held two successful QuIP validation workshops to date. Before COVID-19, we had not considered the possibility of hosting a virtual workshop, so the first meeting was held face to face. During the pandemic, in 2020, we used mobile teleconferencing. In both cases, commissioners debated whether or not to include these workshops in the QuIP study, based on questions of cost and the value-add to the evaluation process.
“Should commissioners and evaluators should have a choice to include or exclude unblinding/validation workshops?”
I would argue, no.
Firstly, there is an ethical imperative to give respondents feedback, especially when conducting blindfolded research. I have not participated in a research study where participants have not asked the question; “how shall we get to know what you have found out?” Furthermore, validations allow for respondents to participate more fully in the evaluation and save the process from being an exercise in information extraction. Participants who have attended our validation workshops have appreciated the fact that we go back to share with them the outcomes of the study and to discuss the findings further.
Secondly, we have come to learn the importance of not finalising findings until the participants give their say so. The respondents’ feedback, by way of confirmation, further explanations, and clarifications have always enriched our analyses and interpretation. Sometimes, during preliminary data analysis, we come across ideas which are incomplete, or responses which raise unanswered questions. The workshops provided a mechanism to follow up on emerging issues and deepen our analysis to present a more complete picture in the final report. Moreover, we could check that our analysis and interpretation reflected the actual experiences and realities of the intended programme beneficiaries. That is why I argue that our findings should not be final until they have been confirmed by those whose perspectives we seek to understand and interpret.
Having made the case for validation workshops as a matter of methodological and ethical importance, one might say but what about the cost? Given the significant amount of funds invested in development programmes, I would expect commissioners to be equally interested in assessing the impact of their interventions in the most accurate way. Face-to-face meetings do require money for transport reimbursements, accommodation, meeting venues, and meals, but they offer an opportunity to make a visual presentation followed by rich and engaging discussions, which are arguably well worth the cost. Workshops held via teleconference do not have all the same benefits but are more affordable as the only additional expense to the facilitators’ time is phone credit for the host. Remote solutions can also enable more participants to be reached – provided they have access to a phone, directly or through a friend, relative, or community member.
Rather than the question of holding or not holding validation workshop, I think the question should be how to hold one effectively within (inevitable) budget limitations.