Crowdsourcing Citizen Feedback on District Development in Ghana Using Interactive Voice Response Surveys

"In Ghana, as in most countries, there is no systematic or easy way to provide inputs on citizen satisfaction and preferences to decision-makers."

This practice paper shares the results of a research project involving a crowdsourcing platform which uses sms (text messages) to gather citizen feedback on social services in order to increase government accountability. The paper was published as part of the Making All Voices Count initiative, a citizen engagement and accountable governance programme that seeks to build an evidence base on: what works in technology for voice, transparency, and accountability; how it works; and why.

In mid-2015, VOTO Mobile, in partnership with the Ghana Center for Democratic Development (CDD-Ghana), applied to Making All Voices Count for a practitioner research and learning grant. VOTO Mobile is a Ghanaian tech company and social enterprise, whose mission is "to increase participation and accountability in the services delivered to citizens, and to empower communities to collect and share information to drive positive social change." One of the tools it uses is the interactive voice response (IVR) survey. This entails survey respondents being called on their mobile phone and prompted by pre-recorded messages to respond to questions by pressing buttons on the phone; the results are then stored on a database.

CDD-Ghana is a non-profit organisation focusing on good governance. It has worked with United Nations Children’s Fund (UNICEF) to develop the District League Table (DLT), a tool for measuring and highlighting discrepancies in local government service delivery in different sectors. The DLT ranks Ghana’s 216 districts by their level of development according to six sectoral indicators: education, sanitation, water, health, security, and governance. The DLT relies only on readily available datasets, and through this collaboration VOTO Mobile and CDD-Ghana wanted to see if they could collect data on district services from a citizen perspective in a rapid, cost-effective manner to feed into the DLT.

Specifically, the two organisations wanted to conduct this research in order to find out the following:

• whether VOTO Mobile’s IVR methodology would be suitable for gathering data for future editions of the DLT
• how far the DLT reflected the priorities of Ghanaian citizens
• how IVR could be adapted to maximise the response rate of rural women.

The practice brief offers a description of the VOTO Mobile platform and the DLT and how the survey process was set up. In order to find out what increases the response rate of women, the project tested whether language option, gender of the recorded voice, the time of day of the call, the inclusion of encouraging messages, and the number of response options would result in more women responding to the survey. Results showed, for example, that response rates were a statistically significant 22% higher with a female voice than with a male voice, that calls in the morning or afternoon were slightly more likely to have complete responses than those in the evening, and that the use of encouragement messages to women to complete the survey had no impact.

The brief also shares the results of the survey. For example, in terms of overall satisfaction with the six DLT sectors, the survey revealed that Ghanaians were mostly unhappy with the services provided to them, showing particular discontent with sanitation. As for the issues that citizens thought were priorities, both rural and urban citizens believed that it was important to improve education and health, but people living in rural areas accorded these issues a higher priority, along with improving the supply of water.

Regarding the extent to which the DLT results correlate with a crowd-sourced method of getting inputs about citizen priorities, the findings showed that there was a very slight district-by-district correlation between the DLT and IVR survey results in the fields of education and water. However, general findings were inconclusive, which VOTO Mobile interpreted to mean that the indicators used by the DLT are not always good predictors of citizen satisfaction for these sectors.

The brief also reflects on the challenges facing a tech provider and a civil society organisation (CSO) such as CDD-Ghana in working together to develop tools for citizen engagement in local government monitoring. One of the challenges they faced was aligning the information needs of CDD-Ghana with an understanding of the limitations and possibilities of administering mobile surveys. For example, in some instances it was necessary to change the type of question in order to minimise survey drop-off rates, which required some compromises on behalf of the CSO in relation to the information they could collect.

Overall, the paper concludes, although the tool as such did not offer the solution both organisations were hoping for - to be able to use survey feedback to feed into the DLT - they gained much from working together on this research project. For example, VOTO Mobile has used some of the findings to tweak its approach to IVR, including changing the voice of the audio recording to female, calling in the evening or afternoon to reach rural female respondents, and designing opinion questions with simpler and fewer response options. Going forward, VOTO Mobile noted that there was "a greater need for ongoing collaborations between technology providers and CSOs, and potentially the need to create opportunities for CSOs to learn about and test various technologies that could enhance their work. As a result of this collaboration, we have been able to continue our own organisational learning with respect to how we can most effectively communicate with first-time users of mobile surveys."