Using Mobile Voice Technology to Improve the Collection of Food Security Data: WFP's Mobile Vulnerability Analysis and Mapping

"Reliable information is critical to understanding food security trends, assessing crises and informing decision-making. However, traditional methods for collecting this information - in particular food security data at household level - are often expensive, slow and cumbersome, which limits its effective use for decision-making."

This case study looks at the United Nations (UN) World Food Programme (WFP)'s innovation into Mobile Vulnerability Analysis and Mapping (mVAM), a programme that integrates mobile technology, including SMS, Interactive Voice Response (IVR), and live calls, into WFP's established food security monitoring systems. This case study is based on a review of project literature and 13 interviews with project staff, partner staff, advisors, and wider stakeholders across the humanitarian sector over a period of two months in 2015.

As the case study explains, WFP's traditional approach to food security monitoring has relied primarily on face-to-face interviews. Although this method typically delivers robust data, it is also slow, expensive, and cumbersome. Data collection can cost between US$20 and US$40 per household. It takes 6-8 weeks for reports to be published, meaning information is often already dated by the time it comes out, and decision-makers do not have the information they need to shape programming effectively. Furthermore, faced with the challenge of reaching affected populations in highly insecure locations, humanitarian organisations are increasingly turning to remote management and monitoring.

In that context, mVAM piloted mobile voice technology in the Democratic Republic of Congo (DRC) and Somalia through contacting a panel of regular respondents with live calls and IVR. Each month, respondents answered questions on their household food consumption and coping strategies, generating high-frequency data that show trends in food security. Although the mVAM tools are best used alongside face-to-face data collection in "mixed-mode" systems, they also enable the collection of data in situations where violence or disease prevent face-to-face surveys from taking place. The mVAM team has documented its learning and achieved proof-of concept for the application of mobile technologies to food security monitoring. mVAM is now operative in 11 countries, with a plan in place for expansion to 30.

To understand how this transpired, the case study uses a model based on 5 stages:

1. Recognition of a specific problem, challenge, or opportunity to be seized - With the aim of integrating mobile technologies into WFP's existing systems in order to collect food security data more effectively, in 2013, mVAM conducted a field test of SMS data collection in DRC in partnership with private polling service GeoPoll. This established the potential of SMS as a high-frequency, low-cost, and user-friendly approach to data collection but also flagged a number of limitations, particularly to collecting complex types of data and in areas with low literacy rates.
2. Invention of a creative solution or novel idea that addresses a problem or seizes an opportunity - The technologies mVAM employed had already been in use in the private sector, and elsewhere in the humanitarian and development sectors, for a number of years. WFP's innovation was to integrate these technologies into its existing food security monitoring systems and to use them to collect data on fewer indicators but with a higher frequency in remote and hard-to-reach areas.
3. Development of the innovation by creating practical, actionable plans and guidelines - The mVAM team built a 6-month preparatory phase into its plans, which made it possible to conduct baseline surveys and experiment with different options for IVR and live calls. The team worked collaboratively with country offices to select locations and draft plans. Face-to-face baseline surveys were conducted in both locations. These were an opportunity to recruit pilot participants and collect background information on respondents, establishing data against which mVAM results could later be compared. In DRC, it emerged only 24% of respondents owned a mobile phone, so the decision was made to distribute mobiles to all those who had signed up to the surveys. The mVAM team tested a number of platforms for IVR and selected the software Verboice for its free, open-source nature and user-friendly interface. There was regular contact with Verboice's developers, InSTEDD, which helped manage technical issues. mVAM trained other WFP staff in how to use Verboice and conducted around 300 test calls to ensure the call flows worked. After researching call centre options in both countries, the decision was made to place live calls "in-house", with WFP staff acting as operator. The team recruited call centre operators and drafted a manual for operators. The project team was proactive in building partnerships and bringing new ideas and expertise into the development process. For example, over time, the mVAM team built partnerships with companies that already had significant experience of mobile data collection, such as GeoPoll.
4. Implementation of the innovation to produce tangible examples of change, testing it to see how it compares with existing solutions - mVAM launched live calls in DRC in January 2014. In February, operators reported problems reaching respondents. A focus group discussion (FGD) revealed that respondents couldn't afford to charge their phones, so a solar charging station was set up in the camp in collaboration with the camp's incoming-generating activity management committee. The team was very quickly able to see it was generating credible data that made sense when compared with the data collected at the baseline and in relation to seasonal events (See Text Box 1). With a number of iterations, it also managed to increase response rates in DRC, from 40% in January, to 45% in February, to 67% in March. There were delays in beginning data collection in Somalia, as a result of difficulties in recruiting operators. Live calls began in Somalia in April. Local WFP teams continued to experiment with the IVR approach over time. For example, at the time of writing, the team in Somalia was planning to record survey questions with the voice of a popular local radio station host. Throughout implementation of the pilots, the mVAM team kept plans flexible and worked closely with local WFP teams to solve problems as they arose. It monitored progress regularly, using a project dashboard to collect key indicators.
5. Diffusion of successful innovations - taking them to scale and promoting their wider adoption - The mVAM team blogged regularly throughout the process, helping build affinity with the project within WFP from an early stage, and wrote or contributed to articles aimed at a wider humanitarian audience. However, it was an unforeseen set of circumstances that contributed most significantly to mVAM's diffusion: the 2014 spread of Ebola virus disease (EVD) spread across West Africa. Movement restrictions and quarantines prevented traditional food security monitoring from taking place. Talking to those directly affected by Ebola in person was not a viable option. So, WFP launched a full deployment of IVR and SMS data collection in Guinea, Liberia, and Sierra Leone in September 2014. The data showed low purchasing power, rather than food price increases, was the main barrier to household food access. Food security indicators were poorer in rural locations than in urban centres, despite having a comparatively low number of Ebola cases. These data were used to inform WFP's strategy for food assistance and enhanced its ability to advocate for funding and support for Ebola-affected communities. "The rollout of mVAM in the Ebola crisis enabled a clear and dramatic example of success early in the lifespan of the project. It marked a tipping point for mVAM, and rapidly increased its visibility and credibility within and beyond WFP."

The research team used evidence collected for this case study to assess the success of the mVAM innovation process against 3 criteria. Overall, it was highly successful in both increasing evidence and learning and in achieving wide adoption. For example: "Although WFP is not actively seeking to diffuse the survey technology externally, this innovation has been successful in achieving wider adoption owing to partnership with the private sector....However, the review of mVAM highlighted the need for considerable improvement in the way mVAM data are presented and disseminated, including the quality of data bulletins, dashboards and reports. The review also recommended the mVAM team place greater emphasis on engaging with its stakeholders, including donors, implementing partners and beneficiaries." It was moderately successful in delivering an improved solution: While mVAM has significant potential, particularly for high-frequency data collection in insecure areas - SMS costs US$3-6 and live calls, and IVR costs $7-9 per questionnaire, with a 1-week turnaround time for SMS and 2 weeks for live calls and IVR - further adaptation and improvement will be required as the programme expands. (For instance, attention needs to be paid to the level of community engagement and beneficiary participation built into the programme, and processes and safeguards need to be in place for protection and privacy of respondents.

Next, the case study explores 6 factors generally held to be fundamental to successful innovation processes and the way in which each works in the context of mVAM. In brief:

• Managing relationships and setting common objectives - "While a structured approach to relationship management was viewed as helpful to the success of the innovation process, informal relationships and infrequent communication with individuals implementing similar projects also contributed significantly to the success of the pilots."
• Dividing tasks and responsibilities - "The clear division of tasks and responsibilities was observed as being supportive to the success of this innovation process. In particular, working through existing structures of command and communication in the EVD response enabled rapid scale-up and helped manage risks."
• Resourcing an innovation - "Each of the pilots and subsequent deployments have required significant human resources."
• Flexibility of process - "For example, achieving high response rates (taking rates from other projects, such as Listening to Dar, as a benchmark) required significant trial and error. The impact of this flexibility on mVAM's success is reflected in rising response rates."
• Assessing and monitoring risk - "The review highlights the need to mainstream privacy and protection commitments in future activities. Lack of attention to these issues significantly weakened the degree to which this innovation offered an improvement over existing practices. However, the constant learning and adaptation built into mVAM's development were an effective way of managing risks to its success because issues could be addressed as soon as they arose."
• Drawing on existing practice - "The outward-facing attitude of the project team brought many new ideas into the innovation process and ensured it built on - rather than replicated - past experience."

It was also noted that there was expertise and passion within the project team, as well as support from senior leadership, which possibly contributed to the innovation process. Several emerging lessons are identified for best practice in innovation, such as: "Great ideas come from diverse sources, at all levels (from senior management to beneficiaries), both within and outside the humanitarian sector - reaching out and creating a network of interested individuals and organisations is crucial."

This study is one in a series of 15 case studies, undertaken by Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) in partnership with Enhancing Learning & Research for Humanitarian Assistance (ELRHA)'s Humanitarian Innovation Fund (HIF), exploring the dynamics of successful innovation processes in humanitarian action. They examine what good practice in humanitarian innovation looks like, what approaches and tools organisations have used to innovate in the humanitarian system, what the barriers to innovation are for individual organisations, and how they can be overcome.