Impacts of Supportive Feedback and Nonmonetary Incentives on Child Immunisation in Ethiopia

"...recommend that this study be carried out in other settings with low immunisation rates that have better cell phone coverage and are not experiencing civil strife."

In an effort to address Ethiopia's low rates of full childhood immunisation, Zerihun Associates, in partnership with Marie Stopes International Ethiopia (MSI-E) and ideas42, identified bottlenecks to immunisation in the Oromia region in Ethiopia and then created and implemented an intervention using behavioural economics (BE) to address these bottlenecks. Designed through an iterative process, the intervention sought to explore two questions: (i) how to engage communities by using behaviourally informed feedback to mobilise health extension workers (HEWs) around improving immunisation rates; and (ii) how to create a positive reinforcement loop for those improvements using non-monetary rewards. This report discusses the implementation of the intervention, including changes made during implementation, and reports on the impact the intervention had on immunisation rates.

In providing context for the BE intervention, the researchers note that, while there have been many attempts to involve communities in increasing immunisation rates, such efforts have mostly focused either on increasing awareness or interest in immunisation, or they have employed direct (often financial) incentives. Underlying the use of these techniques is the assumption that people's actions directly reflect their intentions. However, the field of behavioural science shows us that there are many situations where people's actions may not line up with their desires. Namely, it is possible for a mother to be convinced of the importance of immunisation without her necessarily taking the required steps to immunise her children, so that further education or awareness building may have limited effects.

BE research shows that feedback that focuses on making injunctive norms (what most people typically approve or disapprove) more salient and aligning injunctive norms with descriptive norms (what most people typically do) can lead to an increase in the desired actions. For example, social signalling through a simple, non-monetary intervention was found to increase completion of vaccination schedules among children in Sierra Leone.

Following a site visit, ideas42 translated diagnosis work to designs in consultation with Zerihun Associates and MSI-E. The key behavioural barriers identified during the diagnosis work, which the designs aim to address, were: lack of feedback to HEWs, HEWs being overloaded with multiple tasks/messages and not having a specific moment to focus on immunisation, lack of salience of the benefits of immunisation for HEWs, mothers falling victim to time inconsistency and planning fallacy on taking their child to immunisation days, and a lack of social proof that infants are being immunised. The preliminary design addressed these elements by including an improved tracking system that provides the HEWs visual feedback, a time-specific message that encourages HEWs to accurately plan for immunisation clinics, and social proof to both provide positive feedback to the HEW on her accomplishments and to demonstrate to mothers in the village that other women are immunising their babies.

Specifically, each square on the "Protected Children" tracking poster, hung on the wall of health posts (HPs), represented a child and their specific immunisation schedule. The square was divided into four smaller squares, with each segment representing a follow-up immunisation visit of 6 weeks, 10 weeks, 14 weeks, and 9 months. Each time the caregiver and her infant went to the HP for an immunisation, the HEW put a stamp in the appropriate segment of their child's square and on the immunisation card, if they wished. Caregivers chose from five stamps with aspirational or cultural values (e.g., a woman holding a child). When a child was fully immunised, a completion symbol (e.g., a lion) was placed over the fourth square as visual recognition. The mother also took home a completion symbol.

Initially, the intervention was to include outreach text messages for HEWs along with the tracking poster. As part of what was called the Ethiopia Child Immunisation and Information Network (ECIIN), a text message was to be sent to HEWs 3 days prior to the upcoming immunisation clinic that encouraged them to take a moment to make a plan to perform outreach specifically for immunisation. This text message was designed to focus HEWs' energy when it is most applicable and effective to address immunisation defaulters. However, due to political unrest, causing issues of internet service inaccessibility that resulted from the state of emergency declared by the Ethiopian government, the short messaging service (SMS) intervention was not implemented.

The theory of change postulates that public feedback to frontline health workers will lead to additional outreach to families with children in need of immunisation. In turn, this outreach will lead to parents taking their children for immunisation to the HP or to other centres. The improved community immunisation rates will lead to recognition for the HEW and start a positive reinforcement cycle, leading to improvements in HEWs' self-efficacy and increased engagement with the community. The increase in immunised children will lead to lower overall morbidity for children.

The study was a randomised controlled trial (RCT) that used random assignment at HP, which has a catchment area roughly equivalent to a village or two. Forty-five of 90 HPs from Arsi and East Shewa zones in the Oromia region of Ethiopia were randomly assigned to the treatment group, with the remaining 45 HPs assigned to the control group. 2,760 households, equally split between treatment and control communities based on their HP, were surveyed to collect baseline (April 2016) and follow-up data (May and June 2018), with 30 households with children aged less than 3 years randomly sampled within each HP’s catchment area. Implementation began in September 2016 and continued through February 2018; final data analysis occurred in January-March 2019.

The study found significant increases in diphtheria, pertussis, and tetanus (DPT) and pneumococcal conjugate vaccine (PCV) rates in both intervention and control (from 59% to 87%) groups. Similar results were also observed for immunisation dropout rates, which decreased from almost 20% at baseline to 5.5% at endline. However, the difference between intervention and control at endline was not significant.

On analysing the relationship between HEW household visits and the intervention, the authors reported a marginally significant increase of 7.7% in household visits by HEWs in the treatment areas compared to control. These results indicate that treatment may have induced changes in HEWs' behaviour and effort but had no effect on immunisation rates.

Exploring potential reasons for the null effects, the authors analysed the effects of the civil strife in Ethiopia on the immunisation outcomes, noting that the intervention could not be fully implemented or evaluated as planned. (Even though the study reported larger treatment effects in places where civil strife was rare, the results were not statistically significant.) Furthermore, although qualitative work indicated that HEWs were trusted, instability and lack of trust in the government may have hindered the intervention's effectiveness. A further limitation is that this intervention took place during a period of rapidly increasing overall immunisation rates in the study areas, making it more difficult for the evaluation to detect any additional effect the intervention may have had.

Among the lessons learned: BE research shows that providing feedback that enables individuals or communities to benchmark their own performance against that of an appropriate peer group can be an effective spur to action, especially when it primes and leverages a sense of pride in one's group identity and is combined with recognition for improvement. Such a system can lead to a virtuous cycle, where individuals and communities strive to achieve higher performance on the indicators that are rewarded, albeit non-monetarily. Providing feedback to individuals has been shown to improve pro-social and community outcomes. Therefore, for future research into feedback using salient injunctive norms and non-monetary incentives in improving vaccination rates, the authors recommend incorporating social benchmarking into the feedback.

In conclusion: "This is a low cost intervention that has the potential to use behaviourally informed feedback and reminder mechanisms to increase rates of vaccination follow-through and to motivate HEWs by tapping into their intrinsic motivations....[A] similar study with the ability to more rigorously track vaccination rates and to ensure treatment fidelity may show a positive effect on increasing immunisation rates and minimizing incomplete vaccination."