A Pilot Study of a Novel, Incentivised mHealth Technology to Monitor the Vaccine Supply Chain in Rural Zambia

"More research is needed to identify methods to encourage healthcare workers to engage in timely stock reporting practices."

According to the World Health Organization (WHO), almost half of all vaccines are wasted; improved logistics and stock management systems may generate improved efficiency of the vaccine supply chain. A group of researchers piloted an mHealth solution using short message service (SMS) technology that allowed workers in rural health centres (RHCs) in Kazungula District, Zambia to report vaccine stock levels directly to an online platform. In addition to evaluating the potential of the "Tessellate" technology, the study sought to establish if user engagement could be improved with increased airtime incentives or with increased frequency of SMS prompts.

As the researchers explain, in Zambia, prior to 2015, vaccine stock levels were recorded manually using paper registries. There was no timely coordination of these individual registries at national level. In response to this situation, in 2015, the Zambian Ministry of Health launched a pilot using mHealth technology to enable real-time reporting of vaccine stocks. The project faced 2 main challenges: (i) it required Java- or smart-phones that could connect to mobile data networks, yet rural smartphone penetration is low and access to mobile data highly variable; and (ii) the reporting rate was low, perhaps due to lack of sufficient training and understanding and/or insufficient use of prompts or "nudges" to remind users to engage. The researchers in the present study developed Tessellate in response to these challenges.

Prior to commencement of the trial, the study team visited each of 21 RHC sites and conducted a 30-minute training session with the community health workers (CHWs). They were given written materials and a poster, as well as a study telephone number they could use at any time to seek further information. For the duration of the study, between July and November 2017, identifying labels were placed on every box of pentavalent diphtheria, pertussis, and tetanus (DTP) vaccines at the district vaccine store . Each label displayed a unique 5-digit code. CHWs were asked to use their own mobile phones to send this 5-digit code as an SMS to the project mobile phone number when they collected the vaccines from the vaccine store. An automated reply was sent back, asking to which RHC the vaccines were being taken. Upon reply to this, an automated message was sent with a code for airtime. All users who had communicated a "pick-up" message were sent an automated message once a week asking how many doses remained in the box they had collected. If a user replied to this reminder message, he or she would receive a further airtime code. All messages received by the project mobile phone were automatically time-stamped and aggregated by a custom-written web platform that displayed recorded levels of vaccine stocks across all 21 RHCs in the district.

The primary outcome measured was the percentage-of-doses-tracked, calculated over the study period. Each vaccine box was randomly allocated to offer either a standard or double airtime incentive and either weekly or daily reminders, in a 2 x 2 design; ANOVA was used to calculate if any of these factors affected time to reply.

Among the users who chose to participate (13 people from 10 RHCs), the response rate was "remarkably high, with most CHWs replying to stock update reminder messages in under a day. Within this active subgroup, near-real-time stock data were captured for 93.8% of doses, which is significantly above usual levels of reporting. It was unsurprising that there was a tendency towards more timely replies in the group who received daily reminder messages; this did not reach statistical significance in part because those in the weekly reminder group also replied promptly to their reminder messages. This study did not demonstrate any significant difference in engagement between standard and double remuneration groups....[T]he findings from the present study would support the notion that non-monetary rewards do not significantly improve motivation in Zambian CHWs."

Due to the low engagement rate in the study, a post-hoc structured questionnaire was developed to identify the barriers to uptake from the 11 RHCs that did not send any update data to the project. The majority (64%) of respondents cited inadequate understanding of the project, with a significant minority (27%) identifying a poor mobile phone signal as their reason for non-engagement. Even among those who participated, one challenge for implementation was limited technological understanding of an automated SMS service, although the research team felt that clear instructions were provided. They say that this limited understanding could have been a factor in the 52% of RHCs that did not send an initial "pick-up" message when they collected their vaccine stock.

That said, the study "demonstrates that an automated mHealth system can record vaccine inventory in rural areas at low cost without relying on paper records and using CHWs' own mobile telephones; however, more research is required."