Digital Community Engagement: Year 3 Evaluation Report 2023-2024

"When asked about the impact of participating in uInfluence, results show overwhelmingly positive sentiments toward uInfluence."

This report shares the findings of an evaluation of the third year of the United Nations Children's Fund (UNICEF) Digital Community Engagement (DCE) programme. The DCE supports UNICEF's Country Offices in polio-endemic and outbreak countries by providing a bi-weekly newsletter and communication tools designed to combat misinformation and increase vaccine acceptance. A central aspect of the programme in 2023-2024 was engaging through the uInfluence network of digital volunteers (DVs), which involved executing paid advertising campaigns to educate the public about key polio messages and misinformation. In addition, the programme emphasised strengthening connections within the existing uInfluence network through social media community management, hosting an in-person event, and piloting a country-focused approach to support a specific immunisation campaign. The report presents an evaluation of each of these components, together with key takeaways for each component. 

The following is a summary of each component, the methodologies used to evaluation it, and the findings: 

Biweekly DCE Newsletters
The bi-weekly DCE newsletters are powered by online social listening data designed to support UNICEF field teams globally, providing timely and pertinent insights on polio-related narratives and misinformation. To evaluate their impact, a survey was distributed to all 336 newsletter recipients in 52 countries.

The results showed high levels of satisfaction with the DCE newsletters, with respondents finding them relevant to their needs and current in their updates. The vast majority of respondents agreed that the DCE newsletters are beneficial for strategic planning, and a majority used the information to support their work both internally and externally in the community.

uInfluence Digital Volunteers (DVs)
Throughout the programme, the uInfluence network of DVs was sent a series of seven emails between October 2023 and May 2024. These emails sought to empower DVs with polio education and information to combat misinformation and to encourage them to follow uInfluence on social media and share uInfluence content with their networks. (See Related Summaries below for more information on the uInfluence programme.) To evaluate the impact of the emails, a survey was distributed to all DVs who received emails. It consisted of 24 questions designed to assess key knowledge, attitudes, and intended behaviours related to the polio vaccine. It also explored perceptions about polio misinformation, the ability to identify misinformation, and the capacity to find sources with factual information about polio. The survey also assessed volunteers' engagement with the programme, its impact on their communities, and the effectiveness of the emails in delivering timely and relevant insights.

The findings showed that the programme has motivated volunteers to take tangible actions that amplify polio vaccination messaging within their communities. The majority of respondents expressed strong satisfaction with the uInfluence content and support, and most demonstrated awareness of the impacts of polio misinformation. Volunteers feel confident in recognising and addressing misinformation, and they understand the importance of sharing factual information to combat it.

Targeted Ad Flights
From November 8 2023 to May 27 2024, DCE implemented responsive 2-week Facebook and Instagram ad flights that used both static images and videos to address misinformation trends identified through social listening and highlighted in DCE newsletters. Ads were geared toward parents and caregivers in the countries where misinformation was circulating.

To evaluate the impact of ad flights conducted in Nigeria, Ghana, Uganda, Indonesia, and Kenya, brief surveys of parents/caregivers with children younger than 18 were conducted using Pollfish, an online survey panel. Surveys were fielded immediately after the ad flight.

The report outlines the results for each country as well as a comparison across countries. The findings showed that the ad flights on Facebook and Instagram revealed significant changes in behaviour and knowledge about polio vaccination. The ads positively impacted awareness of the need for vaccination, the importance of multiple doses, and the perception of community risk. Significant improvements in knowledge and attitudes were observed in Indonesia, Kenya, and Ghana, highlighting the impact of pairing real-time media monitoring with rapid content creation.

uInfluence Pakistan In-Person Event
On March 7 2024, The Public Good Projects (PGP) and UNICEF held an in-person uInfluence half-day event in Pakistan. About 15 public health professionals from UNICEF and the Pakistan Ministry of Health joined over 40 digital volunteers to learn about polio eradication efforts and digital health communications. To evaluate the event's impact, a mixed-methods approach was used: a post-event survey of event attendees as well as interviews with event attendees to gain qualitative insights on their perspectives on the uInfluence programme and the event itself.

The findings showed that the uInfluence event in Pakistan improved participants' knowledge and attitudes toward polio, fostering strong interest in continued collaboration and enhancing appreciation for field workers. The event also increased the Country Office social and behaviour change (SBC) teams' confidence in using digital volunteers for targeted health communication, demonstrating the benefits of combining digital and in-person strategies.

uInfluence-Pakistan Activation Campaign
From April 19 to May 3 2024, the uInfluence programme activated the DV network in Pakistan to support a subnational polio immunisation campaign and to amplify polio messaging on social media. Throughout the activation, messages related to the local polio immunisation campaign and health messages were promoted through targeted emails to Pakistan's uInfluence members, organic social media posts, and paid media on Facebook and Instagram. Ads were designed to reach two populations: (i) uInfluence volunteers in Pakistan and (ii) members of the general population who were caregivers of young children or people with similar characteristics to uInfluence volunteers. The following messages were promoted:
 

• Help spread the word today! A polio immunisation campaign is happening now through May 3 across Pakistan.
• Are the children in your life protected from polio? All children under 5 need to be vaccinated to stop polio from spreading.
• Every extra dose of the polio vaccine means extra protection. Answer the door when polio workers come now through May 3.
• Tell your neighbours, family, and friends an anti-polio campaign is happening now through May 3.
• Share this post to help protect children in Pakistan. Tag a parent!

This network activation was evaluated in three ways: digital performance metrics of the paid ads; a survey of 22 uInfluence digital volunteers in Pakistan after the activation to assess their actions during the activation, perceptions of the programme and their involvement in it, and self-efficacy in advocating for polio vaccination; and surveys among the general population in Pakistan before and after the activation to gauge the impact of the activation on public knowledge and attitudes towards polio.

The findings showed that the activation campaign improved knowledge and community support for polio vaccination. The targeted emails, social media posts, and ads led to a better understanding of vaccine safety and an increased willingness to share information. The activation also strengthened digital volunteers' connection to the mission and their active participation in the immunisation campaign.