Evaluation Report on the Reflect ICTs Project

This 32-page document provides an evaluation of the Department for International Development (DFID)-funded project Reflect ICTs, which is being carried out by ActionAid International (AAI) and local partners in Burundi, India, and Uganda. As part of this project, information and communication technology (ICT) is being introduced to existing community-based discussion circles ("Reflect circles"), as part of an effort to enhance the capacity of people to make strategic choices about the media of communication that they have identified as most relevant to their specific needs.

Reflect is an approach to adult learning and social change used by over 350 organisations in more than 60 countries; face-to-face, interpersonal exchange supports the goal of empowering economically poor people and their communities. Based on a rights-based approach to information and communication, the 3-country Reflect ICTs project draws on the Reflect approach in an effort to build community capacity to identify specific information needs, to learn methods to access it, and to gain confidence to claim their rights. In short, the Reflect approach is a strategy for:

• Strengthening voice
• Stimulating participation as a political process
• Creating democratic space
• Drawing on existing knowledge
• Linking reflection and action
• Using participatory tools
• Facilitating power awareness
• Ensuring coherence of practice
• Promoting self-organisation

As documented here, evaluators sought not only to test the validity and adaptability of the Reflect ICT methodology, but to carry out action research to inform and shape the work of Reflect practitioners globally (as well as with those in the ICT for development (ICT4D) field generally). The 3 project sites and the international networking and coordination function were evaluated by separate consultants, who used methods such as reviews of project documents, informal and formal discussions with key stakeholders, focus group discussions, role-plays and other participatory techniques. Their report is organised into several segments: It begins with an Introduction that details evaluation methodology, findings, strengths and weaknesses, and recommendations; this opening portion is followed by specific, more detailed reports on the Burundi, India, and Uganda programmes.

In general terms, the evaluators found that "the project is progressive and heading in the right direction." For instance, in India and Uganda it is supporting the building of good teamwork and stronger partnerships.
Partners and government agencies support the project's continuation, due in part to demonstrable impact such as the generation of "rich sources of research material, with the potential to feed into the wider ICT4D audience." The report also identifies some weaknesses, such as shortcomings in the following areas: action research and progress reporting at country level, implementation, the building up video documenting capacity at community level, administrative communication and controls, and exposure of communities to modern ICTs.

Excerpts from the Introduction:

Findings

Project Model - introducing ICT to Reflect

The project attempts to harness ICTs for the poor using the Reflect methodology. In no case did the introduction of ICTs contradict the principles of Reflect. Instead ICTs helped to galvanize community understanding and participation.

The project had a very cautious and dedicated approach by having two phases. Phase one was dedicated for country level project planning and phase two for the implementation of those plans, both with full participation of Reflect groups. This method helped to select the most appropriate forms of ICT, avoid being top-heavy, techno-savvy and costly and maximize the use of existing local capacity. The model was able to bypass existing barriers to communication to a considerable extent. In Uganda and India, the model managed to break social and cultural barriers kept against the information flow
towards women. In Burundi, the model managed to break ethnic barriers posted by long lasting conflict.

Capacity Building

The project encouraged capacity building of partner organizations in and around the pilot localities...although technology skills could be strengthened through structured training programs, missing in the project.

This development of local partner organizations was visible in information gathering, processing and transferring skills. In both India and Burundi newsletters were produced based on activities at Reflect circles and edited by project and partner staff and trained community members or facilitators. The content was appropriate and the publications were made available regularly far beyond participating Reflect circles.

Information Flow

Resource centres play an important role in the model, the location and function of them depending on the context and needs of participating groups. In India, at village-level these function primarily as information banks, facilitating access to information. Regular information flow is made possible through the network of Reflect circles, which are fed by apex-level resource centres and the project office. This model, designed with community participation and new to conventional Reflect circles, generated additional information flow: it stimulated local information flow on one hand, and information mobility up and down stream on the other.

The project has created awareness and recognition of issues of importance to local people, as they articulate their information needs to implementing organizations. Meetings of [the network of 19 partner organisations of the India pilot: Collective Action for Drought Mitigation in Bolangir, or] CADMB have become a debating space for members, resulting in broadening perspectives of realities and issues faced by local people. In Burundi, issues of importance to local people were the basis of video programs chosen and shown by project team to facilitate debate and provide key information.

Value addition of ICT to Reflect

As observed in all three pilots, ICTs have filled an important gap by improving effectiveness and efficiency in the communication aspect of Reflect. They support the educational value of conventional Reflect graphics and simultaneously increase the volume and timeliness of information. As communities recognize their rights through the
Reflect circle, ICTs provide relatively quick access to specific information to claim the entitlements. The degree of effectiveness was enhanced as multiple ICT components built upon each other. The information that the community read on the posters on peace in Burundi was endorsed by video programmes. Community newsletters built on issues arising in Reflect circles, giving relevant and credible information to trigger community decision making to claim rights in India or return home from refugee camps in Burundi.

All three pilot countries emphasized the enhanced participation in Reflect circles since the beginning of the ICT project. In India, the project generated a new spirit to strengthen and reinvigorate existing Reflect circles. In Burundi, community confidence on peace related information developed and resulted in improved membership of circles. In Uganda, the project enabled the implementing organisation to strengthen existing circles and create new ones in villages and schools.

Meaningful choices of ICTs

On prominent display throughout the project is the careful effort to add the meaning to community choices, without getting distracted by modern ICT4D trends. The ICT components chosen are appropriate to the social context and combine smoothly with traditional communication modes such as meetings, drums and dances.

These choices were influenced by the community during participatory planning in the first phase of the project. In many cases the communities were aware of their shortfalls in information and communication access, but unable to improve them. By facilitating their own analysis of their information and communication needs, the project could make relevant technologies accessible and, more importantly, provide opportunities to improve basic communication skills.

In reality, modern ICTs such as computers and the Internet play a remote role in the project. Their presence is limited to the resource centres located at central, more urban, locations...

With regard to the choice of technologies, the more traditional, less sophisticated approach has created a less costly and more sustainable project. More importantly such selection complements the existing communication patterns of target communities as well as the principles of Reflect.

Impact on people's lives

All three evaluation reports substantiate the impact made by the project from individual to community level: improving their cognitive capacity; influencing attitudinal changes; encouraging more community participation; building stronger pro-poor networks. Case studies reveal numerous accounts of mainstreaming women, amplifying voices of the marginalised, recognizing entitlements and demanding rights. In general empowerment was addressed effectively and broadly by improving the communication.

Interestingly, as observed in India and Uganda, the project could address power structures in a constructive manner. In both countries, women gained better access to information which resulted in improving their participation in decision-making, built family unity, improved health and enhanced peace and harmony. Such circumstances improved the overall unity of the community. In all three countries, participants repeatedly highlighted their enhanced recognition of the power of community.

In Uganda, the project established school knowledge groups, which facilitated improved communication between students and staff. This enabled female students to deal with harassment and focus on education with more communal support.

Knowledge dissemination

...It had been an objective of the project to share the outcomes with ICT4D practitioners and researchers, facilitating the contribution towards the eradication of poverty. The Reflect website was designed to maintain an ongoing dialogue with the web community. An email update was an inspiring effort to maintain an informed analytical dialogue, keeping the spirit of action research yet has failed to establish two-way communication with the target community and hence lost momentum during the second phase...

Almost every known party to the project acknowledge that the project has high potential to contribute important knowledge to the wider ICT4D sector, but lacks the required strategic engagement to make this happen...

Project management

As an action-research project building into ongoing Reflect practice, the role played by project coordinator was very challenging. As this report implies, the project has been steered maintaining the delicate balance between technology and traditions, administration and research...

To request a PDF version of the full evaluation, please contact Hannah Beardon at the email address listed below.