Participatory Monitoring and Evaluation

Compiled by:
Leonellha Barreto Dillon (seecon international gmbh)

Resumen Ejecutivo

With growing emphasis on participatory approaches towards development, there has been recognition that monitoring and evaluation (M&E) should also be participatory. Conventionally, M&E has involved outside experts coming in to measure performance against pre-set indicators, using standardised procedures and tools. In contrast, participatory monitoring and evaluation (PM&E) involves primary stakeholders as active participants and offers new ways of assessing and learning from change that are more inclusive, and reflects the perspectives and aspirations of those most directly affected (WORLD BANK 2010b). This document presents the different concepts related to PM&E, together with the steps and the tools needed for the development of an effective and sustainable PM&E plan.

Definition of Monitoring

Monitoring is a continuous process of collecting and analysing information to compare how well a project, programme or policy is being implemented against expected results. Monitoring aims at providing managers and major stakeholders with regular feedback and early indications of progress or lack thereof in the achievement of intended results. It generally involves collecting and analysing data on implementation processes, strategies and results, and recommending corrective measures (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007).

 

Definition of Evaluation

Evaluation is the systematic and objective assessment of an ongoing or completed project, programme or policy, its design, implementation and results. Evaluation determines the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling incorporation of lessons learned into the decision making process of both recipients and donors (INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES 2007).

 

Definition of Participatory Monitoring & Evaluation

Participatory monitoring & evaluation (PM&E) is a process through which stakeholders at various levels engage in monitoring or evaluating a particular project, program or policy, share control over the content, the process and the results of the monitoring and evaluation (M&E) activity and engage in taking or identifying corrective actions. PM&E focuses on the active engagement of primary stakeholders (WORLD BANK 2010a).


Participatory Monitoring and Evaluation is one of many approaches to ensure that the implementation of the different projects within the action plan ― or smaller individual projects ― leads to the expected outcomes. As with all other monitoring and evaluation elements, the process for PM&E has to be prepared prior to project implementation (PHILIP et al. 2008).


The stakeholder groups typically involved in a participatory M&E activity include: the end users of project goods and services, including both men and women at the community level; intermediary organisations, including NGOs; private sector businesses involved in the project; and government staff at all levels (RIETBERGEN-McCRACKEN et al. 1998).

 

Principles of Participatory Monitoring & Evaluation

 WSP (2007)

This problem might be recognised and solved by participatory monitoring and evaluation. Source: WSP (2007)

Conventionally, monitoring and evaluation has involved outside experts coming in to measure performance against pre-set indicators, using standardised procedures and tools. PM&E differs from more conventional approaches in that it seeks to engage key project stakeholders more actively in reflecting and assessing the progress of their project and in particular the achievement of results (THE WORLD BANK 2010a). Core principles of PM&E are (RIETBERGEN-McCRACKEN et al. 1998):

 

  • Local people are active participants — not just sources of information.
  • Stakeholders evaluate, outsiders facilitate.
  • Focus on building stakeholder capacity for analysis and problem-solving.
  • Process builds commitment to implementing any recommended corrective actions.

Steps in the Development and Implementation of a PM&E Process

(Adapted from AUBEL 2004)

Step 1: Planning the PM&E Process and Determining Objectives and Indicators

At this initial stage, the stakeholder groups to be involved in the planning of the PM&E process must first be identified. Stakeholders must define the objectives of the PM&E, including what will be monitored, how and by whom. The planning stage requires a lengthy process of negotiation, contestation and collaborative decision-making among various stakeholders. Identifying objectives and monitoring indicators can be the most difficult part of planning a PM&E process. In some cases, a common set of indicators is developed, while in other instances different stakeholder groups develop their own sets of indicators.
 

Step 2: Gathering Data

Data collection can include the use of both quantitative and qualitative methods and tools. Quantitative methods can include: community surveys; interviews; and observations. Qualitative methods can include various participatory learning methods using visual, interviewing and group tools and exercises.


Step 3: Analysing Data

While data analysis is often thought of as a rather mechanical and expert-driven task, PM&E should be an opportunity to actively involve various categories of program stakeholders in the critical analysis of successes and constraints and the formulation of conclusions and lessons learned.


Step 4: Sharing the Information and Defining Actions to Be Taken

However participatory the M&E process in Steps 1-3 is, not all stakeholders can be involved in M&E data collection and analysis. In this step, the results of M&E activities are shared with other stakeholders, and there is discussion of appropriate actions to be taken based on the findings.

Techniques and Tools for PM&E (Adapted from RIETBERGEN-McCRACKEN et al. 1998)

A participatory approach to monitoring and evaluation will usually make use of a number of techniques and tools, selected and combined to suit the objectives of the M&E work and the resources available. Many of the techniques associated with Participatory Rural Appraisal (PRA), Beneficiary Assessment (BA), and SARAR have been used in the context of monitoring or evaluation. Some examples of these methodologies’ trademark techniques and applications to M&E are highlighted below:

 

Aplicabilidad

Although monitoring and evaluation only becomes relevant once a project is up and running (for example at regular intervals when results become available) it nevertheless has to be considered prior to project implementation that is already in the planning phase (PHILIP et al. 2008).

Ventajas

  • Involving beneficiaries in evaluation increases its reliability and provides the opportunity to receive useful feedback and ideas for corrective actions
  • PM&E allows for flexibility ― Activities should be stopped or adapted when evaluation makes it clear that they are not contributing to the intended improvements
  • Strengthens ownership regarding successful outcomes of planned initiatives
  • Widens the knowledge base necessary for assessing and ― if required ― correcting the course of action
  • Increases the motivation of stakeholders to contribute ideas to corrective actions
  • Creates trust in Local Government policy and action (provided that the stakeholders’ input is genuinely taken into account)
  • Contributes to the learning of all involved

Desventajas

  • Needs skilled facilitator to ensure everyone understands the process and is equally involved
  • Can be dominated by strong voices in the community (for example, men dominating women in discussions, political, cultural or religious leaders dominating discussions and decision making)
  • Can be time consuming - needs genuine commitment
  • Needs the support of donors as does not always use traditional indicators
  • Those responsible for implementation of certain projects may not want the administration or public to learn about failures or mistakes due to a fear of disciplinary action. Evaluation should be conducted in a fair and constructive way (PHILIP et al. 2008)

Referencias Library

AUBEL, J. (2004): Strategic Report 9, Participatory Monitoring & Evaluation for Hygiene Improvement, Beyond the Toolbox: What else is required for effective PM&E? A literature Review. Washington, D.C.: Office of Health, Infectious Diseases and Nutrition, Bureau for Global Health U.S. Agency for International Development. URL [Accessed: 27.01.2011]. PDF

INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES (Editor) (2007): Monitoring and Evaluation in a nutshell. INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES . PDF

PHILIP, R.; ANTON, B.; BONJEAN, M.; BROMLEY, J.; COX, D.; SMITS, S.; SULLIVAN, C. A.; NIEKERK, K. van; CHONGUICA, E.; MONGGAE, F.; NYAGWAMBO, L.; PULE, R.; BERRAONDO LOEPEZ, M. (2008): Local Government and Integrated Water Resources Management (IWRM) Part III: Engaging in IWRM – Practical Steps and Tools for Local Governments. Freiburg: ICLEI European Secretariat GmbH. URL [Accessed: 17.04.2012]. PDF

RIETBERGEN-McCRACKEN, J.; NARAYAN, D. ; WORLD BANK (Editor) (1998): Participation and Social Assessment: Tools and Techniques. Washington: World Bank. URL [Accessed: 10.05.2010]. PDF

THE WORLD BANK (Editor) (2010): Participatory Monitoring and Evaluation, in Topics: Participation and Civic Engagement (a). Washington D.C.: The World Bank. URL [Accessed: 27.05.2010].

THE WORLD BANK (Editor) (2010): Participatory Monitoring and Evaluation, in Topics: Community Driven Development (b). Washington D.C.: The World Bank. URL [Accessed: 27.05.2010].

WSP (Editor) (2007): Calendar 2007 June. Washington, DC: Water and Sanitation Program (WSP). URL [Accessed: 03.04.2012].

Lectura Complementaria Library

Reference icon

APWELL Project (Editor) (2003): Judicious management of groundwater through participatory hydrological monitoring. A manual. Andhra Pradesh, India.: Andhra Pradesh Groundwater Bore well Irrigation Schemes Project (APWELL Project). URL [Accessed: 15.01.2013]. PDF

This report developed under the APWELL project deals with participatory hydrological monitoring in an effort to sensitize the individual groundwater users on judicious use of groundwater. Participatory hydrological monitoring improves the users’ understanding of local groundwater resource characteristics and helps local communities to form a community opinion to support appropriate measures for managing the available resources equitably.


Reference icon

GERMAN, D. ; GOHL, E. (Editor) (1996): Participatory Impact Monitoring Booklet 4: The concept of participatory impact monitoring. Eschborn: GATE/GTZ. URL [Accessed: 22.04.2012]. PDF

This short booklet explains the concept of participatory impact monitoring in depth, but in a simple and richly illustrated way.


Reference icon

GTZ (Editor) (n.y.): Participatory Impact Monitoring: Selected Reading Examples. Eschborn: GATE/GTZ. PDF

This short booklet presents further reading examples (i.e., several papers) on participatory impact monitoring.


Reference icon

INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES (Editor) (2007): Monitoring and Evaluation in a nutshell. INTERNATIONAL FEDERATION OF RED CROSS AND RED CRESCENT SOCIETIES . PDF

As part of the process to systematise and enhance the quality of monitoring and evaluation processes, this simple monitoring and evaluation guide has been developed. This guide includes practical guidance on how to do monitoring and evaluation: including developing simple monitoring and evaluation tools giving practical examples, a set of formats to facilitate the evaluation process and basic monitoring and evaluation terminology to ensure coherence and consistency.


Reference icon

RIETBERGEN-McCRACKEN, J.; NARAYAN, D. ; WORLD BANK (Editor) (1998): Participation and Social Assessment: Tools and Techniques. Washington: World Bank. URL [Accessed: 10.05.2010]. PDF

This resource kit aims to share information and experiences on participatory methods in the context of development cooperation. The primary focus concentrates on providing practical guidance and case examples.


Reference icon

TOUBKISS, J. (2010): How to Manage Public Toilets and Showers. (= Six Methodological Guides for a Water and Sanitation Services' Development Strategy, 5). Cotonou and Paris: Partenariat pour le Développement Municipal (PDM) and Programme Solidarité Eau (pS-Eau). URL [Accessed: 19.10.2011]. PDF

The purpose of this decision-making aid is to provide practical advice and recommendations for managing toilet blocks situated in public places. It is primarily aimed at local decision-makers in developing countries and at their partners (project planners and managers).

See document in FRENCH


Reference icon

THOMAS, A.; BEVAN, J. (2013): Developing and Monitoring Protocol for the Elimination of Open Defecation. (= ODF Protocol). Nairobi: UNICEF. URL [Accessed: 06.06.2013]. PDF

This paper reviews process and protocol for defining, reporting, declaring, certifying ODF (Open Defecation Free) and sustaining ODF, highlighting where the process varies between countries and potential determinants of sustainability within the process itself.


Reference icon

ZEWO (Editor) (n.y.): Outcome and Impact Assessment in International Development. Zewo Guidelines for Projects and Programmes. Zurich: Schweizerische Zertifizierungsstelle fuer gemeinnuetzige, Spenden sammelnde Organisationen (ZEWO). URL [Accessed: 04.10.2013]. PDF

These guidelines are designed to help project managers to assess the outcomes of their projects and programmes. They demonstrate how development agencies can implement an appropriate outcome and impact assessment system.


Reference icon

COWLING, R.; NORMAN, G. (2013): Achieving Sustainability: Changing the Ways in Which we Define Success. (= Perspective, 1). London: Water and Sanitation for the Urban Poor (WSUP). URL [Accessed: 25.11.2013]. PDF

Implementing agencies like WSUP, Water For People and IRC are of course accountable to their funders, including major bilaterals and foundations. And naturally, these funders must track the effectiveness of their spending. But short budget cycles and the need to demonstrate “value for money” can often encourage over-simplistic measurement of success in terms of short-term outputs, rather than genuinely sustainable services. This note proposes some ways forward.


Estudios de Caso Library

Reference icon

SCHWARZ, B.; GERMAN, D. ; GOHL, E. (Editor) (1996): Participatory Impact Monitoring Booklet 3: Application Examples. Eschborn: GATE/GTZ. URL [Accessed: 22.04.2012]. PDF

This booklet presents several examples from Bolivia, the Philippines and Argentina of group- or NGO-based impact monitoring.


Materiales de Formación Library

Reference icon

GERMAN, D. ; GOHL, E. (Editor) (1996): Participatory Impact Monitoring Booklet 1: Group-based impact monitoring. Eschborn: GATE/GTZ. URL [Accessed: 17.04.2012]. PDF

This short booklet is written for leaders or members of self-help groups and describes how group-based impact monitoring works in a simple and easily understandable style.


Reference icon

GERMAN, D. ; GOHL, E. (Editor) (1996): Participatory Impact Monitoring Booklet 2: NGO-based impact monitoring. Eschborn: GATE/GTZ. URL [Accessed: 17.04.2012]. PDF

This booklet on NGO-based impact monitoring is addressed to staff members of development organisations, i.e. national organizations such as NGOs, federations or government organizations which promote self-help groups. It explains how NGO-based impact monitoring works in a simple, illustrated and easy-to-understand manner.


Vínculos Importantes

http://www.irc.nl/ [Accessed: 16.08.2010]

Web page of the International Water and Sanitation Centre (IRC) with several publications on water supply systems operation and maintenance.

http://portals.wi.wur.nl/ [Accessed: 27.05.2010]

This website is essential in providing introductions to project and program management, as well as providing a quick selection of key resources in relation to specific areas of the Logical Framework Approach.

http://go.worldbank.org/ [Accessed: 27.05.2010]

This web page contains several links and documents related to PM&E.

http://www.scn.org/ [Accessed: 07.08.2010]

This website contains a collection of training material intended to assist practitioners in helping low-income communities to overcome poverty, emphasizing methods and principles, not theory. One of the modules deals with the community project resources, including project proposals.

http://www.fao.org/ [Accessed: 04.03.2011]

Training module on participatory community monitoring and evaluation. In: A Handbook for Trainers on Participatory Local Development: the Panchayati Raj model in India. Although designed for the training needs of all categories of local functionaries associated with the decentralisation process in India, the handbook provides guidance on core issues in institutional capacity building for local development planning, which are, to a large degree, similar in other developing countries within the region.

http://www.washdoc.info/ [Accessed: 18.12.2013]

IRC Sanitation Pack, SanPack for short, contains an overview of available methods, techniques and tools in a low-cost, non-sewered sanitation service model, including participatory approaches. It is a reference guide containing links to relevant documents explaining the different stages in the sanitation cycle.

http://www.washplus.org/ [Accessed: 19.12.2013]

The WASH Sustainability Index Tool, developed for the USAID-Rotary International H2O Collaboration, is a tool to assess sustainability of WASH programs. The tool considers the sustainability of institutional, management, financial, technical and environmental factors.