Monitoring and evaluation (safe water business)

Monitoring is the routine collection of high-priority data and information on activities and operations (e.g. from customers, markets etc.). Monitoring helps the entrepreneur to track how effective and efficient the business is using its resources towards operations and whether the business is on track to meet its objectives. For safe water businesses, monitoring can for instance entail record keeping such as sales or operational reporting, profit and loss reporting, and tracking of commercial or financial records. For measuring impact also other data are needed such as figures on (continuous) product use, health data, and medical expenses over time etc. (PATH, 2011). Collecting these data can be costly and time consuming. However, this investment can also pay off, the data collection can for instance help to convince investors (KUSTERS ET AL., 2011) and generate revenue from carbon credits (for more information on carbon credits see factsheet on carbon credits).

Evaluation is used to compare actual project results against strategic plans. Unlike monitoring, evaluation is not done on a routine basis but done at specific times (i.e. monthly, bi-yearly, yearly etc.). Evaluation refers to a range of activities that are undertaken to determine if the project activities had the desired impact. There are several types of evaluations that may be appropriate for social businesses: formative evaluations, outcome evaluations, impact evaluations (PATH, 2011).

Monitoring and evaluation (M&E) are important for a safe water businesses in several ways:

• An M&E system helps the entrepreneur to understand the market and customers’ needs, to inform the management and adapt business strategies accordingly where needed.
• A well-established M&E system can also help the entrepreneur to determine whether he/she is accomplishing the desired social/health/market impact. This includes monitoring effective and/or continuous use of household water treatment products (HWTS), for instance to apply for carbon finance, to attract social impact investors and/or to meet (inter)national drinking water quality standards.

M&E is useful for any kind of safe water business throughout the business life cycle from blueprint to scale. M&E helps to steer and adapt sales and marketing strategies, improve the business model and attract investors. The same relevance has M&E also for NGOs implementing a market based safe water approach and governments that are looking for specific measurement methods for safe water implementation.

For investors M&E is also relevant in terms of understanding how social enterprises do measure their impact.

The businesses in the Safe Water Program Phase II are at different monitoring stages. Some have only collected basic information about their customers, water quality in the target area and the number of products they sell. These data were used to develop the sales strategy. To improve their business model and fine-tune their sales strategy, some businesses collect information about the performance of their retailers, the actual use of the product and the water quality at the point of use. Finally, there are some businesses such as Hydrologic who aim to estimate the impact of the use of the product on the customer’s health and economic situation.

This is required for instance to acquire carbon finance (for more information see factsheet on carbon credits) and attract impact investors (see section 6). The figures below show these different stages of the M&E of safe water businesses; from the first stage of informing sales and marketing strategies to improving the business model as such and to assess impact and attract investors through being able to outline its company’s social impact.

Tip: the Path M&E Toolkit includes a hands-on approach and tools to help safe water entrepreneurs to develop an M&E framework including monitoring plans, indicator definition and templates to demonstrate social impact.

1. M&E methodologies and costs involved

The figure above shows that at every stage additional data is being collected for M&E compared to the previous stage. This comes with extra costs for data collection such as software, staff time, equipment (smart phones, tablets), water quality tests, transportation, etc. The businesses in the Safe Water Program Phase II are using various methods to collect data, ranging from paper-based registries to mobile-based data collection (see the figures below). A sales tracking system that, preferably, integrates real-time data in an online platform is aspired by most businesses in the Safe Water Program Phase 2 to steer management and directly address problems. Hydrologic in Cambodia is the only business that currently has such a system. With this system (i.e. Taroworks) sales agents process sales data via tablets and the management can access these data real-time on an online platform. Spring Health and TARA are piloting similar platforms (see subsequent case studies for more information).

The costs of the various methods are difficult to compare, as the businesses are selling different products and services, in very different contexts. MinErgy is for instance spending about USD 290-340 per year on M&E including staff time. Whereas Hydrologic spends yearly about USD 25,000 a year plus 0.5 FTE (full-time equivalent) staff time on its M&E. The household surveys that Hydrologic does with this budget to collect data on the impact of their product, allows them to generate about USD 540,000 on carbon finance each year (excluding 15% costs for carbon consultant services) (see factsheet on carbon credits for more information).

2. Indicators suggested by WHO

Tip: For businesses that sell HWTS products, the WHO M&E Toolkit for HWTS (WHO, 2012) provides guidance to measure a number of these indicators such as product use and water quality. It includes 20 indicators specifically designed for HWTS M&E. The indicators are grouped as per the following 5 categories:

1. reported and observed use;
2. correct, consistent use and storage;
3. knowledge and behaviour;
4. other environmental health interventions;
5. water quality.

The WHO Toolkit is not specifically tailored to businesses but (some of) the indicators (shown in the figure above), resources and survey samples can be useful for safe water companies too. Impact indicators are not included, but monitoring the indicators gives an indication of the potential health benefits of the product. At a minimum, WHO suggests that all M&E efforts aim to know if households use HWTS selected indicators 1-4, which concern self-reported use and observations (see also WFP, 2002). The decision tree shown below helps businesses to identify which indicators to measure.

3. Which indicators do impact investors want to see?

Impact investments are investments made into companies, organisations, and funds with the intention to generate measurable social and environmental impact alongside a financial return. These impact investors can be banks, pension funds, insurance companies, NGOs, family foundations, government investors and development finance institutions, etc. Well-known impact investors include for instance funds and banks such as Grassroots Business Fund, Acumen and Root Capital.

It can be interesting for safe water entrepreneurs to attract impact investors to generate extra revenue to grow their business. The entrepreneurs do need to show that the business is generating measurable social and/or environmental impact. Collecting data on a regular basis is therefore essential. It is difficult to say which data impact investors want to see in general, because this depends on the investor’s profile and intentions. In the IRIS catalogue you can find which performance indicators leading impact investors use to measure social, environmental, and financial success. This catalogue, hosted by Global Impact Investors Network (GIIN), can also be filtered for specific topics such as water. It also includes several user profiles and a practical guideline.

Tip: on the Global Impact Investors Network website you can find more about impact investment, leading impact investors and performance indicators that they use to measure social, environmental, and financial success.

The case study of TARA provides further hands-on insights on how a safe water enterprise can leverage impact through a thorough M&E scheme.