Collecting & Analysing Data

Good data provides you with evidence and justification for your decisions and enables you to explain your reasoning with confidence.

Identify the intents and purposes of your data archive

Generally speaking, studies are performed at the outset of sanitation projects to thoroughly analyse the project context, collect technical and non-technical data, and better understand the needs and interests of stakeholders in the target community. To this end, a variety of comprehensive methodologies and data collection approaches may be utilized (e.g. Baseline Data Collection, Focus Groups, Locality Mapping, Semi-Structured Interviews, Transect Walks, etc.). Whether your own data collection team applies them directly or you use them as guideposts for your consultants’ Terms of References (ToRs), these processes of information gathering can quickly result in very comprehensive studies that cover much more information than you actually need for project planning and decision-making processes. Conversely, it may also happen that much-needed and important information is not covered sufficiently. Therefore, you should reflect on the intents and purposes of your data collection activities before you launch the relevant studies. Since it may not be altogether clear in advance how project planning and implementation processes will unfold, it can be helpful to think in terms of scenarios. In ISSRAR, for example, technical and social data as well as official data and documents provided by governmental departments were collected and analysed during the site selection process to produce three different project implementation scenarios.

Verify and optimise collected data

There are different methods of collecting quantitative and/or qualitative data. In underserved communities, it can be challenging to procure up-to-date, aggregated information about the targeted area and population. To generate accurate, precise, and consistent data you need to design questionnaires that are simple, clear, well-structured, and easy to understand. Some of the questions you would like to ask can be sensitive, controversial, or even unacceptable to the community. Therefore, we advise that you to test your questionnaires on a representative sample of respondents before the actual roll-out. This will tell you which questions are easily understood and which ones should be re-phrased or omitted altogether.

It also matters who asks the questions. In the ISSRAR project, a group of local youths was trained to conduct the survey. This helped to easily convey information in a context-friendly language and to build trust with the local community, who consequently were more open and comfortable in answering questions.

Keep reports short and compile an overview of the collected information

In complex sanitation projects, a large amount of technical, social, and official data is collected, analysed, processed, and compiled into reports. This can easily lead to a situation in which the implementing organization loses track of what information has been collected and where it can be found. In ISSRAR, this was further exacerbated by external consultants who handed in overlong reports and staff tasked with data collection and reporting who later left the ISSRAR team without properly instructing new incoming staff members.

We therefore strongly recommend that you:

• keep a central log file that documents which information has been collected for which purposes and where it can be found,
• highlight the most important data in the executive summary of every report, and
• specify the maximum number of pages for consultant’s reports to avoid overlong reports.

Document all decision-making processes as well as the information / data used to support them

Important decisions (like those resulting from discussions with key stakeholders) are made at various stages of long and complex sanitation projects. As key staff at relevant institutions such as ministries, partner organisations, community-based organisations (CBOs), companies, etc. might be replaced over the course of the project, all decisions must be properly documented to support your rationale and insulate you from changes of (public) opinion.

One of the most important decisions regarding ISSRAR was choosing an Operator Model. Therefore, we signed a formal Memorandum of Understanding (MoU) that documented the decision we mutually arrived at with the Secretary General (SG) of the Jordan Ministry of Water and Irrigation (MoWI). This MoU then could be used to brief the SG’s successor.

Be careful in dealing with politically sensitive topics

In sanitation projects, it is common for stakeholders to hold differing views and opinions. Therefore, one of the challenges you are likely to face is contradictory information provided by different stakeholders on politically sensitive topics. It is in your best interest to double-check and verify such information before acting on it.

In ISSRAR, we “triangulated” information by comparing local and national information sources as well as contacting organizations who had already worked in the respective communities and getting their take on the issues in question.

One of the politically sensitive issues during was land allocation: while the government officially owned the land that was identified as most suitable for constructing the WWTP, members of the local community laid claims to the land as well. To complicate matters even further, another stakeholder, the Ministry of Municipalities Affairs (MoMA), also had plans for using this land under a governmental masterplan. In this complex setting, we made sure to adhere to the legal processes, to follow up with the concerned governmental departments through formal and continuous communication, and to have the land allocated in a transparent process in order to avoid potential conflicts with the local community.

Manage and optimise the use of interlinked studies

Given the different and interconnected types of data needed to implement a sanitation project, financial, Institutional, environmental, technological, and social studies may feed in to one another to achieve sustainable solutions.

As the ISSRA project included both a technical and a social study, it was vital to connect them at the right time and make sure that all important information had been covered and no details were missed. This required focused project planning, continuous coordination and updates between the various sub-project teams, as well as adherence to fixed deadlines.

For example, the socio-economic study that the ISSRAR team conducted contained quantitative and qualitative data that was also relevant for the technical study. Therefore, the two studies interlinked and supported each other and eventually combined to yield a bigger picture of the targeted project area. Thus, it is highly recommended to periodically look at all data that is collected throughout the project and consider how it may benefit parallel or future studies. This way, you can save time and effort and avoid duplicate work. Relevant data must always be shared with the whole team and documented in an accessible way.

Data verification and approval

Data review, verification, and validation are needed to accept, reject or qualify data in an objective and consistent manner. In sanitation projects, many different types of data will be collected, and all will need to be analysed and verified to achieve the proposed objective.

In ISSRAR, the following points proved to be important:

1. Approval of data collection: to be able to gather and use data, we needed the consent/approval of both the local community and the Head of District (HoD) representing the government.

2. Data verification: all collected data was verified with other related sources. For example, information on water consumption collected from local households was cross-checked with data from the water company. This enabled us to confirm the accuracy of the information.

3. Data Validation: After verifying the data, it is important to share your results with the stakeholders who have the ability and authority to approve them. For example, after the ISSRAR team concluded the WWTP design parameter report and sludge testing, the analyses and their results were submitted to the Water Authority of Jordan (WAJ) for approval.

At times, however, the ISSRAR team also failed to share and properly present their results: while the location of the WWTP was selected in a transparent manner, the criteria and processes utilized in the selection process were only shared with the relevant governmental organisations, neglecting various interest groups in the local community. As a result, the local populace maintained a high level of concern about the selected site. In an effort to make up for this omission, relevant information was later also shared with engineers and key stakeholders and decision makers from the local community.

• Did you reflect on how and for what purpose the results will be used before you launch a study or data collection exercise?
• Did you design simple, clear, well-structured and easily understandable questionnaires to generate accurate, precise and consistent data?
• Are you keeping a central log file to document what kind of information is collected and in which reports it can be found? Are the most important data points highlighted in the executive summaries of the reports? Did you specify the maximum number of pages for consultant’s reports to keep reports concise and to the point?
• Are you properly documenting and recording all decisions to support your course and insulate you from opinion changes by key stakeholders?
• Have you put measures in place to deal with (confirm and verify) contradicting information/data?
• Does your project planning support continuous coordination and updates between the various sub-project teams, as well as adherence to fixed deadlines to ensure that all important information is covered and no details are missed when conducting the various technical and social studies?
• Do you have procedures in place to review, verify and validate data in an objective and consistent manner?