Invitation for Expressions of Interest – Consultancy to Undertake an End Line Survey/ Evaluation of the Impact of the Clean and Safe Drinking Water Promotion Project Phase II (Nyimba, Mumbwa And Shibuyunji)

TERMS OF REFERENCE

1.0  Project Summary:

Diarrheal diseases have continued to be a major contributor to loss of human lives in Mumbwa, Nyimba and Shibuyunji Districts. Sources of drinking water include streams, rivers, shallow wells and boreholes. However, people draw water from the rivers and streams which are typically contaminated, especially during the rainy season as they tender to their fields. The use of water purification sachets from Procter and Gamble (P&G) will help households to ensure clean water consumption and aid in reducing morbidity and mortality from water borne diseases. The water contamination situation is compounded by lack of latrines whereby open defecation among communities is quite common. However, due to extensive sanitation and hygiene trainings throughout Procter and Gamble’s Phase 1 project, communities have started to build regularly used toilets in the sites where the project was being implemented (Mumbwa and Nyimba Districts).

The project seeks to reach 39,000 individuals of approximately 6,500 households each year in the districts of Mumbwa, Nyimba and Shibuyunji with the distribution of one sachet per household per day for water purification to reduce incidences of water borne diseases over a period of three (3) years. With Phase II, the project will scale up the provision of P&G purifier of water sachets to beyond the current sites to include Shibuyunji District.

To ensure sustainability of the project, ChildFund will work with trained P&G Distributors, District Directors of Health, health centre staff, community leaders, community members and other community structures who will take the lead in project implementation process in each district. Youth drama groups, focus group meetings and community meetings will be organized to help raise awareness among adults and the younger generation on the health effects of using polluted water sources for household activities and the need to purify the water before use to reduce incidences of water borne diseases and improve personal hygiene.

Awareness on the use of the sachets, conducts practical demonstrations, and monitor and evaluate the effect of the project through a baseline/process evaluation and project end surveys.

ChildFund, which has been working in these districts over the past decade, will use its strong relationships that were built over time among the project’s communities. ChildFund also has several years of experience in community development approaches for implementing project activities within these communities. After the project will also use the services of community volunteers to carry out education and Phase I of the project, ChildFund is in a better position to implement Phase II of the project more effectively.

ChildFund will work to continue reducing the incidences of water borne diseases in schools and communities using improvement in and availability of safe drinking water as the centrepiece of the approach. The total cost of the project, which covers three (3) years (2017-2020) will be US$930, 926 , to be funded annually ($ 385, 524 – YR 1; $260, 466 –   YR 2 and $284,936 – YR 3), with annual extensions and future grant payments dependent on progress updates.

2.0     Rationale, Evaluation Users and Stakeholder

2.1     Rationale

The end of project evaluation is being commissioned for the following reasons:

• The P&G project conducted an evaluation for phase 1 study before the start of the project that provided a situational analysis and benchmarks of key indicators. This allowed P&G project to establish indicator baseline information and to verify the targets established in the project agreement. The evaluation conducted provided progress made towards set project targets after implementation of phase. Therefore, end of project evaluation will showcase overall P&G performance in comparison with the evaluation study.
• Project accountability and transparency – to allow stakeholders and funder to be aware of the project performance. End of project evaluation will also provide a platform of obtaining feedback from the supported communities and other stakeholders
• Provide strategic and concrete evidence on the relevance, results, processes, and resource utilization for the project.

2.2     Users of the Evaluation

The expected users of evaluation results will be project stakeholders, ChildFund Zambia management, Government of the Republic of Zambia (GRZ) line ministries, P&G, and other key partners that are involved in project implementation.

2.3     Stakeholders

A number of stakeholders have interests in the results of the evaluation.  and therefore, the Evaluation Team (ET) will provide a detailed stakeholder analysis as part of the Inception phase of the evaluation process.

Accountability to affected populations is tied to ChildFund’s commitments to include beneficiaries as key stakeholders in ChildFund’s work. ChildFund is committed to ensuring gender equality and women’s empowerment in the evaluation process, with participation and consultation in the evaluation with women and men from beneficiary groups.

3.0     Evaluation Purpose, Objective and Scope

3.1     Purpose

The purpose of the end of project evaluation is to document the overall project performance towards set project goals, objectives, and outcomes and draw out lessons useful for future Water and Sanitation (WASH) programming.

3.2     Specific Objectives

In particular, the final evaluation will aim to:

• Assess the relevance of the strategies and the approach used in the implementation of P&G Project.
• Assess the effectiveness and efficiency in the implementation of P&G Project activities at province, district, and sub-district levels including community structures.
• Assess the results achieved in P&G Project and the extent to which sustainability considerations were built in.
• Establish end-line values for outcome level indicators for P&G Project and compare with evaluation data for phase 1 P&G project.
• Carry out a Gap Analysis which will act as our baseline and collect information in the proposed new areas in Nyimba and Mumbwa district targeted areas
• Establish the extent of coordination with partners and other actors in the area of WASH programming with a view to strengthening linkages to service provision; and
• Identify lessons and good practices from the implementation of P&G Project and opportunities to improve overall WASH programming among other stakeholders.

3.3     Scope

The evaluation will cover P&G project interventions during the period August 2017 – October 2020. The focus will mainly be on five (5) indicators related to households accessing safe and clean drinking water and the geographical scope will include 3 districts in which the P&G project was implemented. The districts must include urbanized and rural areas and as well urban and rural communities in the selected areas.  Data must be segregated according to district, rural and urban areas. The evaluation will also include a gap analysis in the new areas of Nyimba and Mumbwa to establish baseline line values in water borne disease cases, levels of awareness in sanitation and hygiene, gaps in WASH coordination and stakeholder engagement in the targeted district areas,

4.0       Evaluation Approach

4.1      Evaluation Criteria and Questions

• Evaluation Criteria: The evaluation will apply the international evaluation criteria of Relevance, Effectiveness, Efficiency, Impact and Sustainability.
• Evaluation Questions: Allied to the evaluation criteria, the evaluation will address the following key questions, which will be further developed by the evaluation team during the inception phase. Collectively, the questions aim at highlighting the key lessons and overall performance of the P&G project. Below is a table with relevant criteria and evaluation questions.

Table 1: Criteria and evaluation questions

Criteria	Evaluation Questions
Relevance	To what extent were the operations and objectives of the P&G project consistent with beneficiaries’ needs, GRZ and donors’ requirements? Were the approaches and strategies used relevant to achieve the intended outcomes of the project?
Impact	To what extent did the P&G project meet the set targets on key performance indicators in the log frame?
Effectiveness	What best practices can be documented with regard to service delivery and implementation?
Efficiency	To what extent have the resources of the project been efficiently utilized?
Exit and sustainability	What has the communities learnt from the project and will continue to do after the life of the project? What will the community likely going to continue beyond the life of the project? What supportive measures have been put in place to ensure sustainability?
Cross-cutting Issues
Coordination, Gender, and Environment.	What has been the environmental concerns and mitigations? To what extent has the project implementation mainstreamed gender issues?

5.0     Approach and methodology

The evaluation approach will be both process and outcome based.

The project implements a routine monitoring system based on a Log Frame and corresponding data collection plan to collect data against key outcome indicators.

The consultant is expected to employ a variety of data collection and analysis techniques for both quantitative and qualitative data to ensure a comprehensive evaluation exercise. It will include but not limited below listed:

• Document and systems review: Review of existing documentation, including; project reports, project log frame and monitoring and evaluation data.
• Surveys: Application of structured survey questionnaires with a representative, random sample of target population to quantitatively assess outcomes.
• Focus Group Discussions: With target groups and other stakeholders to assess implementation experiences and effectiveness, document successes, challenges and lessons learned, and develop recommendations for future WASH programming.
• Key Informant Interviews: Consultations with key project stakeholders, including field staff and partners. Guidance on appropriate stakeholders will be provided.

The consultant will be expected to:

• Review relevant project documents, including but not limited to: project reports, project log frame and previous survey for phase 1 project.
• Collect and review relevant guideline document and relevant secondary data etc.
• With the guidance of ChildFund M&E and P&G project manager develop, test and apply survey questionnaires. A representative random sample of the target population should be interviewed to assess outcomes and establish prospect impact of the project interventions in line with log frame indicators. The project is putting a strong emphasis on the need to collect information’s on outcome indicators.
• Design and conduct focus group discussions with relevant community members, both male and female, to assess implementation experiences and effectiveness, document successes, challenges and lessons learned, and develop recommendations for future WASH programming.
• Carry out key informant interviews i.e. consultations with key project stakeholders, including field staff and partners. Guidance on appropriate stakeholders will be provided by ChildFund P&G manager.
• Collate and analyses data.
• Present preliminarily findings to internal ChildFund key staff and invite comments at a one day workshop.
• Draft report and submit to ChildFund for comments.
• Incorporate comments and produce final report.

6.0     Expected Deliverables and Timeline

All written documentation is to be submitted using Microsoft Word in both soft and hard copy. The main body of all reports should be written in simple, non-technical language (i.e. plain English), with any technical material to be presented in annexes. All primary data collected and analysis conducted for the purpose of the evaluation will remain the property of ChildFund Zambia and must be submitted electronically and in a clear and comprehensible format in Excel.

The evaluation exercise should begin no later than the 15^th September 2020, with the evaluator(s) expected to take a total of 30 working days from the day of contracting to complete the assignment.

The consultant will provide the following deliverables to ChildFund within the timeframe stated:

1. Inception Report: within 5 working days of evaluation launch, a detailed report on the evaluator’s proposed approach to the evaluation will be submitted for approval. This will provide preliminary findings/understandings based on document review, rationale and a detailed description of the methodology and tools, research questions, analytical methods, budget with a breakdown of costs and detailed work plan for the entire exercise. Any draft questionnaires or interview forms will also be submitted for review at this stage.
2. Data collection: Training of numerators, testing questionnaires, refining data collection tools, and administer data collection within 10 working days.
3. Data analysis and reporting: Preliminary Report and Presentation: within 15 working days, the consultant will present the preliminary findings for discussion. This should include a draft set of recommendations and lessons learnt. A draft report incorporating comments by ChildFund should be produced.
4. Final Report: within 5 days, a detailed report of the overall findings of the end of project evaluation will be submitted to the Country Director for approval. The report structure is annexed to this ToR.
5. A Power point presentation outlining key findings and implications, and recommendations to be presented to ChildFund and key stakeholders.

7.0     Management and Implementation Responsibilities

The consultant will report directly to the M&E/KML Manager and P&G project Manager. However, s/he will also be expected to work closely with the field staff. Any proposed changes to the personnel listed in the application must be explained in the inception report and approved by M&E and P&G Project manager

ChildFund will provide the following:

• Relevant project documents.
• Guidance and technical support as required throughout the evaluation;
• Copies of all key background resources identified;
• Introductory meetings with key government staff;
• Comments and feedback on, and approval of, all deliverables within agreed timeline

The consultant will be responsible for:

• Review documents and submit inception report.
• Developing the detailed evaluation methodology.
• Conducting all data collection.
• Analysis of data and reporting in a clear and accessible format.
• Regular progress reporting to the evaluation manager, including responding to any comments or technical inputs wherever reasonable.
• Production of deliverables within agreed timeline and in accordance with quality requirements of ChildFund
• Seeking comments and feedback from P&G Manager regularly, through M&E Manager Coordinator, in sufficient time to discuss and incorporate these into the final report.
• Production of the final evaluation report containing data against all indicators in the project log frame, evidence-based responses to the key evaluation questions, summary of lessons learnt and recommendations for future programming.

Qualification and Desirable Competencies:

Applications from individuals or teams are welcome and will be assessed on their ability to demonstrate the following qualifications and competencies: 

Essential requirements:

• A minimum of 5 years’ experience in carrying out impact evaluations, demonstrable academic and practical experience in qualitative and quantitative research methodology, evaluation design and implementation.
• Lead consultant should have at least a masters degree in a social science or related field.
• Experiences in health-related programs, preferably including WASH programs
• Experiences in community engagement and System strengthening programs

issues

• Strong analytical and communication skills, evaluation experience and familiarity with Zambia WASH landscape
• Familiarity with the approaches, rules, regulations of the Donors Quality Assurance

ChildFund recommends the evaluation quality assessment checklist (see below) is used as an element of the proposed quality assurance system for the draft and final versions of the thematic evaluation report. The main aim of the checklist is to ensure that the thematic evaluation complies with evaluation professional standards

Evaluation quality assessment checklist:

Components	Explanation
1. Structure and Clarity of the Report	To ensure report is user-friendly, comprehensive, logically structured and drafted in accordance with international standards.
2. Executive Summary	To provide an overview of the evaluation, written as a stand-alone section including key elements of the evaluation, such as objectives, methodology and conclusions and recommendations.
3. Design and Methodology	To provide a clear explanation of the methods and tools used including the rationale for the methodological choice justified. To ensure constraints and limitations are made explicit (including limitations applying to interpretations and extrapolations; robustness of data sources, etc.) Ensure a reasonable sample seize which reliably capture data from
4. Reliability of Data	To ensure sources of data are clearly stated for both primary and secondary data. To provide explanation on the credibility of primary (e.g. interviews and focus groups) and secondary (e.g. reports) data established and limitations made explicit.
5. Findings and Analysis	To ensure sound analysis and credible evidence-based findings. To ensure interpretations are based on carefully described assumptions; contextual factors are identified; cause and effect links between an intervention and its end results (including unintended results) are explained.
6. Validity of conclusions	To ensure conclusions are based on credible findings and convey evaluators’ unbiased judgment of the intervention. Ensure conclusions are prioritized and clustered and include: summary; origin (which evaluation question(s) the conclusion is based on); detailed conclusion.
7. Usefulness and clarity of recommendations	To ensure recommendations flow logically from conclusions; are targeted, realistic and operationally-feasible; and are presented in priority order. Recommendations include: Summary; Priority level (very high/high/medium); Target (administrative unit(s) to which the recommendation is addressed); Origin (which conclusion(s) the recommendation is based on); Operational implications.

8.0     Budget

The proposed budget should include a breakdown of all costs including local transportation, field work, and consultancy fees.

9.0     Submission Details

Interested evaluators or firms are requested to submit:

1. An Expression of Interest detailing their interpretation of the ToR, proposed methodology including sampling framework, work schedule and proposed budget in Zambian Kwacha (ZMK)
2. A capability statement demonstrating how you meet the required qualifications and competencies;
3. Copies of all relevant Curriculum Vitae (CVs). Only CVs for the specific individuals that will form the proposed evaluation team – and will be directly involved should be included;
4. Two references (including one from your last client/employer).

All documents must be submitted by email to the procurement specialist [email protected],    copied to the Sponsorship and Programs Director: [email protected] and the M&E/KML Manager [email protected] by close of business 10^th September, 2020 with “Expression of Interest for End Line Survey” in the subject line.