End of Project Evaluation for Improving WASH in Schools and Healthcare Facilities in Zambia Project

1. Introduction

WaterAid Zambia (WAZ) is an international Non-Governmental Organization dedicated exclusively to the provision of safe domestic water, sanitation and hygiene education to the world’s poorest. WAZ currently operates in six (06) districts; Monze, Kazungula, Mwense, Samfya, Mwandi and Lusaka districts. WAZ is now inviting sealed bids from eligible Consultants to undertake the End of Project Evaluation of the project Improving WASH in Schools and Healthcare Facilities in Zambia.

The purpose of this Terms of Reference is to provide a framework for planning and conducting the Final Evaluation for the Helmsley Charitable Trust funded Grant 2018PG-VCA005, Improving WASH in Schools and Healthcare Facilities in Zambia.

This Final Evaluation will use both quantitative and qualitative methods to ascertain the impact of the project. It will also assess what factors enhanced and/or limited achievement of project targets as well as documentation of the results achieved and lessons learned for future programming. An external evaluation will provide a view of the project that is considered more objective by the intended users. It is further expected that the evaluator will conduct a theory-based evaluation.
This Terms of Reference describe the purpose, methodology, and procedure of the End of Project Evaluation of the project and describes the major issues to be assessed and discussed.

The Project was launched on 1 August 2017 and ended on 30 November 2020, with the overall goal of ensuring access to sustained and inclusive WASH services in schools and healthcare facilities in five districts of Zambia

As the Project has reached the end, the End of Project Evaluation will be conducted from 21st January 2021 to 26th February 2021.

1. Description of Project Being Evaluated

The overall purpose of the project was to improve education and health outcomes of community members, particularly women and children, in the five districts in Southern, Western and Luapula provinces. To achieve this, the project used a mix of strategies:

• Building WASH demonstration facilities in schools and HCFs that are inclusive, gender sensitive and easily maintained;
• Mobilizing and empowering communities to demand their rights to WASH;
• Strengthening institutional capacity to provide WASH in schools and HCFs;
• Influencing and engaging the public around sustainable and inclusive WASH.

The overall goal of the project was to “Ensure access to sustained and inclusive WASH services in schools and healthcare facilities (HCFs) in five districts of Zambia”. The overall strategy of the project is to “Develop and demonstrate alternative WASH services and management models, empower communities and strengthen local structures to improve sustainable and inclusive access to WASH services in schools and healthcare facilities”.

The project had four (4) outcomes:

1. Increased access to inclusive and sustainable water supply and sanitation services for 94,500 people in 14 schools and 40 healthcare facilities. To achieve this outcome, WaterAid is focusing on modelling water and sanitation infrastructure, which especially benefit women and children. WaterAid will help rehabilitate and install WASH infrastructure for water supply, sanitation and hygiene that is inclusive and takes into account gender, disability and age in 25 rural health centres (RHC), 15 health posts (HP) and 14 schools.
2. Adoption of handwashing practices by 25,500 people in schools and healthcare facilities. To achieve this outcome, WaterAid is focusing on promoting hygiene behaviour change around handwashing with soap among teachers, students, healthcare workers, patients and community members.
3. Increased knowledge and capacity of communities in five districts to demand their rights to quality WASH services in unserved schools and healthcare facilities. To achieve this outcome, WaterAid will work with child rights clubs and other partners in 14 schools to raise awareness and provide training on children’s rights to WASH.
4. Improved institutional management for sustainability of water supply and sanitation services and hygiene behaviours. To achieve this outcome, WaterAid will focus on enhancing the capacity of district level structures responsible for delivery and maintenance of water supply and sanitation infrastructure in the schools and HCFs. District level structures include the local authorities, District Health Officers and District Educational Boards.

The purpose of Grant 2018PG-VCA005 was in line with WaterAid Zambia’s 2016-2021 Country Program Strategy with the objective of increasing access to quality and sustainable water, sanitation and hygiene services, particularly for marginalized groups, through a well-coordinated and effective WASH sector that fulfils the rights of citizens.

It was also in line with WaterAid’s global vision of a world where everyone, everywhere has safe water, adequate sanitation, and hygiene by 2030. WaterAid hopes to achieve this goal by transforming the lives of the poorest and most marginalized people by improving access to safe water, sanitation and hygiene.

2. Purpose and Objectives

The purpose of the end of project evaluation is to establish and document the impact and effectiveness of the project interventions to render accountability to HCT, stakeholders, MOGE, MOH and communities. The evaluation is expected to provide data on the performance, impact and sustainability of project interventions. The findings and recommendations will contribute to a learning process which enables WAZ to draw lessons from its experience in order to improve the equitable, inclusive quality WASH service provision in Schools, HCFs and communities in the target districts.

The evaluation will assess the extent to which the outcomes of the project have been achieved, determine Relevance, efficiency, effectiveness and sustainability of the interventions and document new knowledge and important topics for further inquiry, action, lobbying and influence.
The evaluation will also help to assess the effectiveness of beneficiary accountability system and mechanisms that were used during the project implementation period. A team of independent, external consultant will be contracted to assess the performance and results against the mandate that was set in the project design, and to determine the reasons for success or lack thereof, draw lessons and recommendations for improved performance in future similar interventions.
The evaluation will employ the six specific evaluation criteria for humanitarian action from the Organization for Economic Cooperation and Development – Development Assistance Committee (OECD-DAC) to guide the study with special emphasize on sustainability.

Based on the Project Outcomes and Outputs Table, the evaluation is designed to assess the Project from the perspectives of:

1. Project achievement,
2. Implementation process, and
3. Five evaluation criteria.

3. Scope of Work

Secondary information analysis:

• Desk review of relevant programme and project documents and reports such as proposals, assessments, project budgets, monitoring and assessment reports, WaterAid guidelines
• Establish working contacts with all the relevant stakeholders in the targeted areas
• Prepare and submit to WAZ and relevant partners the evaluation proposal and the inception report including methodology to be used, work plans and schedules for both quantitative and qualitative aspect of the assignment for review and feedback and approval by WaterAid.

Primary information analysis:

• Field visits to selected sites; among other surveys, interviews and/or focus group discussions with local partners, MOGE, MOH, learners, teachers, community members and other stakeholders by the evaluator;
• Evaluator will submit an inception report (with detailed methodology, research tools and timeline/logistics) for further discussion.
• Evaluator will lead a kick-off workshop in Lusaka (inviting Enumerators and key stakeholders e.g MoH, MWDSEP, Implementing partners), train enumerators in tools to used
• Design data collection tools
• Interview selected respondents during the evaluation
• Hire and train the data collection team
• Lead and supervise the data collection, including all required logistics e.g transport and stationery
• Conducting entry and exit meetings (de briefing) with WAZ staff and key stakeholders
• Submission of draft evaluation report and finalize it based on the feedback from WAZ, WAA WA-SARO
• Submission of individual recommendations for WAZ and local partners

4. Key Activities, Deliverables and Timeframe

Inception Report: Deadline: One week before the data collection starts. The inception report needs to be approved by WAZ. The consultant will prepare and submit an inception report detailing how the evaluation will be carried out from their point of view. The report will outline the evaluation design, sampling methods to be used and questions to be answered and detailed work plan for the entire exercise. Draft questionnaires, interview guides and other data collection tools will be submitted to WAZ for review and approval before data collection starts. As part of the inception report, the consultant must provide a data analysis plan showing the questions and analysis for each of the project indicators to be investigated.

Research methodology and respective tools should be developed by the consultant and will be reviewed and approved by WAZ.

Preliminary Evaluation Report: 15th February 2021. The consultant will submit draft evaluation report to WAZ. The draft report will be reviewed, and comments provided on the report within a week of submission.

Final evaluation Report: 26th February 2021 The consultant will submit detailed final report outlining the evaluation methodology, findings, lessons learned and recommendations. The report shall incorporate specific simple and achievable recommendations, including the most appropriate strategies that can be undertaken and/or incorporated by WAZ and partners/stakeholders to attempt to address the issues identified. The final report should address the issues and questions raised in this ToR and correspond to the evaluation objectives set out above.

The Report format should include (but not limited to) the following:

• Executive Summary presenting the major findings and recommendations
• Evaluation aims, objectives, and scope
• Assessment of the project’s underlying impact logic
• Description of the methodology used
• Limitations
• Description of the assessment context and process including its constraints and challenges
• Detailed findings (related to the objectives and structured considering DAC criteria and questions)
• Analysis of the findings (following the key questions outlined in the ToR)
• Conclusions
• Recommendations for the project approach
• Lessons learned

The annexes of the report should contain (but not be limited to):

• The evaluation Terms of Reference.
• Inception report
• Maps
• List of interviewed people, with affiliation, contact details and bibliography
• Raw collected data and the data base on SPSS, STATA, Excel as an attached file
• Research Tools.

Criteria to ensure the quality of the evaluation report

• The evaluation report should represent a thoughtful, well researched and well-organized effort to objectively evaluate what worked in the project, what did not and why.
• The evaluation report shall address all questions included in the scope of work.
• The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by WAZ
• Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.
• Limitations to the evaluation shall be disclosed in the report, with attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).
• Evaluation findings should be presented as analysed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence
• Sources of information need to be properly identified and listed in an annex
• Recommendations need to be supported by a specific set of findings.
• Recommendations should be action oriented, practical and specific, with defined responsibility for the action.

5. Budget Estimate

The consultant will develop a detailed budget (including all logistics from training to data collection) and work plan based on the details in the TOR.

6. Qualifications

The consultant must have undertaken similar work in the last three years and should have the following competencies:

• Consultant must have excellent skills in research, monitoring and evaluation of WASH projects including accountability to beneficiaries.
• A good understanding of the WASH sector in Zambia.
• Experience in multi-methodological and interdisciplinary approaches and data collection and analysis techniques in evaluation of development programmes.
• Demonstrable experience (at least 5 years) in research/evaluation of projects related to the WASH sectors, research and analysis
• Ability to conduct high quality research, meet deadlines and respond to requests and feedback provided timely and appropriately
• Excellent track record in designing and conducting quantitative and qualitative research, analysis and evaluation
• Familiarity with international quality and accountability standards applied in development cooperation.
• Strong analytical and conceptual skills to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner
• Excellent facilitation skills, co-ordination, negotiation skills and oral and written communication skills (particularly report writing).
•  Demonstrated capacity to work both independently and as a team

7. Quotation Requirements

The consultant is expected to submit technical and financial proposals, separately comprising the following components:

a) The consultants understanding of the Terms of Reference
b) A clear and elaborate work plan
c) Proposed evaluation budget that include the schedule of payments
d) Organizational capacity statement, experience and activities related to Project Evaluations.
e) Curriculum Vitae for all the consultants and proposed capacity of any complementary staff (CVs submitted should be of staff that will be on this assignment)
f) Names, addresses, telephone numbers of three organizations that will act as professional referees
g) Copies of relevant work undertaken in the last 2 years (to be returned after evaluation of bids).
h) Full names, telephone numbers, e-mail address, physical address and contact person of the consultant

8. Tenders

• Tenders/offers will be accepted from consultants as well as from commercial companies, NGOs or academics.
• Duration of evaluation: It is anticipated that the evaluation will last 25 – 30 working days.
• The consultant is responsible for their own travel itinerary. Costs and logistics for data collection should be part of Consultant budget submission

To request for a detailed Terms of Reference, contact [email protected].
Please submit via email, a complete financial and technical proposal by Thursday, January 14th, 17:00hrs to: Natasha Yambayamba (Head of People and Organisational Development)
Email: [email protected] using “End of Project Evaluation for Improving WASH in Schools and Healthcare Facilities in Zambia Project” as the subject.
Late submissions will not be accepted.