End of Project Evaluation for The Clean Cities for Growth and Development Project

1. Introduction

WaterAid Zambia (WAZ) is an international Non-Governmental Organization dedicated exclusively to the provision of safe domestic water, sanitation and hygiene education to the world’s poorest. WAZ currently operates in six (06) districts; Monze, Kazungula, Mwense, Samfya, Mwandi and Lusaka districts. WAZ is now inviting sealed bids from eligible Consultants to undertake the End of Project Evaluation of the project Clean Cities for Growth and Development in Kanyama.

The purpose of this Terms of Reference is to provide a framework for planning and conducting the Final Evaluation for the Swedish Postcode Lottery funded project Clean Cities for Growth and Development in Kanyama.

This Final Evaluation will use both quantitative and qualitative methods to ascertain the impact of the project. It will also assess what factors enhanced and/or limited achievement of project targets as well as documentation of the results achieved and lessons learned for future programming. An external evaluation will provide a view of the project that is considered more objective by the intended users. It is further expected that the evaluator will conduct a theory-based evaluation.
This Terms of Reference describe the purpose, methodology, and procedure of the End of Project Evaluation of the project and describes the major issues to be assessed and discussed.

The Project was launched on 1 December 2018 and ended on 30 November 2020, with the overall aim of contributing to building inclusive, safe and sustainable cities and human settlements. As the Project has reached the end, the End of Project Evaluation will be conducted from 18th January 2021 to 5th February 2021.

Description of Project Being Evaluated

The overall purpose of the project was to empower the Kanyama community to claim their rights to adequate sanitation and land rights and to hold government accountable for the fulfilment of these rights

The aim of the project was to contribute to building inclusive, safe and sustainable cities and human settlements. In order to drive this, the following outcomes were key;

1. Governance and participation – Empowered citizens actively engage local government structures that are responsive and accountable to citizens for the fulfilment of rights to WASH and other social service.
2. Access to sanitation services – Increased number of households with access to effective and safe sanitation and solid waste management services (especially for women and girls)
3. Clean, Safe and Green Environment: Improved health, wealth and well-being of residents of Kanyama.

The project was also in line with WaterAid’s global vision of a world where everyone, everywhere has safe water, adequate sanitation, and hygiene by 2030. WaterAid hopes to achieve this goal by transforming the lives of the poorest and most marginalized people by improving access to safe water, sanitation and hygiene.

2. Purpose and Objectives

The purpose of the end of project evaluation is to establish and document the impact and effectiveness of the project interventions to render accountability to Swedish Postcode Lottery, Ministry of Local Government, Lusaka City Council, Lusaka Water and Sanitation Company, stakeholders and communities. The evaluation is expected to provide data on the performance, impact and sustainability of project interventions. The findings and recommendations will contribute to a learning process which enables WAZ to draw lessons from its experience in order to contribute to building inclusive, safe and sustainable cities and human settlements in Zambia.
The evaluation will assess the extent to which the outcomes of the project have been achieved, determine Relevance, efficiency, effectiveness and sustainability of the interventions and document new knowledge and important topics for further inquiry, action, lobbying and influence.
The evaluation will also help to assess the effectiveness of beneficiary accountability system and mechanisms that were used during the project implementation period. A team of independent, external consultant will be contracted to assess the performance and results against the mandate that was set in the project design, and to determine the reasons for success or lack thereof, draw lessons and recommendations for improved performance in future similar interventions.
The evaluation will employ the six specific evaluation criteria for humanitarian action from the Organization for Economic Cooperation and Development – Development Assistance Committee (OECD-DAC) to guide the study with special emphasize on sustainability.

Based on the Project Outcomes and Outputs Table, the evaluation is designed to assess the Project from the perspectives of:

1. Project achievement,
2. Implementation process, and
3. Five evaluation criteria.

3. Scope of Work

Secondary information analysis:

• Desk review of relevant programme and project documents and reports such as proposals, assessments, project budgets, monitoring and assessment reports, WaterAid guidelines
• Establish working contacts with all the relevant stakeholders in the targeted areas
• Prepare and submit to WAZ the evaluation proposal and the inception report including methodology to be used, work plans and schedules for both quantitative and qualitative aspect of the assignment for review and feedback and approval by WaterAid.

Primary information analysis:

• Field visits to selected sites; among other surveys, interviews and/or focus group discussions with local partner, MLG, LCC, LWSC, community members and other stakeholders by the evaluator;
• Evaluator will submit an inception report (with detailed methodology, research tools and timeline/logistics) for further discussion.
• Evaluator will lead a kick-off workshop in Lusaka, train enumerators in tools to used
• Design data collection tools
• Interview selected respondents during the evaluation
• Hire and train the data collection team, including all required logistics e.g venue
• Lead and supervise the data collection
• Conducting entry and exit meetings (de briefing) with WAZ staff and key stakeholders
• Submission of draft evaluation report and finalize it based on the feedback from WAZ, WAA WA-SARO
• Submission of individual recommendations for WAZ and local partners

4. Key Activities, Deliverables and Timeframe

Inception Report: Deadline: 3 days before the data collection starts. The inception report needs to be approved by WAZ. The consultant will prepare and submit an inception report detailing how the evaluation will be carried out from their point of view. The report will outline the evaluation design, sampling methods to be used and questions to be answered and detailed work plan for the entire exercise. Draft questionnaires, interview guides and other data collection tools will be submitted to WAZ for review and approval before data collection starts. As part of the inception report, the consultant must provide a data analysis plan showing the questions and analysis for each of the project indicators to be investigated.

Research methodology and respective tools should be developed by the consultant and will be reviewed and approved by WAZ.

Preliminary Evaluation Report: 01 February 2021, The consultant will submit draft evaluation report to WAZ. The draft report will be reviewed, and comments provided on the report within a week of submission.

Final evaluation Report: 05th February 2021, The consultant will submit detailed final report outlining the evaluation methodology, findings, lessons learned and recommendations. The report shall incorporate specific simple and achievable recommendations, including the most appropriate strategies that can be undertaken and/or incorporated by WAZ and partners/stakeholders to attempt to address the issues identified. The final report should address the issues and questions raised in this ToR and correspond to the evaluation objectives set out above.

The Report format should include (but not limited to) the following:

• Executive Summary presenting the major findings and recommendations
• Evaluation aims, objectives, and scope
• Assessment of the project’s underlying impact logic
• Description of the methodology used
• Limitations
• Description of the assessment context and process including its constraints and challenges
• Detailed findings (related to the objectives and structured considering DAC criteria and questions)
• Analysis of the findings (following the key questions outlined in the ToR)
• Conclusions
• Recommendations for the project approach
• Lessons learned

The annexes of the report should contain (but not be limited to):

• The evaluation Terms of Reference.
• Inception report
• Maps
• List of interviewed people, with affiliation, contact details and bibliography
• Raw collected data and the data base on SPSS, STATA, Excel as an attached file
• Research Tools.

Criteria to ensure the quality of the evaluation report

• The evaluation report should represent a thoughtful, well researched and well-organized effort to objectively evaluate what worked in the project, what did not and why.
• The evaluation report shall address all questions included in the scope of work.
• The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by WAZ
• Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.
• Limitations to the evaluation shall be disclosed in the report, with attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).
• Evaluation findings should be presented as analysed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence
• Sources of information need to be properly identified and listed in an annex
• Recommendations need to be supported by a specific set of findings.
• Recommendations should be action oriented, practical and specific, with defined responsibility for the action.

5. Budget Estimate

The consultant will develop a detailed budget (including all logistics from training to data collection) and work plan based on the details in the TOR.

6. Qualifications

The consultant must have undertaken similar work in the last three years and should have the following competencies:

• Consultant must have excellent skills in research, monitoring and evaluation of WASH projects including accountability to beneficiaries.
• A good understanding of the SDG target 11.1: By 2030, ensure universal access to adequate equitable, sustainable and affordable housing and basic services and secure land tenure for all, and improve slum-like conditions everywhere through a participatory approach.
• Experience in multi-methodological and interdisciplinary approaches and data collection and analysis techniques in evaluation of development programmes.
• Demonstrable experience (at least 5 years) in research/evaluation of projects related to the WASH sectors, research and analysis
• Ability to conduct high quality research, meet deadlines and respond to requests and feedback provided timely and appropriately
• Excellent track record in designing and conducting quantitative and qualitative research, analysis and evaluation
• Familiarity with international quality and accountability standards applied in development cooperation.
• Strong analytical and conceptual skills to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner
• Excellent facilitation skills, co-ordination, negotiation skills and oral and written communication skills (particularly report writing).
• Demonstrated capacity to work both independently and as a team

7. Quotation Requirements

The consultant is expected to submit technical and financial proposals, separately comprising the following components:

a) The consultants understanding of the Terms of Reference
b) A clear and elaborate work plan
c) Proposed evaluation budget that include the schedule of payments
d) Organizational capacity statement, experience and activities related to Project Evaluations.
e) Curriculum Vitae for all the consultants and proposed capacity of any complementary staff (CVs submitted should be of staff that will be on this assignment)
f) Names, addresses, telephone numbers of three organizations that will act as professional referees
g) Copies of relevant work undertaken in the last 2 years (to be returned after evaluation of bids).
h) Full names, telephone numbers, e-mail address, physical address and contact person of the consultant

8. Tenders

• Tenders/offers will be accepted from consultants as well as from commercial companies, NGOs or academics.
• Duration of evaluation: It is anticipated that the evaluation will last 15 – 20 working days.
• The consultant is responsible for their own travel and data collector’s itinerary.

To request for a detailed Terms of Reference, contact [email protected].
Please submit via email, a complete financial and technical proposal by Wednesday, January 13th, 17:00hrs to: Natasha Yambayamba (Head of People and Organisational Development)
Email: [email protected] using “End of Project Evaluation for the Clean Cities for Growth and Development Project” as the subject.
Late submissions will not be accepted.