Habitat for Humanity Zambia
Call for Consultancy: End of Project Evaluation: Habitat for Humanity Zambia.
Terms of Reference (TOR) for End Line Evaluation of Habitat’s Irish Aid/ Habitat Ireland funded project in Zambia.
End of Project Evaluation.
In 2016 Habitat Ireland were successful in attracting funding from Irish Aid for a project in Zambia entitled “Building Stability and Resilience: Good governance, empowerment, land rights, housing and livelihood for Lusaka (Kamanga, Linda, Chazanga, Chainda and Bauleni) and City of Ndola (Twapia and Chipulukusu)
In compliance with funder requirements and to ensure its own learning and good practice, Habitat Ireland and Habitat Zambia wish to commission an end of project evaluation which will focus on the communities supported by Habitat. A baseline survey was conducted prior to implementation of the 3 year programme and a subsequent Logistical Framework was developed to track impact and performance.
Project Description and Details
End of Three Year Project on July 31st 2019
Programme Coverage and Partners
The project was implemented in the aforementioned 7 communities in 2 districts and with 9 Community Based Organisations and three Technical partners. The CBOs partners are, Zambia Open Community Schools (ZOCs), Salvation Army (SA), Community Based TB Organisation (CBTO), Emmanuel Transit Centre (ETI), Tadhiwa Charity Organisation (TCO), Bauleni United Sports Academy (BUSA), Bwafwano Integrated Services Organisation (BISO) and SOS Children’s Village in Lusaka district. Samaritan Strategy in Ndola district.
The Technical partners are, Women and Law in Southern Africa (WLSA), Centre for Governance and the Zambia land Alliance (ZLA) operating in Lusaka and Ndola districts.
End Line Evaluation
Objectives of the Programme
- To improve access to land and secure tenure rights at the local level by strengthening and empowering communities to participate in governance and advocate for their rights.
- To mobilise community members and leaders and strengthen community responses to improve social perceptions, protection, prevention and services for HIV/Aids and gender based violence (GBV)
- To strengthen the capacity to protect and care for OVC, improving their livelihood and access to housing and sanitation
- Increased understanding, awareness visibility about secure tenure and housing rights and potential for citizen participation
- Increased ability to tract budgets, advocacy and improved participation of active citizens in accountability, advocacy and engagement with LA (CBO members, civic leaders and Ward Development Committee – WDC Members)
- Improved awareness, ability and will of local authorities (LAs) to implement pro vulnerable rights based land access and titling systems (raising their awareness, holding them accountable and collaborating to improve the system.
- Actions are taken by CBO and their members in holding authorities to account for advocating for policy changes or improved implementation.
- Strengthened CBO Programmatic/organisational capacity to collaborate and capacity to promote and advocate for HIV, OVC, reducing GBV and increasing gender rights at the community level
- Increased access to a new house and sanitation
- Increased number of OVCs are protected and their assets supported by knowledge in inheritance rights and developing a will and basic home maintenance.
Working Arrangements with Govt. of Zambia
The existing working agreement is a mutual one and we have been working and collaborating within an open framework, in this case the collaboration has been with the Local Authorities. HFHZ is part of the District Development Collaborating Committees. There is no tripartite agreement. However, HFHZ has signed agreements with local partners.
Overall project goal
To promote equitable development and human rights for women, orphans and other children made vulnerable by HIV and AIDS in Zambia by:
- Improving good governance and increasing their participation in governance and accountability
- Facilitating a supportive environment with a culture of empowerment, inclusion, prevention, child-focused and gender-equity service
- Improving the livelihood potential and access to resources of most vulnerable OVCs and Women.
Nature and Scope of the Project
This project will ensure the stability, resilience and rights of poor OVC and their caregivers (mostly women), affected by HIV and AIDS in Zambia. Good governance and management of land and housing policy will be improved by empowering citizens and increasing their participation in advocacy and accountability activities. A change of culture is being facilitated, for more inclusion, prevention of HIV/AIDS and gender based violence (GBV), better services and stronger local organisations. OVC and their families will have improved livelihood and assets, through secure land tenure, inheritance protection, and for some, new houses and sanitation and vocational training.
Timelines for End Line Evaluation
Invitation to Tender Circulated on Go Zambia Jobs /Firms April 16th2019
Deadline for Submission of Tender to Habitat Zambia – April 25th 2019
Result of Selection of Consultant notified to successful Consultant – May 8th.2019
Field work completed – end of July 2019
Collation, Analysis by Consultant and first Draft of Report to Habitat Zambia and Ireland by mid-August 2019
Comments and amendments back and forth (Habitat and Consultant) during August.
Final Evaluation Report submitted to Habitat end August 2019
Purpose of the Independent final evaluation.
The independent final evaluation report will be used to inform Habitat, its partners, and Irish Aid of the performance of the project. The Independent End Line Evaluation report must be a substantial document that (a) answers all the elements of this Terms of Reference (TOR); (b) provides findings and conclusions that are based on robust and transparent evidence; and (c) where necessary supplements Habitat’s MEAL data with independent research. This is in line with Habitat’s best practice expectation.
Key Objectives of the Evaluation
- To independently verify (and supplement where necessary), Habitat’s record of achievement as reported through its various Reports and defined in the project log frame.
- Value for money: To assess the extent to which the project was good value for money, which includes considering: (a) How well the project met its objectives. (b) How well the project applied value for money principles of effectiveness, economy, efficiency and equity in relation to delivery of its outcomes. (c) How well the project aligns with Irish Aid’s goals of supporting the delivery of the SDGs
- To assess the extent to which organisational structural challenges affected final project outcomes and if any, what mitigation initiatives were effective in minimising negative impact?
- Design and sustainability: To assess the effectiveness of Habitat’s programme approach in securing the sustainability of the work beyond the lifetime of funding. In particular, the Evaluation should consider Habitat’s (a) project design vis a vis sustainability (e.g. community ownership) (b) Partnership approach vis a vis sustainability (c) The relationship with GOZ staff and officials vis a vis sustainability.
- Impact: a) To assess what has happened because of the project that wouldn’t otherwise have happened. b) To chart any unintended benefits of the programme – social, economic, political, environmental and c) To chart any unintended negative impacts of the programme – social, economic, political, environmental
Verification of Habitat’s Reporting
The first task of the final evaluation will be to verify Habitat’s achievements and under-achievements measured against the Log Frame. A number of Reports will be made available to the appointed Consultant to include; Baseline Study, Mid Term Evaluation, IA Monitoring Report 2018
It will be the responsibility of the Consultant to verify the information that was collected by Habitat from partners and beneficiaries for reporting purposes and possibly supplementing this data with additional information collected through primary and secondary research.
The appointed Evaluator will also be expected to verify the results from the project log frame which will capture what the project has achieved. However, there are other activities and results that occurred outside the log frame that will require examination in order to respond to the different evaluation questions. The required verification reporting must include a review of the data and systems that were used by Habitat to populate results.
Assessment of Value for Money
The final evaluation needs to assess the extent to which the delivery and results of the project are good value for money. Value for money can be defined in many ways, but at a minimum the evaluation report should include an assessment against:
- How well or otherwise the project applied value for money principles of effectiveness, economy, efficiency, equity, in relation to delivery of its outcomes.
- Did Habitat do the right things in the right ways at the right time to ensure successful programme delivery and completion.
Evaluation Questions: Please note that the attention given to each evaluation question may vary depending on the availability of data, so the independent evaluator should use her/his discretion in the level of effort used to respond to these questions. Furthermore, the Habitat National Director Kanyata Mukelabai, who will manage the evaluation process, together with other Habitat staff will guide the appointed consultant in this regard. The Evaluator is asked to structure research questions according to the OECD-DAC criteria of relevance, effectiveness, efficiency, sustainability and impact.
- To what extent did Habitat support achievement towards the intended SDGs?
- To what extent did the project target and reach the most marginalised?
- To what extent did the project mainstream gender equality in the design and delivery of activities?
- How well did the project respond to the needs of the target beneficiaries, including how these needs evolved over time?
- To what extent do the results that are reported to Irish Aid and Habitat Ireland represent a fair and accurate record of achievement as reported in quarterly financial reporting, annual financial and narrative reporting and midterm evaluation?
- To what extent has the project delivered results that are value for money? To include but not limited to the questions provided at Value for Money section above.
- To what extent has the project used learning to improve delivery?
- What are the key drivers and barriers affecting the delivery of results for the project? This question should include comment on Habitat Zambia and Habitat Ireland roles in delivery of results.
- To what extent were lessons learnt from the experience of project delivery and were these lessons applied in a timely fashion to the work of the project going forward?
- Comment on the type and quality of Project adaptations that led to enhanced results, if any.
- To what extent did Habitat deliver results on time and on budget against agreed plans?
- To what extent did the project understand cost drivers and manage these in relation to performance requirements?
- To what extent has the project levered additional resources (financial and in-kind) from other sources? What effect has this had on the scale, delivery and/or sustainability of the activities?
- To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends.
- Do partners feel better able to continue delivering after the project ends?
- To what extent did project design assist with sustainability?
- To what extent and how has the project built the capacity of civil society?
- To what extent and how has the project affected people in ways that were not originally intended?
- To what extent has the project impacted on relationships with GOZ officials and in engaging those officials into the project/project outcomes?
- What level of sustainable impact has the project had on beneficiaries. What proportion of them feel enabled and empowered to continue applying knowledge and skills learnt after the project?
- To what extent has housing poverty been reduced?
Are homes built to required House Quality Standard?
- Do beneficiaries and Change Agents feel empowered to continue to lobby local government on Housing Rights issues?
Most significant changes
The evaluator should collect stories of change from identified beneficiaries (beneficiary voices), civic leaders and change agents. The stories should focus on what was considered to be the most significant change the project has brought to their lives, capacities or way of operating.
The Evaluator is encouraged to apply a mixed method approach for assessing impact. This will combine qualitative data to provide an explanation of ‘why’ and ‘how’ the project has achieved the type and scale of results that are quantitatively observed, or failed to do so. A wide range of evidence sources is recommended which, when brought together will produce a plausible assessment of the contribution Habitat made to outcomes and impact. The evaluator is encouraged too to triangulate data sources so that findings are as robust as possible. E.g. Household level/ home owner level, group/partner, GOZ officials, Habitat. Possible methods for conducting primary and secondary research can include: interviews with staff at Habitat offices in Zambia and Ireland, with delivery Partners, focus group discussions with ultimate beneficiaries, surveys with project partners, GOZ officials and other relevant stakeholders, systematic reviews of secondary data, verifying reported data through back checking and quality control assessments. An indicative list of Materials to Review as secondary research is contained in Annex 1 of this TOR.
Contractual and Reporting Arrangements
The Independent Evaluator should be a suitably qualified and experienced consultant or consulting firm.
The Consultant/ Consultant’s team should include:
- An evaluation specialist with a minimum of seven years’ experience in programme/project evaluation in an international development context.
- Experience in results-based M&E
- Ability to plan and design the evaluation approaches and research methodologies, including quantitative and qualitative research methods.
- Relevant subject matter knowledge and experience such as Housing sector, Advocacy, Gender.
- A demonstrated understanding of the external context of Zambia.
- Track record in interpreting baseline data and conducting a final evaluation
- Experience in implementing primary research methods and interpreting secondary research sources.
- The extent to which the Evaluator or Team has appropriate country knowledge/experience. E.g. Language proficiency/ arrangements for translation
- The appointed Evaluator must not have a conflict of interest with the ongoing activities of Habitat, its Partners, or the GOZ officials involved in the project or the beneficiaries of the project.
The appointed Evaluator must uphold the safeguarding of the vulnerable children and adults from exploitation, any form of harassment and abuse during and after the interaction with them in the course of the evaluation.
Approximately 17 consultancy days are available for this assignment. A work plan and budget to be agreed between Habitat and the consultant. The evaluation will be managed by the National Director based in Lusaka. The consultant will report directly to the National Director during the process and to the CEO of Habitat Ireland regarding the draft and final report.
Terms and Conditions Logistics and professional fee:
Applicants will be asked to submit a budget. The consultant will be paid a professional fee for consultancy days agreed in the work plan and costs to cover transport, communication, research assistants’ costs and report production.
Tax and insurance: The consultant/s shall be responsible for own tax and insurance during the assignment.
Code of conduct: The consultant will be expected to adhere to Habitat’s values and principles.
Additional terms and conditions of service shall be spelt out in the contract.
Application with CVs, testimonials and information on previous work should be submitted by the deadline of midnight April 25th 2019 by email to the Habitat Office. Email [email protected]
The End Line Evaluation Report – Mandatory Report Structure. The following structure must be used for the Final End Line Evaluation Report which must be limited to 40 pages (to include or exclude annexes as desired.)
- Executive Summary
- Introduction a. Purpose of the Evaluation b. Organisational Context c. Logic and assumptions of the Evaluation
- Evaluation Methodology a. Evaluation Plan b. Strengths and weaknesses of selected design and research methods c. Summary of problems and issues encountered
- Findings a. Overall Results b. Assessment of accuracy of reported results c. Relevance d. Effectiveness e. Efficiency f. Sustainability g. Impact
- Conclusions (a). Summary of achievements against evaluation questions (b) Overall impact and value for money.
- Lessons Learnt a. Project level – management, design, implementation b. Policy level c. Sector level
Annexes (such as) · End Line Evaluation Terms of Reference · Evaluation Research Schedule · Evaluation Framework · Data Collection · List of People Consulted · List of Supporting Documentary Information · Details of Evaluation Consultant/Team · Table summarising the Findings according to the OECD –DAC criteria (Mandatory) (Reminder – the Final Report must number no more than 40 pages inclusive or exclusive of Annexes)
Application with CVs, testimonials and information on previous work should be submitted. Applications should include: a) A cover letter. b) Technical proposal (max 3 pages) including:
- Brief explanation about the consultant with particular emphasis on previous experience
- Profile of the consultant
- Understanding of the TOR and task to be accomplished
- Draft evaluation framework and plan.
Financial proposal (max 2 pages). The financial proposal should be submitted together with the technical proposal.
Documents to be made available to the successful Consultant
Mid term Evaluation
Log Frame and Revised Log Frame
Reports to Funders