Terms of Reference for Specsavers Final Evaluation

Download the ToR here

Vision Action-Specsavers Phase 6 program evaluation summary

• Program/Project: Embedding sustainable eye care services in Zambia
• Partner Organisation: Ministry of Health and Ministry of Education (Zambia)
• Project start and end dates: 1st March 2020 to 31st March 2023 (3 years)

 

Evaluation Purpose:

1. To evaluate the program’s performance by assessing its outputs against set targets.
2. To determine the motivating and inhibiting factors to the success of the project.
3. To determine the impact of eye health management (spectacles or/and eye medications) on the affected target populations.
4. To make recommendations for the future expansion (replication or scale-up) and sustainability of the programme.

• Evaluation Type: Endline evaluation
• Proposed Evaluation Start and End Dates: 7thth March 2023 to 24th April 2023
• Anticipated Evaluation Report Release Date: 31st April 2023

I. BACKGROUND OF PROJECT 

Program Description

Uncorrected refractive error has been acknowledged as one of the major causes to be addressed in the fight to address preventable visual impairment and blindness. Therefore, with support from Specsavers, Vision Action (VA) (formerly Vision Aid Overseas) has been partnering with the Zambian government to develop optometry services within the country. In 2020 VA embarked on a program to further extend the scale and scope of sustainable eye health services in Zambia, to address the main cause of visual impairment, Uncorrected Refractive Error (URE). This programme was developed in consultation with partners at the Ministry of Health and Ministry of Education, and the beneficiaries in which VA aimed to serve. The three-year programme focused on consolidating the work that has been undertaken in the previous years to fully embed it into the health system.  The program proposed to:

1. Integrate primary eyecare into primary healthcare in Zambia, through the adoption of the WHO-AFRO Primary eyecare guidelines.
2. Integrate school eye health into a national school health plan.
3. Improve the capability of the eye health workforce, in refraction and low vision.
4. Complete the integration of the Vision Centres into the national health system.

In line with the above objectives, VA embarked on achieving the set objectives by implementing the following activities:

• Introduction of Primary Eye Care Services in 30 facilities in Mazabuka district of southern province.
• Implementation of the School Eye health Program in 41 schools in Kabwe District of Central province where over 25,000 learners were screened for eye health.
• Training of six Master Trainers in Low vision who would train 12 health care staff in low vision from different Districts of Zambia.
• Continue supporting 11 Vision Centres from 10 provinces of Zambia.

II. EVALUATION OBJECTIVE, SCOPE, AND INTENDED USE

The general objective of the final evaluation is to:

1. To evaluate the program’s performance by assessing its outputs against set targets
2. To determine the motivating and inhibiting factors to the success of the project
3. To determine the impact of eye health management (spectacles or/and eye medications) on the affected children
4. To make recommendations for the future expansion (replication or scale-up) and sustainability of the programmes

2.1        Target audience and intended use.
The evaluation will serve VA and the partner organisation Specsavers as an important tool for learning, creating knowledge, and demonstrating accountability and transparency. Assessing project management and performance also helps VA and Specsavers to have a basis for potential follow-up on the Scale-Up of sustainable eye health services within Zambia. The evaluation will involve a wide range of stakeholders through a participatory approach. The evaluation results will be shared with these stakeholders to contribute to organizational learning.

2.2 EVALUATION QUESTIONS

1) Relevance/Quality

• How far has the project responded to the needs and priorities of the target groups as identified in the project plans?
• To what extent have the project’s planned activities and expected outputs been consistent with the overall goals and the attainment of their objectives?
• To what extent has the project approaches and activities been appropriate to the local context?
• How have the working relationships, communication and coordination between stakeholders been managed?
• Has the project design and implementation considered other sectoral interventions in the area?
•  What are the views of the target group on the project, and what was the extent of their involvement in their implementation?

2) Effectiveness/Equity

• How effective has the project been as a disability-preventative and public health interventions in the districts of implementation in Zambia? What gaps/ opportunities exist?
• Are the objectives, approaches, and project design coherent and complementary?
• To what extent have planned targets, objectives and activities been delivered? In case of targets, objectives and activities were not successfully implemented, what is/are the reason (s)?
• How effectively did the project reach persons in need? How inclusive was the project benefiting persons in need in all service areas?
• How well did target populations participate in project decision-making processes?
• Has the awareness and understanding of eye health increased amongst partners (MoE, specifically teachers trained)? Has this occurred in the community? What can be learned about the awareness initiatives undertaken? Were some more successful than others?
• Have project implementors (trained teachers and health staff) received knowledge and training on the specific requirements of school learners living with permanent eye conditions? How are they using that knowledge? Did their behaviour change?
• Do women and girls, men and boys have equal access to the project activities, and do they benefit on an equal basis?
• What needs did each of those groups have in accessing the services offered through the project?
• Has the project considered those different needs and addressed them in its activities?
• Are different gender and age groups accessing eye health interventions equally? If not, what are the future recommendations for addressing the barriers?

3) Efficiency and accountability

• How efficiently has the project been implemented as compared with alternatives? Were the project structures adequate, and what adjustments are recommended if it was inefficient?
• How well have the project’s human, financial and technical resources been managed regarding transparency and accountability?
• What processes have been implemented to ensure transparency and accountability between Specsavers and VA?
• How is monitoring data being collected and stored? Is the data used to inform relevant platforms (Health, Education, etc.)?
• Are existing learning processes such as reflection and internal review used sufficiently to keep the project plan updated?

4) Impact

• To what extent did the project contribute to preventing, treating, and managing eye health problems?
• What proportion of the target groups have benefited from the project interventions, and in what meaningful ways have their lives changed? Did the project improve the quality of life of the target group?
• What are, if any, negative or unintended outcomes on the target groups from the project?
• How effective has the project been in creating changed behaviour regarding face washing, wearing prescribed spectacles, and seeking eye health services?
• What broader change in the health service provision and personnel can be seen as a result of the project?

5) Sustainability

• What is the sense of community ownership of the project? What changed for the persons with disabilities: lasting change in perception by the community and health service providers?
• To what extent has the project strengthened the capacities of local government and district-level stakeholders? Can they sustain the project’s achievements after the end of external support? If so, how?
• What major factors will influence the project’s achievement of, or lack of, sustainability?
• Is there any environmental impact of the project’s actions?
• What role will the government play in maintaining the project’s momentum?

6) Ethics and Child Safeguarding

• To what degree do the teachers and health personnel understand the child safeguarding standards?
• To what degree did VA and other implementers uphold ethics and child safeguarding, was the process efficient and effective in protecting potential beneficiaries?
• How has a safe environment for children been established and maintained throughout the project’s planning and implementation stages?
• Are any activities related to the project that have created unforeseen negative impacts on children’s safety?

III. METHODOLOGY

A participatory approach will be adopted for the process of this final evaluation. This evaluation will require using multiple data collection methods, combining qualitative and quantitative methodologies, being conducted in close collaboration with VA and other stakeholders (e.g. Ministry of Health and Ministry of Education).

IV. LIMITATIONS

• Some project sites may be cut off due to rain.
• Some beneficiaries may be unavailable due to school term breaks/holidays, transfers, and graduations.

V. EVALUATION TEAM AND MANAGEMENT RESPONSIBILITIES

5.1 Commissioning Responsibility

The Final Evaluation will be commissioned by VA and Specsavers. Their responsibilities will include the following:

• Planning the evaluation with all stakeholders
• Commissioning the evaluation
• Approval of final evaluation report
• Dissemination of evaluation results
• Provision of HR support
• Facilitation of communication with local stakeholders

5.2 Evaluation Team

The Evaluation Team’s responsibilities will include.

Major Tasks:

• Lead the evaluation preparation and implementation work with the support of VA Zambia
• Prepare detailed technical and financial proposals, and submit them to VA
• Develop assessment tools, methodologies and other works related to the assessment and revise them after comments from VA
• Conduct site visits/field observation.
• Conduct interviews and focus group discussions.
• Prepare a detailed evaluation report.
• Prepare and participate in meetings and debriefings during the process.

The team needs to have the following qualifications:

• Advanced degree in Public Health, Medicine, Development, or other relevant fields.
•  At least 5 years of work experience in this field.
• Knowledge of health sector policies and health systems in Zambia
• Substantial experience in conducting programme evaluations of similar nature.
• Experience conducting population-based surveys.
• Knowledge and experience in policy analysis, health sector planning and development.
• Knowledge and experience in strengthening partnerships and aid effectiveness, experience with donor coordination.
• Strong interpersonal, negotiation and communication skills.

5.3 Management of the evaluation and logistics

The VA team will support access to all relevant documents, including planning documents, reports, results framework and assessment reports. Furthermore, VA will provide administrative and office logistics and support in scheduling the site visits in coordination with MoH and MoGE.

5.4 Expected Results

1)  Inception Phase: Desk Review and presentation of the work plan and evaluation methodology (2 days) until 21st March 2023.
The consultants will prepare a work plan including their understanding of the issues addressed in the evaluation and the evaluation methodology in English. This will be presented to VA Field visits (10 days from inception)

The consultants will conduct fieldwork at the project site and hold meetings with key resource persons to collect information per the requirements in the approved evaluation work plan. Stakeholders in the field are to be briefed on arrival and debriefed after the review.

3) Data analysis and presentation of preliminary findings (5 days after field visits)

The consultants will conduct an in-depth analysis of the data gathered during the inception and field phases. After the data analysis is finalized, an online presentation for VA should be organized to present and validate preliminary findings.

4) Final evaluation Report (5 days after data analysis)

The consultants will prepare a report describing the evaluation process, findings, learnings, and recommendations. A draft report will be submitted 3 days after data analysis (for distribution to the implementing partner) to VA for feedback. Comments will be incorporated in the final report by the consultant team. The final report is to be submitted electronically, not later than 7 days after receipt of comments on the draft evaluation report.

5.5 Deliverables:

The consultants will prepare the following:

• A detailed methodology and work plan
• An Inception meeting and report
• A briefing workshop and presentation on preliminary findings for VA
• A final evaluation report in English (25 to 30 pages max)

The stated dates are indicative and will be discussed during the inception phase.

VI APPLICATIONS

The consultant/s shall provide an offer (Expression of Interest) including the following:

• Description of the consultant, including CVs of the suggested evaluators.
• A detailed approach and methodology for achieving the goal of the assignment.
• A detailed schedule including steps to be taken and responsibilities of team members.
• Statement of availability for conducting the assignment.
• Financial offers, including professional fees, taxes, and incidentals.

The commissioning entity may require proof of similar evaluation experience in the form of previous reports and/or references during the selection process.

Selection of the candidate will consider the above criteria/requirements and the financial offer. VA reserves the right to end the contract if the agreed-upon and contracted team members are unavailable for the evaluation and if no adequate replacement is provided.

The offer shall be sent in electronic format to [email protected]