SNV Netherlands Development Organisation
Request for Proposals
Terms of Reference for an External Mid-Term Evaluation of The Sida funded project “Sustainable Integrated Land Management Solutions”
Background
SIDA approved a 3-year grant to SNV Netherland Development Organisation to implement the Sustainable Integrated Land Management Solutions (SILMS) project, in partnership with the Stockholm Environment Institute (SEI), World Agroforestry Centre (ICRAF), Zambia Agricultural Research Institute (ZARI) and the District Councils of Katete and Lundazi (Eastern Province). The Project started on December 1, 2015 and will end on December 31, 2018.
The purpose of the project is to increase incomes for smallholder farmers through the sustainable adoption of ISFM, Agroforestry practices and deforestation free supply chains. The project is built around four objectives which address the major constraints faced by smallholder farmers in Zambia such as lack of access to markets, finance, inputs and extension services:
Objective One: Incentivise and enable small holder farmers to improve productivity using ISFM and Agroforestry practices through increased access to affordable inputs and extension services;
Objective Two: Increase access to markets and finance for small holders to support and sustain their adoption of ISFM and Agroforestry practices;
Objective Three: Mitigate further deforestation and damage to land from uncontrolled agricultural expansion in Eastern Province; and,
Objective Four: Support the development of nationwide frameworks of practice that will enable wide spread adoption of ISFM and agroforestry practices and replication of the deforestation free supply chain model to other commodities.
PURPOSE AND USE OF THE EVALUATION
By June 2017 the project would have reached the half way point, therefore, a mid-term evaluation is being commissioned to assess the project progress toward achieving its objectives. The evaluation will analyse the SILMS project achievements to date in areas related to its performance, including relevance to identified needs; ability to achieve critical results; efficiency in achieving those results; and steps made towards sustainability. As part of this overarching purpose, the evaluation should also analyse gender implications so that gender equity can be achieved in the SILMS interventions.
The study should identify lessons learned and provide concrete recommendations for the refinement of the project approach, if necessary, and to inform subsequent implementation phases of the project. The evaluation should point out areas that require greater attention in future programming.
EVALUATION QUESTIONS
The following key questions will guide the mid-term project evaluation:
1. To what extent is the project on track to achieve its objectives and outcomes? What progress has been made so far?
In order to address the above question the Consultant must assess whether the implementation is on track to achieving the planned results. The assessment must focus on quality and quantity of outputs for the activity. Assess the factors that are facilitating or inhibiting the achievement of these results. Specific attention should be made to increased productivity and increased incomes of farmers and agro-dealers. Recommendations for improvement should be provided.
2. To what extent is the project design, implementation, and management effective and why?
In order to address the above question the Consultant must assess the best practices and lessons learned during project design, implementation, and management. The Consultant should indicate any changes that occurred during implementation of the project, both the external environment and or factors internal to the project, in the evaluation report especially where identified changes may have had a bearing on project outputs and outcomes. The consultant should also assess the effectiveness of the project partnership in carrying out their respective tasks. Recommendations for improvement should be provided.
3. What progress has been made towards ensuring the sustainability of SILMS approaches?
In order to address the above question the Consultant must assess the actions and processes put in place and the influence of the project outcomes and lessons on development and natural resources management strategies, programming, and policy at the national, sub-national and community levels. Assess the likelihood of adoption of SILMS approaches by external stakeholders such as government counterparts, companies or other organizations. Assess the evidence of organizational and technical capacity built among the government extension officers, communities. Assess (if any) the observed evidence that SILMS models will continue to be implemented beyond the life of the project. Recommendations for improvement should be provided.
4. To what extent are the monitoring and evaluation activities serving to inform the planning, implementation and monitoring of the project?
In order to address the above question the Consultant must assess the extent to which project monitoring information was used in programming for results and decision-making. Assess the strategies used to disseminate strategic information by the project and the demand for and use of this information at different levels (provincial/district/community). Analyse the extent to which the monitoring and evaluation system was utilized for program improvement. In addition, the consultant should also assess if the information generated in SILMS project is the kind the stakeholders (agriculture officials, etc.) require to enable them to manage more effectively.
EVALUATION DESIGN AND METHODOLOGY
4.1 Evaluation Design
The evaluation will be carried out in Katete and Lundazi districts of Zambia by an independent evaluation team using a combination of qualitative and quantitative methods. Given that a control or comparison group was not established at the beginning of the project, SNV anticipates use of a non-experimental evaluation design for this evaluation. However, a Baseline survey was conducted and the Consultant should endeavour to make comparisons to assess the immediate results observed. The Consultant is required to come up with creative ways to assess the effectiveness of interventions, which may require reconstructing the baseline and designing new tools to assess in a more rigorous way the outcomes of the SILMS interventions. Consultant is required to elaborate a detailed evaluation design and methodology as part of their Work plan. The evaluation design document should include a series of data collection instruments.
4.2 Data Collection Methodology
The project implements a participatory monitoring system based on an M&E Framework developed at the beginning of the project and the corresponding Performance monitoring plan (PMP) to collect data against key outcome indicators. The evaluation methodology is expected to review this data and, as far as possible, allow comparability taking into account any issues around data collection for the first half of the project.
The consultant is expected to employ a variety of data collection and analysis techniques for both quantitative and qualitative data to ensure a comprehensive evaluation exercise. This will likely include, at a minimum:
Document and systems review: Review of existing documentation, including; project reports, project results framework and monitoring and evaluation data.
Surveys: Application of structured survey questionnaires with a representative, random sample of target population to quantitatively assess outcomes. This will be in greater scope and depth compared to baseline survey and standard routine project monitoring.
Focus Group Discussions: With target groups and other stakeholders to assess implementation experiences and effectiveness, document successes, challenges and lessons learned, and develop recommendations for improvement.
Key Informant Interviews: Consultations with key project stakeholders, including field staff and partners. Guidance on appropriate stakeholders to engage during the exercise will be provided by field and Country Office staff.
Project beneficiary Interviews. Consultations with the principle project beneficiaries, mainly: lead farmers, follower farmers, tree nursery farmers, seed multipliers, agro dealers/soil testers.
4.3 Data Analysis
Prior to the start of data collection, the evaluation team shall develop and present, for SNV review and approval, a data analysis plan that details how: (1) qualitative data such as key informant, stakeholder, and beneficiary interviews and/or focus group discussions will be transcribed and analysed; (2) quantitative data will be analysed and presented to determine trends over time, including dummy tables; and (3) the evaluation will weigh and integrate qualitative data from these sources with data from project capacity assessments, service delivery data, and project monitoring records to reach conclusions and recommendations. Whenever possible, data will be disaggregated and analysed by gender. Data collection and analysis shall also emphasize utilization of routine monitoring data for program improvement.
TIME FRAME: 25 calendar days
3 working days – Desk review and meeting with Project stakeholder in Lusaka and field offices.
14 days (including travel days and weekend while in the field) – Training of research assistants, tools testing and data collection in the field.
5 working days – Organising, analysing data, writing the draft report and submission of a draft report to SNV for comments.
One-day Meeting for presenting draft report to project team
2 working days writing final report incorporating feedback from project team and submission final evaluation report
DELIVERABLES
All written documents are to be submitted in English using Microsoft Word in both soft and hard copy. The main body of all reports should be written in simple, non-technical language (i.e. plain English), with any technical material to be presented in annexes. All primary data collected and analysis conducted for the purpose of the evaluation will remain the property of SNV and must be submitted electronically and in a clear and comprehensible format in Excel.
The evaluation coincides with the agricultural marketing season and should begin no later than the July 15, 2017, with the evaluator(s) expected to take a total of 25 days from the day of contracting to complete the assignment.
The consultant will provide the following deliverables to the SNV within the timeframe stated:
1. Inception Report: within 3 working days of evaluation launch, a detailed report on the evaluator’s proposed final evaluation design and methodology will be submitted to SNV for approval. This will provide preliminary understandings based on document review, rationale and a detailed description of the methodology and tools, research questions, analytical methods, budget with a breakdown of costs and detailed work plan for the entire exercise. Any draft questionnaires or interview forms will also be submitted for review at this stage.
2. Interview Notes and List of Resource Documents: The Evaluation Team shall provide SNV summaries of all key meetings, workshops, and discussions conducted during the course of the evaluation and copies of any relevant documents and reports gathered during the evaluation.
3. Summary Presentation of Findings to SNV and Stakeholders: The evaluation team shall present initial findings to SNV for review, comment and feedback. A PowerPoint presentation and handout (maximum of two pages) shall be prepared for the presentation through a workshop. The team shall consider SNV and stakeholder comments and revise the draft report as appropriate.
4. Evaluation Report: A draft evaluation report is due five business days after the field visit is completed. Within 10 business days of receiving SNV’s feedback to the draft report, two hard copies and one electronic (MS Word) copy of the final evaluation report are due to SNV. The final evaluation report should not exceed 50 pages excluding annexes
The evaluation report shall include the following:
Executive Summary (3 – 4 pages)
Background;
Introduction;
Methodology;
Findings, including Lessons Learned;
Recommendations (1-2 pages);
Conclusions (1 -2 pages) and
Annexes, including:
Scope of Work
Data collection tools
Key data sets, including interview transcripts
List of key informants
Documents consulted
MANAGEMENT AND SUPERVISION RESPONSIBILITIES
The consultant will report directly to the SNV Zambia Country Director (CD) and the SILMS Project Manager. However, s/he will also be expected to work closely with the field staff. Further details is provided in the table below.
SNV Zambia shall also provide the following to the consultant;
All relevant project documents
Logistical arrangements for all field travel;
Copies of all key background resources identified;
Field staff time to assist with data collection;
Introductory meetings with key government staff;
Project partners
Stockholm Environment Institute (SEI),
World Agroforestry Centre (ICRAF)
Zambia Agriculture Research Institute (ZARI)
Project Location
Katete and Lundazi Districts of the Eastern province, Zambia
Project time frame
1st December 2015 to 31st December 2018
Overall supervision of the evaluation exercise
Ms Marjon Tuinsma
Country Director
[email protected]
Operational and Technical supervision
Mr Arend van der Goes
Project Manager
[email protected]
Evaluation Manager
Mr Kennedy Chipampe
Monitoring and Evaluation Manager
[email protected]
Field supervision – Katete
Mr Enock Mashikinyi
Agribusiness Project Officer
[email protected]
Field supervision – Lundazi
Mr Zacchaeus Mtonga
Natural Resources Management Officer
EVALUATION TEAM QUALIFICATIONS
The evaluation is to be done by a local consultant.
The lead consultant should have wide knowledge and experience of the Zambian agriculture and natural resources sector, and good understanding of the set project objectives. Should have a minimum of a Master’s degree in Agriculture, Agricultural Economics, Agribusiness, Rural Development or a related field, and at least have a minimum of 7 years’ experience working in agriculture or related fields and in carrying out impact evaluations, demonstrable academic and practical experience in qualitative and quantitative research methodology, evaluation design and implementation. The evaluator should also have strong analytical, facilitation and communication skills (both in written and oral) and demonstrated experience in rural areas in Zambia.
Interested evaluators or firms are requested to submit:
An Expression of Interest detailing their interpretation of the TOR, proposed methodology including sampling framework, work schedule and proposed budget;
A capability statement demonstrating how they meet the required qualifications and competencies;
Copies of all relevant Curriculum Vitae (CVs). Only CVs for the specific individuals that will form the proposed evaluation team should be included;
Two references (including one from your last client/employer).
Submission of proposal:
All submissions should be sent via:
Email to: [email protected] and [email protected] the subject line should read ”Mid-Term Evaluation -Sustainable Integrated Land Management Solutions”
Postal: Procurement Office, SNV Netherlands Development Organisation
7, Nkanchibaya Road, Rhodes Park, Lusaka, Zambia
Submissions are to be made available to SNV no later than 2nd July 2017.