USAID Outcomes Evaluation Consultant

Job Description

Organizational and program background

The International Rescue Committee (IRC) responds to the world’s worst humanitarian crises and helps people whose lives and livelihoods are shattered by conflict and disaster to survive, recover, and regain control of their future. The IRC Yemen began its programs in October 2012 and currently operates in the south and north of the country through direct programming and local partnerships.

The country program operates in Aden, Abyan, Lahj, Al Dhale’e and Shabwa, and Sana'a governorates and continues to assist the affected population through:

• Provision of health, reproductive health, nutrition, water and sanitation services to more than a quarter-million people.

• Delivering essential drugs and medical supplies to hospitals.

• Improving access to education for out-of-school children.

The IRC is implementing a three-year program, Improving Access to Quality Education in Yemen (IAQY). The program's goal is to restore education opportunities for out-of-school children (OOSC) in Al Dhale’e governorate of Yemen.

The program also intends to increase access to safe and quality education through improved service delivery at all levels and expanded equal access to education that meets the needs of crisis-affected OOSC. It will also enhance the quality of teaching and learning, increase the wellbeing of children and teachers, provide teacher training, ongoing school-based support, and professional development, and raise the knowledge, skills, and abilities for inclusive education of local and education district government officials, caregivers, and community members. By the end of year 3, the program will have reached 14,400 OOSC aged between six and 14 years and bring the benefits of improved community and parental engagement and safer school environments. The program will also support and train 630 Father and Mother Councils (FMCs) members in schools and 830 teachers (500 MOE: 330 IRC supported).

Below are the four intermediate results (IR) that the program intends to achieve:

- IR1: Access to safe quality education is expanded. - IR2: Teachers are trained and supported. - IR3: Children are provided with effective psychosocial support. - IR4: Education service delivery is ensured and strengthened.

Rationale and Purpose

The overall objective of the evaluation is to assess the extent to which the program impacted the school children, teachers, and the whole education system in Al Dhale'e governorate and derive lessons learned and key recommendations from the findings.

Objectives of the Evaluation

The main purpose of the evaluation is to assess the impact, relevance, sustainability, effectiveness, and efficiency of the program. The report will present achievements against project targets and compare the endline data findings against the baseline data findings. The information and recommendations from the evaluation will inform both the IRC and USAID of the existing gaps and possibly better program design and implementation. As such, the Consultant is expected to provide information about factors for success and recommendations for areas of improvement.

Specifically, the evaluation should be able to:

- Assess the impact of the program since its inception in 2018 and the validity of the theory of change. - Identify lessons learned, good practices and provide concrete recommendations. - Assess and identify critical internal and external factors that have contributed, affected, or impeded the program's achievements and how IRC has managed these factors. - Determine the intended and unintended outcomes of the program and provide recommendations on how the unintended negative outcomes of the program can be addressed. - To assess the impact of COVID-19 on the achievement of the program goal and outcomes.

Evaluation Criteria and Questions

The evaluation should explore the program's relevance, efficiency, effectiveness, sustainability and impact. The following questions, among others, should be addressed under each criterion:

Relevance/ Appropriateness

- How appropriate and relevant is the current program design related to Yemen's micro and macro socio-cultural, political, and economic context? - To what degree did the program address the needs and priorities of the Education sector? How can the program better address the educational needs of learners in schools, Community based centers (CBCs) and Homebased learning (HBL)? - How appropriate were IRC’s Clients Responsiveness and Accountability (CRA) mechanisms to the context and the identified needs? Were feedback mechanisms appropriate and accessible for all the school children, teachers and other stakeholders?

Effectiveness

- What program outcomes have been achieved as detailed in the program proposal? - To what extent were the expected outputs of the program achieved? - What program design, implementation and monitoring elements can be added or amended to improve the effectiveness of the program? - What were the major internal and external factors influencing the achievement or non-achievement of the program goal and outcomes? - How effective was the Third-Party Monitoring activities in improving program quality and implementation? - Is there a proper monitoring system in place, which is organized according to the logical framework and its indicators?

Efficiency

- Did the program demonstrate a good Value for Money (VfM) approach? - To what extent has the program exploited synergies among key humanitarian stakeholders and their respective competencies? - How efficient were the management structures and the implementation modalities in terms of timeliness of delivery and cost-effectiveness of the interventions? - What are the barriers to timely and quality implementation of the program? How can those barriers be addressed?

Sustainability

- What are the possible factors that enhance or inhibit sustainability, including economic/financial, institutional, technical, socio-cultural, and environmental sustainability aspects? - What other factors can affect the sustainability of the program? Are the activities implemented sustainably in the long run? What can be done to enhance the sustainability of these activities?

- To what extent would the program's benefits continue after the cessation of the program? - Impact - What were the positive and negative, intended and unintended, changes produced by the program? - Critically analyze the program's contribution to any observed impact (intended, unintended, positive, and negative). - What difference has the intervention made to the children in schools, CBCs and HBL, and program stakeholders?

Scope of Work:

The Consultant (s) will design an appropriate evaluation methodology based on their understanding of the expectations of the terms of reference. Depending on the security and Covid-19 situation, the Consultant should come to Yemen to undertake the data collection exercise. Primary data collection will be conducted in Al Dhale’e governorate. The Consultant should propose and adopt methodologies that combine qualitative and quantitative research techniques, including extensive desk review and research. The Consultant is expected to propose his/her methods for the evaluation that should include but not limited to:

- Drafting data collection tools - Developing the data collection methodology, including sampling. - Developing the quantitative and qualitative data analysis plan - Production of the evaluation report

All data collected must be disaggregated according to the USAID requirements that include but are not limited to sex, age, disability and location. The Consultant should submit a detailed action plan that includes data collection instruments, clear roles and responsibilities, timeframe, data analysis process, and report writing.

Intended Users

The intended users for the evaluation will be:

- The IAQY program team based in Yemen. - All interested parties in IRC including the Senior Management, MEAL Department, Grants Management and Education Technical Advisors. - Representatives of USAID as the donor that has funded the project.

Evaluation Roles and Responsibilities

IRC will:

- Provide logistical and accommodation support to the Consultant while in Yemen. - Provide ongoing security advice and support when traveling to the field. - Facilitate engagement with the schoolteachers, education staff, and other key stakeholders. - Provide all necessary program documents and contacts of relevant stakeholders. - Review the Consultant's proposal, tools and evaluation report.

The Consultant will:

- Be responsible for all aspects of the entire evaluation process, including travel and permit applications, evaluation preparation, data collection, analysis, and report writing. - Be responsible for paying any tax or other fees related to this assignment. - Be responsible for their working tools such as computer and data analysis software.

Reporting Arrangements

The Consultant will report to the MEAL Coordinator but with close collaboration with the Technical Coordinators and the Deputy Director Programs.

Duration of assignment

The consultancy should not last more than 40 working days, excluding weekends.

Deliverables:

The Consultant shall be expected to produce an inception report upon commencement of the assignment.

The Inception Report will detail the agreed methodologies to be employed during the evaluation. The report should also include the finalized activity plan and a structural outline of the final evaluation report and should be shared and approved by IRC before the data collection and analysis commencement. The Inception report should also contain:

- A detailed methodology for the evaluation implementation. - A detailed schedule for the program review. - The indicators that fall within the scope of the program review. - Draft data collection tools (qualitative and quantitative). - A work plan that sets out the preparatory activities, specific deliverables, and timeline related to the program review and budget for the data collection activities.

Evaluation Report: The report should address the above consultancy objectives and contain an executive summary, acknowledgments, introduction including program summary and purpose of the evaluation, a detailed methodology (including limitations); key findings (covering both document review and primary research), lessons learned, evidence-based recommendations, conclusion and annexes. Annexes should include, at a minimum: field sites assessed and a list of key informants. A soft copy of the report should be shared with the MEAL Coordinator, and the report should not be more than 30 pages, excluding the cover page and annexes.

A summarized four pages evaluation report that summarizes the evaluation purpose and background, evaluation questions, findings, lessons learned, conclusion and recommendation).

A PowerPoint presentation outlines the evaluation process, key findings, lessons learned, and key recommendations.

The deliverables above will be accompanied by regular communication and feedback from IRC Yemen and a validation workshop where the evaluations’ preliminary results will be presented to the IRC Yemen team. After incorporating comments from the validation workshop, the report should be shared with the Deputy Director Programs, MEAL Coordinator, and Education Coordinator.

Qualifications

Requirements:

Please note that this consultancy is only open to individual(s) consultants and not companies or other evaluation firms.

- A qualification in Monitoring and Evaluation, Research, Statistics, or any other related field is required. A qualification in Education, Social Sciences or other related field is also strongly required. - Experience in designing, planning, and conducting impact evaluation using experimental or quasi-experimental design approaches. - Knowledge and experience in conducting research with children and the education sector. - Significant experience in carrying out evaluations in Yemen or similar contexts. - Excellent skills in quantitative and qualitative data collection and analysis. - Excellent skills in data analysis software such as SPSS or STATA (for quantitative data analysis) and ATLAS.ti, NVivo for qualitative data analysis. - Experience in using mobile data collection systems such as KoBoCollect or Commcare will be an added advantage. - Excellent organizing, facilitating, presentation, and communication skills, including good report writing in English. - Ability to communicate in English required, including advanced English writing abilities.

Application Process:

Interested applicants who meet the required profile are invited to submit an application in English. The application package should include:

The Consultant CV with details of qualifications and relevant experience (Maximum 5 pages)

- The technical proposal that summarizes understanding of the ToR, methodology, and expected outputs. The Technical Proposal should contain; a complete description and explanation of the proposed methodology for the assignment, work plan, timeline, and any other resources that the Consultant will make available to execute the task and achieve. - Financial proposal showing the evaluation costs. - Applications that fail to include one or more of these elements will not be considered.

Payment rate and schedule will be determined between IRC and the selected consultant based on the financial proposal.