Home-based: Evaluation Team Member Consultant (28 working days between Aug 2023 and Aug 2024) - Europe and Central Asia Regional Office (ECARO)

Contract

This is a Consultancy contract. More about Consultancy contracts.

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.

And we never give up.

For every child, a champion

>> Click here for the full/detailed Terms of Reference: ToR - Evaluation Team Consultant_final (Aug23).pdf

Purpose of Activity/Assignment

To serve as an evaluation team member to conduct an Evaluation of immunization programming at the system-level in Europe and Central Asia region (5 Country Case Studies and Synthesis Report)

Scope of Work

1. Introduction

As part of organizational learning and accountability towards UNICEF’s result areas, ECARO planned an evaluation of immunization programming as part of its Regional Office Management Plan, 2022-2025. This Terms of Reference (ToR) is a multi-country, systems-level evaluation of immunization programming in selected countries of the UNICEF Central Asia and Europe region (ECA). This ToR outlines the context of the evaluation, conditions and requirements for the evaluation as well as its scope, objectives and future use, and the technical requirements that the prospective evaluation team should meet.

2. Evaluation Context

Vaccinations play a critical role in preventing early-childhood infectious diseases and enabling children to grow into healthy adults, and in particular support Sustainable Development Goal 3: “to ensure healthy lives and promote well-being for all at all ages.” Immunization routinely reaches more households than any other health service and brings communities into regular contact with the health system, which provides an effective platform to deliver other primary health care services. Towards that end, UNICEF embeds immunization programming as part of high impact interventions within its Strategic Plan, 2022-2025 and its programmatic goal that every child, including adolescents, survives and thrives, with access to nutritious diets, quality primary health care, nurturing practices and essential supplies. For UNICEF Europe and Central Asia (ECA) region, immunization is a Flagship Result Areas for Thriving.

3. Object of the evaluation

For ECA region, the regional flagship aims specifically to increase (where numbers are low), and sustain (where numbers are adequate), immunization coverage with a systems-strengthening focus to ensure greater likelihood to sustained improvements to the provision, utilization, quality, efficiency of services. As such, UNICEF with its partners support governments to reinforce immunization programmes, as part of health system strengthening, to ensure they are well organized and financed to reach out to every child with life-saving vaccines. These feed into organizational objectives and measured by the following indicator: By 2030, all countries in the region have 95% of children at national level and at least 80% of children in every district vaccinated with DTP/Penta 3.

4. Rationale, purpose and use of evaluation

Rationale: As a flagship priority, the object of the evaluation was identified as part of the UNICEF ECA Regional Office Management Plan, 2022-2025. This is a particularly relevant evaluation given the limited number of health-focused evaluations in the region with only eight since 2017 per UNICEF’s Evaluation and Research Database. With the decreasing rates of immunization coverage, noted disparities within and across countries, and the growing increase in vaccine hesitancy, the evidence emerging from this evaluation will be instrumental to inform the region’s strategic approach to addressing immunization needs at the national and sub-national levels. The evaluation should further interrogate the progress in country with the focus on outcomes and impacts, but also for looking specifically at key common bottlenecks, and exploring where and what types of effective solutions to those bottlenecks have been implemented by national governments including those conducted with support of UNICEF and why they were or were not successful. Therefore, the evaluation will be important for identifying good practices, and sharing experiences on what worked across the region to inform next stages of strengthening immunization programmes.

Purpose: The purpose of the evaluation is to provide a rigorous assessment of national governments’ and UNICEF’s results to date (outcome and impact level) in contributing to the immunization coverage – taking into consideration considerable variability across countries and sectors. This will provide objective assessment of strengths and weaknesses in approaches taken by different countries as well as insights on how to address possible system level bottlenecks. The evaluative focus is on contribution of multi-layered strategies and policies to the observed outcomes and impacts. The evaluation will also be a formative and forward looking, being an important learning opportunity, both for UNICEF and its partners, especially governments in deriving lessons from the experience and existing evidence that can bring attention to the policies and good practices to successfully tackle more than one determinant to improve vaccination coverage.

Use and Users: The primary users of the evaluation will be the UNICEF ECARO Health and SCB sections, UNICEF Country Offices, national health government counterparts, operating national and regional partners in the country. The evaluation will be used to inform programming at country level and help finetune regional technical assistance to countries.

5. Objective of the evaluation

The objective of the evaluation include:

1. To assess the impact of immunization programming in the health care system, looking at both supply and demand, and understand what worked and what did not in improving immunization coverage, especially for the hard-to-reach individuals/communities, how and why;
2. To determine the effectiveness, impact, coherence, relevance and efficiency of immunization programming with a system’s lens;
3. To assess the actual and potential contribution of UNICEF work to the national and sub-national progress (outcome and impact) in immunization coverage, especially the hard-to-reach individuals/communities;
4. To draw lessons and provide recommendations for the refinement and potential scaling up of good practices to further support national governments in their efforts of strengthening immunization programming within the health care system.

6. Scope of the evaluation

Temporal Scope: The evaluation will focus on the last 5 years, period 2018 – 2022.

Thematic Scope: The evaluation will examine the immunization programming in the health care system at national and potentially sub-national level with a focus on UNICEF’s programmatic implications listed under Section 3. Additional attention will be paid to Area 6: demand creation given its increased relevance and attention.

Geographic Scope: The evaluation will be regional for Europe and Central Asia, with five (5) in-depth case studies.

7. Evaluation Questions

This systems-level evaluation would examine the overall contribution to maintain and sustain high vaccination coverage in the region by learning from five country case studies, which will be selected as part of the inception phase of the evaluation. The evaluation would look at factors and approaches that have a positive or negative effect on improving vaccination coverage rates. Key evaluation questions, organized by the OECD-DAC criteria, include:

• Relevance
• Coherence
• Effectiveness
• Efficiency
• Impact

8. Evaluation approach and methodology

The approach and methodology of the evaluation should be guided by UNICEF’s revised Evaluation Policy5, the Evaluation Norms and Standards of the United Nations Evaluation Group (UNEG)6, UNICEF Procedure for Ethical Standards in Research, Evaluations and Data Collection and Analysis7 and UNICEF’s reporting standards. Proposals should set out an approach and methodology for data gathering (primary for five countries and secondary for focus countries and overall in the region). It will also include methodology for data processing and data and evidence analysis allowing theory-based evaluation of impact and outcome. Moreover, applicants are welcome to suggest ideas about how they would approach this assignment in order to complete it as efficiently and timely as possible.

9. Evaluation Process

The evaluation will be structured in the following main phases defined by accompanying activities as described below. It is foreseen that the lead consultant will be required first to lead the inception phase with the strong engagement and support by a member of the RO Evaluation team. The rest of the team will be selected based on the outcome of the inception report to then proceed with the other phases of the evaluation.

INCEPTION PHASE

The inception stage is key in further exploring the feasibility of the appropriate approaches and designs to this evaluation to meet ECARO goals, and consequently identification of country cases (based on objective, purposeful sample criteria), evaluation team profile to meet the requirements of the agreed methodological design.

The inception phase must include but not be limited to the following:

• Initial Briefings
• Desk Review
• Refinement of the evaluation methodology
• An Inception Report (IR)

IMPLEMENTATION AND CONSULTATION STAGE

The evaluation will follow the design agreed in the Inception Report.

• Hybrid data collection
• Data analysis and writing up a country evaluation report (CER) for each focus country
• Synthesis Report

10. Roles and Responsibilities in the Evaluation Process

The Evaluation Team Member Consultant will be an integral part of the evaluation team and outputs. The consultant will work under the stewardship of the Evaluation Team Leader and will be responsible for supporting the overall evaluation and corresponding tasks required to meet the Evaluation Deliverables.

The evaluation team will report to the evaluation Manager and conduct the evaluation by fulfilling the contractual arrangements in line with the TOR, UNEG/OECD norms and standards and Ethical Guidelines; this includes developing of the inception report, drafting and finalizing the final reports and other deliverables, and briefing the commissioner on the progress and key findings and recommendations, as needed. The evaluation team should also adhere to UNICEF’s Evaluation Policy, to UNEG’s ethical guidelines for UN evaluations and to UNICEF Reporting Standards. Evaluation team members will sign a no conflict-of-interest attestation. The evaluation contractor(s) must demonstrate personal and professional integrity during the whole process of the evaluation. The evaluation team members must respect the right of institutions and individuals to provide information in confidence and ensure that sensitive data cannot be traced to its source. Further, the team must respect ethics of research while working with children including using age appropriate consent forms, age appropriate data collection, and principle of do no harm. Furthermore, the team and its members must take care that those involved in the evaluation have an opportunity to examine the statements attributed to them. The evaluation process must be sensitive to beliefs, manners, and customs of the social and cultural environment in which they will work. Especially, contractor(s) must be sensitive to and address issues of protection, discrimination and gender inequality. Furthermore, the evaluation team is not expected to assess the personal performance of individuals and must balance an assessment of management functions with due consideration of this principle.

Work Assignment Overview

The evaluation team will report to the UNICEF Evaluation Manager. All work assignments and outputs under this TOR will be under the leadership and guidance of the Evaluation Team Leader.

Tasks / Milestone******Deliverables / Outputs******Timeline / DeadlineDocument library: Assist in compiling, classifying, and structuring relevant documents for the initial desk review during the inception phaseFinal Inception Report10 working days;

Estimated after 12 weeks (in alignment with the overall evaluation project)

Identify relevant sources of quantitative data on results, especially immunization coverage at the regional and country level, and potential conduct initial analysisContribute initial desk review analysisContribute to the production of the draft Inception reportWork with the team leader to develop templates for the collection of expenditure, activity and results data for use by national consultantsFinal Country Reports (5) and Synthesis Report, and corresponding presentations18 working days;

Estimated within 18 weeks of implementation and consultation stage (CER) (in alignment with the overall evaluation project)

Thereafter, within 14 weeks of implementation and consultation stage (synthesis report) (in alignment with the overall evaluation project)

Data analysis and workshop preparation for the Data Collection and Analysis phaseSupport the overall coordination/organization of the data collection in countriesReview and provide feedback and support on the data / analysis provided by national consultants in the country case study draft reports.Develop graphic presentations of results data and populate graphs and tables with appropriate results data for the ReportsSupport the organization and preparations of Evaluation Reference Group meetings.Support the writing and review of country case study draft reports, and synthesis, as needed.Any other tasks identified by team leader, as needed.Estimated Duration of the Contract

28 working days (between August 2023 and August 2024)

Consultant's Work Place and Official Travel

The assignment will be home-based with no travel foreseen.

Estimated Cost of the Consultancy & Payment Schedule

Payment will be made on submission of an invoice and satisfactory completion of the above-mentioned deliverables. UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if work/outputs are incomplete, not delivered or for failure to meet deadlines. All materials developed will remain the copyright of UNICEF and UNICEF will be free to adapt and modify them in the future.

Please indicate a professional fee (in USD) based on 28 working days to undertake this assignment.

To qualify as an advocate for every child you will have…

• Advanced university degree, preferably in evaluation, public health, social studies or a related field. A first University degree may be considered in lieu of the Advanced degree when combined with relevant academic background and additional two years of relevant work experience.
• At least 5 years of experience in evaluation or research, including experience of evaluating/researching health programmes (immunization is highly preferable, social and behavioral change is advantageous) and familiarity with UNICEF and/or the UN system.
• Experience with qualitative data collection and analysis, literature reviews, syntheses, and other relevant studies.
• Experience working as part of a larger research or evaluation team.
• Experience with data analysis and management software applicable for this project (e.g. NVivo, Dedoose).
• Preferred work experience with UNICEF or other UN agencies, and familiarity with immunization programming, gender, policy dialogue, organizational development.
• Strong analytical skills and statistical data analysis experience.
• Ability to produce content for high standard deliverables in English.

>> Click here for the full/detailed Terms of Reference: ToR - Evaluation Team Consultant_final (Aug23).pdf

For every Child, you demonstrate…

UNICEF’s core values of Care, Respect, Integrity, Trust, Accountability, and Sustainability (CRITAS), and core competencies in Communication, Working with People and Drive for Results.

To view our competency framework, please visit here.

UNICEF is committed to diversity and inclusion within its workforce, and encourages all candidates, irrespective of gender, nationality, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of the organization.

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.

Remarks:

Please include a full CV and a Financial Proposal in your application. Additionally, indicate your availability and your professional fee (in USD) to undertake the terms of reference above. Applications submitted without a professional fee will not be considered. Only shortlisted candidates will be contacted and advance to the next stage of the selection process.

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.

The selected candidate is solely responsible to ensure that the visa (applicable) and health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fully-vaccinated status against SARS-CoV-2 (Covid-19) with a World Health Organization (WHO)-endorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts.

UNICEF offers reasonable accommodation for consultants with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment.