Consultancies: Early Childhood Development (ECD) and Early Childhood Education (ECE), Two Researchers, 9 months, Evaluation Office, NYHQ

Contract

This is a Consultancy contract. More about Consultancy contracts.

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.

And we never give up.

For every child, evaluate

The Evaluation Office seeks to issue a contract to eligible consultants to conduct an evaluation of UNICEF work in early childhood development (ECD) and early childhood education (ECE). The evaluation aims to interrogate whether UNICEF-supported ECD/ECE programmes for early stimulation and learning do achieve intended developmental and school readiness outcomes. The evaluation will assess whether there is clarity in UNICEF country offices about what constitutes inclusive and effective learning programmes for caregivers and children in the early years - from birth to the pre-school year (pre-Grade 1 year). It will also interrogate, with the aid of a draft theory of change, whether UNICEF-supported programmes prepare ECD/ECE pre-school centers and schools seek out and engage with the most vulnerable populations of young learners and their caregivers. More importantly, the evaluation will assess whether the choice of an integrated ECD approach has resulted in the intended cohesiveness, efficiencies, and complementarities within UNICEF goal areas, and between UNICEF and partners in the ECD/ECE subsector, to provide holistic developmental support to young children and their families.

This evaluation will focus on ECD and ECE programmes that address the policy development, system strengthening and service delivery objectives, and contribute to children’s developmental readiness and school readiness for children from 0 to 8 years.[1] The five components of the Nurturing Care Framework will be used to interrogate the content of the programmes, and to assess whether there is clarity in UNICEF country offices about what constitutes an inclusive and effective early learning programmes for caregivers and children to achieve the intended developmental outcomes. On the other hand, the evaluation will look to the Build to Last framework for the ‘how to’ – i.e., to tease out the role of the education sector and key components that are necessary to prepare ECD/ECE centers and schools to engage young learners and their caregivers. The evaluation draft theory of change, provided in Annex 2, attempts to bring these two aspects together. However, more inputs will be required to update the draft theory of change and to provide an exhaustive catalogue of UNICEF-supported programmes, interventions, and/or activities during the inception phase, while the data collection phase will be used to update the theory of change further against the workplans from the countries that will be selected to participate in the evaluation, and to validate it for use in making evaluative judgments.

While ECD was previously viewed as one of many interventions, programmes, or even as a sub-sector in many organizations, there is now wider recognition of early childhood development as the intended outcome of the work that governments, national stakeholders, UNICEF, and the development community do in the ECD and the ECE sub-sector. Put differently, positioning children to reach their developmental potential in the early years is identified as a key ingredient for enhancing the achievement of almost every child outcome that we can think about.

Second, notable developments have taken place, key of which was the increased visibility of ECD due to being championed by the Executive Director during the period of UNICEF Strategic Plan (2018-2021). This had a positive effect on the early learning agenda. Also, UNICEF has since articulated significant conceptual frameworks around which the ECD and ECE work is organized, namely the Nurturing Care Framework in 2018 and the launching in 2019 of the Build to Last framework, to support governments in strengthening systems to provide pre-primary education. At best, an examination of these investments is necessary, both through formative assessments of the more recent commitments, and summative assessments of the work that was done from 2018 (underpinned by approved strategic plans, results, operational modalities and staff structures) for accountability purposes.

Third, despite the bold commitment to SDG 4 and associated targets (SDG 4.2) the requisite action of increasing the share of education spending that goes to pre-primary education has not been easy to undertake. If anything, emerging evidence suggests that the ECD and the ECE sub-sector experienced setbacks during the COVID-19 pandemic as governments prioritized higher grades and completing classes in providing access to digital learning or mitigating learning loss, at the expense of children in the earlier years of primary school. Relatedly, with the decline in ECE expenditure from 7 percent of the total education expenditure in 2018, to 4.2 percent in 2021, UNICEF seems to be regressing on its commitment to dedicate 10 percent of the education resources to early childhood education. Consequently, the organization needs to re-examine this and other commitments to determine, through evaluative evidence, what adjustments and/or actions are necessary to meet those commitments, and to create value for the sub-sector through its investments in ECD/ECE.

Fourth, while having benefitted from the evaluation of the Early Learning Development Standards (ELDS) initiative conducted in 2016 and the more recent evaluation of early stimulation and care in 2021, the need for evidence-based decision-making has been highlighted in the past three strategic planning periods. As such, the evaluation will help determine if any changes are needed to increase the capacities of UNICEF and partners to deliver on their objectives and commitments to inform policy and programme management decisions. At the corporate policy level, the evaluation will position the organisation to inform mid-term reviews of the global education strategy - Every Child Learns, UNICEF Education Strategy, 2019-2030, and UNICEF Strategic Plan (2022-2025).

How can you make a difference?

EO seeks to issue a contract to eligible consultants to be part of a 5-person core team that will conduct an evaluation of UNICEF work in early childhood development (ECD) and early childhood education (ECE). You are invited to apply, to join the evaluation team as one of two researchers.

The purpose of the evaluation is twofold - to enhance organizational learning and to strengthen organizational accountability. On the learning side, the evaluation will interrogate whether UNICEF has leveraged the organization’s investments on the integrated ECD approach and used its knowledge resources and experiences to support countries in strengthening their ECD and ECE programmes. The evaluation will also advance the accountability objectives of the organization by interrogating and/or verifying UNICEF achievement of reported results, aimed to increase access to early childhood education opportunities and the care that supports learning. Evidence from the evaluation will be used to facilitate decision-making, advocacy, and resource mobilization and to contribute to global learning within the ECD and the ECE sub-sector.

The evaluation will pursue three specific objectives, namely:

1. Assess UNICEF work in support of national policy reform, financing strategies, and budgeting to expand access to opportunities for early stimulation and learning for children from 0-8 years old, including children in the pre-school grade, and for strengthening the ECE sub-sector in countries of different typologies.
2. Examine the efficacy of UNICEF integrated ECD approach and the adequacy of available capacities (core skills, tools, processes, partnerships, coordination mechanisms, and institutional arrangements) to support countries of different typologies in the adaption, acceleration, and scale-up efforts for providing early stimulation and learning support and services for different profiles of children from 0-8 years old, including children in the pre-school grade.
3. Evaluate UNICEF inputs and outcomes in supporting countries of different typologies to develop effective programmes to increase access to opportunities for early stimulation and learning for children from 0-8 years old, including children in the pre-school grade (the grade before Primary 1) to achieve universal access to pre-primary (SDG 4.2), and to promote school readiness among children and caregivers.

The evaluation will occur in four phases (i) inception; (ii) desk-based document review and analysis; (iii) field-based data collection; and, (iv) data analysis, reporting and communication of evaluation results. This section offers a proposal for evaluation approach, and evaluation process

Phase 1-Inception: The inception face will feature five activities, as described below:

1. Search, compile and conduct a preliminary review of UNICEF documents and academic and grey literature: This activity is meant to deepen the understanding of the context of UNICEF ECD/ECE programming, and to develop a succinct synthesis of the global context and UNICEF role in early childhood care and education.
2. Conduct a systematic stakeholder analysis: Concurrent with compilation of documents, key groups of stakeholders will be identified (i.e., UNICEF staff members, consultants, international NGO partners, cluster members/partners, donors, researchers/consultancies and other independents subject matter experts, etc.), and a stakeholder analysis will be conducted to determine the role of each key stakeholder and inputs in ECD/ECE, and in the evaluation process. Groups of stakeholders will be judged on three parameters: (i) interest and involvement in ECD/ECE work; (ii) influence/power that they exercise in major decisions about UNICEF work and, (iii) involvement of the stakeholder in the evaluation and expected impact of their involvement. Based on the stakeholder analysis, the evaluation team will articulate an approach for engagement and management of the different stakeholder groups, to be used by the evaluation manager and the evaluation team. The stakeholder engagement strategy should be included as an annex of the inception report.
3. Compilation of all elements of the inception report and submission: The inception report will include (i) revised evaluation questions (improved from the ToR version and informed by users as necessary) presented in an evaluation matrix; (ii) expand on the methodology featuring a confirmation of countries to be included in the desk-based review (approximately 15), based on a well-articulated sampling strategy; (iii) a proposal for the case study approach (thematic, country-based, or both), avoiding undue duplication with on-going and/or planned evaluations and/or research studies; and, (iv) draft evaluation tools for piloting. A more detailed outline for the inception report is provided in Section X (Para. 62).
4. Convene a 1-day inception workshop: After circulating the draft inception report to the primary users, the evaluation manager will convene a virtual workshop to provide an opportunity for interaction between the evaluation team and the primary users, namely ECD and education teams in New York, and to the extent possible, the counterpart in country offices sampled for in-depth case studies. The focus of the workshop will be to validate evaluation questions, discuss the scope of the evaluation and key technical elements, and to communicate/manage expectations of the primary users. The inception workshop will also be used to familiarize the evaluation team with UNICEF standards, processes, and tools, clarify UNICEF expectations and contractual provisions, and to refine the work plan.
5. Presentation to the meeting of the Evaluation Reference Group, and revision of the inception report: The final activity of the inception phase will be to submit and present the inception report to the reference group. The evaluation team will produce a revised inception report, after consideration of the advice of the reference group.

Phase 2-Desk-based document review and country self-assessments: This phase will be data collection from secondary sources, featuring three activities:

Comprehensive search and compilation of documents from 15 country offices selected for the desk-based document review: Continuing from the search described in the inception phase, relevant UNICEF documentation from the global and regional levels, will be collected. These will include work plans, corporate policies, strategies, programme guidance, position papers, inter-agency and global education cluster documents, studies, reviews and evaluations from UNICEF and other organizations. From the country offices which will be a focus of the desk-based review, the team will collect planning and programming documents; situation reports (SitRep); needs assessments; surveys and statistics; donor proposals and reports; field monitoring reports from UNICEF and partners; country audits and financial data; data and reports. A significant amount of data is readily available from existing UNICEF and non-UNICEF databases. These must also be reviewed before any requests for data can be made to UNICEF country office staff, government counterparts, or any other key informants.

Conduct a detailed desk-based document review and analysis and light-touch country self- assessments: The focus of this activity is to review and analyse secondary data and other documentary evidence, and data from 15 selected countries. Task for this activity are: (i) to pre-populate the evaluation matrix for the 15 desk review countries; (ii) execute self-assessments by the ECD/Education teams in the countries that will selected for field-based data collection (using and existing/adapted self-assessment tools) to ensure that the evaluators understand the context and the ECD/ECE programmes fully and refine the methodology for the theme-based case studies accordingly; (iii) refine data collection and data analysis tools for Phase 3; and, (iv) to orient and inform countries that will be asked to participate in the evaluative case studies and host primary data collection missions.

Drafting and submission of the document review report: To be reviewed only by the evaluation manager, the desk review report should present an updated chapter of the global context and UNICEF role in ECD/ECE, with an updated methodology section, and a 3-page summary on the context of each of the 15 countries featured in the desk-based review, confirm the themes for the evaluative case studies (2-3 themes). The format for this report is discussed in Section X (Para. 63) of these terms of reference.

Phase 3: Primary data collection: The activities proposed for this phase are presented below. However, evaluation consultants are free to augment or propose alternatives data collection approaches.

In depth interviews with key informants at the global and regional level: Using the stakeholder analysis generated in Phase 1, a list of key informants will be updated by the evaluation team, in conjunction with the evaluation manager, UNICEF staff, and members of the Evaluation Reference Group (ERG). Primary data will then be collected through key informant interviews, in part to answer a subset of questions for which these informants will be the primary sources, and also to triangulate and/or corroborate some of the data from the desk-based review.

In-country visits: Field visits will be undertaken in a subset of countries that were designated for theme-based evaluative case studies (3 or 4 of the 15 countries, depending on the budget), also for the purpose of collecting primary data. The duration for each visit will be up to 10 working days. Each field visit will commence with a briefing meeting with UNICEF country office education teams, where the 3-page summary about the country context can be shared and subjected for review by the country office. Field-based data collection methods will include validation of the self-assessment, a country level stakeholder analysis, field observations, key informant interviews for UNICEF staff and implementing partners, as well as interviews and/or focus groups for beneficiaries. It is recommended to pilot the methodology and data collection tools during the first field mission before proceeding to subsequent missions. Two debriefing meetings will be conducted, one with the UNICEF country office, and another with interested evaluation participants to present key observations from the field missions, and where feasible, preliminary findings. The debriefing meeting will also be used as an opportunity to fill data gaps, and/or correct the record on factual information, were necessary.

Phase 4 - Data analysis and reporting: The activities proposed for this phase are presented below. However, evaluation consultants are free to augment or propose alternatives data analysis and reporting approaches.

Updating the data analysis plan: Accuracy, triangulation and disaggregation of data will be of crucial importance for the comprehensiveness, usefulness and credibility of the evaluation findings. hence data analysis and interpretation methods proffered in the earlier stages will be updated accordingly. Also, the evaluation will seek to identify contextual and structural/systemic factors explaining the evaluation findings – supporting or impeding performance of UNICEF. Some of the bottlenecks may relate to corporate characteristics, system-wide processes and cross-sectoral issues that lie beyond the education sector. These need to be listed but not examined in great depth.

Drafting and submission extended outline (zero draft): The data phase will conclude with the submission of an extended outline as described in Section X (Paragraph 64), which will be a mock-up of the final report, and if possible, a collection of insights and/or preliminary findings from the field visits, which will be subjected to more systematic and comprehensive analysis.

Confirmatory online survey of findings: At the reporting stage, a single round of an online survey using the Delphi methodology, or a comparable technique will be executed to validate preliminary findings, and to establish the extent to which there is consensus and/or generalizability of findings across UNICEF offices. All UNICEF country and regional offices will be included (approximately 150 offices, which makes this a census approach). Delivered online, the survey will be addressed to the Education/ECD Specialist who will be requested to complete it, ideally with inputs from other programme leads. Each country office will be asked to return only one completed questionnaire.

Drafting and submission final report: Based on the outline in Section X (Paragraphs 66 and 67), the report will be presented in four drafts (completed first draft, revised draft, penultimate draft, and final report). The first draft will be reviewed by the evaluation manager and second reader within the Goal Area 2/5 team, while the revised draft will be presented to ERG for review. To be submitted for clearance by the Director, the penultimate draft should incorporate feedback from the ERG. The final draft will be submitted for copyediting and publishing.

To qualify as an advocate for every child you will have…

Expertise and skills that are outlined in a-d below. One of the two researchers will also assume the role of assistant to the team leader, providing support on the day-to-day management of the project and logistics (tasks associated with e and f).

1. Research, data analysis, and data presentation/display skills
2. Strong skills in quantitative and qualitative data collection and analysis
3. Fluency in English (oral communication, reading complex documents, writing reports)
4. Functional proficiency in Arabic, French, and/or Spanish
5. Excellent organization, consistency, deadline keeping and follow-up; and,
6. Strong communication /people /team coordination skills.

Former UNICEF staff or consultants that have worked on education and/or ECD programmes may be members of the evaluation team if they meet the technical qualifications for skills. However, any prior involvement with UNICEF should be declared during the bidding/contracting stages so that prospective team can be vetted and cleared of possible conflicts of interest.

The duration for the evaluation team is estimated at 9 months, from December 2023 to August 2024. Evaluation consultants should provide the total level of effort to be divided between proposed team members depending on the responsibilities, indicating each member’s accountabilities and effort (in person days).

ToR evaluation of ECD and ECE_November 2023_advert.pdf

How to Apply:

The evaluation team will be made up of a team leader, two evaluators, two researchers, and national consultants recruited from each of the 4 countries that will be hosting a field data collection mission. Each prospective evaluator or researcher should submit an application as an individual. The application packet should contain the following:

1. a letter of expression of interest, stipulating: (i) the specific role for which the applicant wishes to be considered; (ii) expertise in early childhood development and/or early childhood education; (iii) availability to undertake international duty travel; and, (iv) the level of effort (in person days) to be contributed towards each of the deliverables and the proposed remuneration fee;
2. updated CV/resume; and,
3. a writing sample (i.e., evaluation or research/study reports, etc.).

Applicants are free to indicate the names of persons that they would like to team up with if selected. However, all prospective team members should submit an individual application, specific to the role they want to fill in the evaluation team. Also, contracts will be issued to individual consultants, and not consultancy firms or institution. Additional details about the application process are provided in Section XII of the attached terms of reference.

Remarks:

Only shortlisted candidates will be contacted and advance to the next stage of the selection process.