Developmental Evaluator for the LearnIn initiative in ECAR, ECARO, 165 days (Remote)

Contract

This is a Consultancy contract. More about Consultancy contracts.

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.

And we never give up.

For every child, results

The start of COVID-19 pandemic in early 2020 stalled progress toward ensuring quality education for all children and widened existing gaps in learning quality and equity. Europe and Central Asia (ECA) was no exception, with 53.4 million children across ECA experiencing unprecedented disruption to their learning during COVID-19 compounded by challenges such as the digital divide in access to technology and digital kills. Understanding the necessity for schools to provide quality, relevant learning for every child and to be adaptive to such shocks, the Europe and Central Asia Regional Office (ECARO) is supporting digital learning solutions, such as by upskilling teachers to provide quality, inclusive instruction via digital platforms. As digital learning becomes increasingly relevant, however, such solutions must ensure that learners remain the end-users and beneficiaries. Opening up better schools and building more resilient systems will necessitate valid and reliable data not only on the role that such solutions play in ensuring that effective, quality, and inclusive learning continues during crises like COVID-19, but also on the extent of the learning crisis in ECA and effective strategies for bridging learning gaps. While schools aim to recover lost learning and prepare for future emergencies, the central focus must remain on ensuring that quality education leads to relevant and effective learning outcomes for all girls and boys.

In April 2020, ECARO launched LearnIn, a digital learning initiative that aims to support teachers and students as lifelong learners by empowering teachers with high quality, culturally relevant, and collaborative learning opportunities and supports them in creating meaningful learning experiences for their students across diverse learning environments. LearnIn is designed to be an evidence-based initiative, informed, contextualized, and improved by research and adapt to the contexts.

UNICEF ECAR seeks a skilled developmental evaluator to design and implement a Developmental Evaluation (DE) of the LearnIn initiative informing adaptive decisions about the strategic direction of LearnIn, documenting the evolving history of LearnIn, and facilitating learning and reflective practice. The detailed ToRs can be found here: DE ToRs.docx.

How can you make a difference?

The Evaluator will contribute to the core group of LearnIn moving through a range of roles such as observer, questioner and facilitator[1]. As observer, the evaluator will be watching both content and process. What is being tried? What is being decided? How is it being done? How is it being decided? The primary purpose of making observations is to generate useful feedback for the team. As facilitator, the Evaluator may help move a conversation forward. There are times when a group has sufficiently explored a set of ideas but cannot seem to move forward. By framing and synthesizing these ideas for the group, the Evaluator can help the group to make sense of its deliberations, fine-tune and move on. In the same way, the Evaluator as facilitator supports the group as it interprets data so that it can feed directly into the development process. In some cases, the Evaluator is present in meetings that are not evaluation-focused but instead are specific to project tasks or strategy discussions already taking place. Here, the Evaluator may probe to clarify intent and purpose, or may capture information for use at another time. The Evaluator would not need to be present at every meeting. Debriefing with those who did attend may be useful for clarification and surfacing discrepancies in interpretation.

The Evaluator will have overall responsibility for the design and implementation of the DE and for ensuring its quality. Some responsibilities will include the following:

• Facilitate an Acculturation Workshop for DE partners, which will help to collaboratively conceptualize and develop the DE design and approach (learning framework, methodology, work plans, reporting, etc.).
• Develop a living work plan for the evaluation, including an agreed set of deliverables (which may change over time, depending on the needs).
• Orient and manage other data collection activities as they arise.
• Produce good quality deliverables, including regular intermediate report logs (that track details of DE and document all events that occur — i.e., planned, unplanned, and associated outcomes) and the final report of the evaluation.
• Facilitate regular meetings with programme leadership and other staff to collect data.
• Conduct analysis of the data collected and present digestible analysis regularly to programme staff and key partners through the data-review process and other participatory meetings.
• Track and communicate high-quality information to programme staff, and key partners in on the programme’s progress, which can inform timely and data-driven decision-making; regularly record decision-making, programme changes, and/or changes in the environment in report log.
• Facilitate periodic reflections as to whether the LearnIn TOC continues to hold true over time and determine what changes, if any, may be necessary. The Evaluator will not update the TOC, but rather support the program team to do so if and when deemed appropriate
• Track, document, and help interpret the nature and implications of innovations and adaptations as they unfold
• Facilitate real-time, data-driven decision-making in strategy and across functions
• Provide accountability for external stakeholders and help to refine their contributions as solutions evolve
• Extract lessons and insights from processes and outcomes to inform ongoing adaptive processes
• Provide some capacity building events/exercises on monitoring to ensure data is collected on a regular basis and leveraged for learning and improvement.

The Evaluator will be supported by UNICEF ECARO Education section and report to UNICEF ECARO Evaluation section. The ECARO Evaluation section member will be the Evaluator’s manager and act as a DE Administrator with two primary roles: being in charge of launching and overseeing the DE, and providing technical support to the Evaluator. The Administrator will primarily be responsible for the management of the DE, e.g., ensuring adherence to agreed-upon budgets, contracts, and timeline, and etc. The Administrator will also serve as a sounding board for thinking through complex and emergent issues and providing technical and methodological support as required.

The Evaluator is responsible to submit the following Deliverables.

Stages

Tasks and Deliverables

Proposed person days (international)

Inception (total 2 months)

Signing contract; Initial do review; Initial outreach and introductions to teams; Stakeholder map; Draft SOW; DE questions; Acculturation Workshop; Inception report

40 days

Implementation (total 7 months)

SOW; Complexity map; Collecting and analysing data; Any adaptive activities and outputs (strategic learning debriefs, pause and reflect sessions, adaptive checklists, revised ToC and results framework, etc.); producing outputs as agreed during inception stage

105 days (15 days a month)

Wrapping up (total 3 months)

A road map for implementation of suggested adaptations; knowledge management systems; transitioning key tasks and processes; simple toolkit for use in future activities or processes; final report

20 days (10 days a month)

Total

165 days

The consultant should propose a timeline to submit the deliverables in their implementation plan (in proposals). Necessary and adequate time (at least two weeks) should be allocated for review and quality assurance processes of the key deliverables by the ECARO sections, ERG and External Institution

Contractual modalities:

• Duration: The consultant is expected to work for 165 days within a 11 month period between December 2021 and November 2022.

• Location: The consultant will work remotely and is not expected to travel.

• Payment: Payment is contingent on approval by the Evaluation Manager and will be made in four instalments: - 30% upon clearance of inception stage deliverables and activities

• 30% upon clearance of implementation stage deliverables and activities
• 40% upon clearance of wrapping up stage deliverables and activities

To qualify as an advocate for every child you will have…

• An advanced university degree (Master’s or higher) preferably in social science, organizational theory/behavior/communications, international development, or related focus. A combination of relevant academic background and relevant work experience may be accepted in lieu of the advanced university degree.

Knowledge/Expertise/Skills required:

• At least 7 years of experience in developing M&E systems, conducting evaluation, and research with extensive technical knowledge, skills, and expertise in evaluation design, concepts, and approaches, and evaluating complexity in particular;
• Proven programming and/or evaluation experience in inclusive education or/and digital learning with a focus on marginalized populations preferable;
• Strong facilitation skills, particularly related to programmatic and organizational learning;
• Excellent familiarity with DE, including skills such as respect for local stakeholders and partners, ethical research (maintains confidentiality and anonymity when required), flexibility, energy, humility, willingness to learn on the go, and ability to resolve conflicts;
• Strong analytical skills to support both qualitative and quantitative research;
• Excellent oral and written communication and report-writing skills in English;
• Keen attention to detail, especially related to documenting data and associated processes; and
• Familiarity with UNICEF organizational structure/context.

Strongly Preferred:

• Leadership and strategic thinking skills;
• Active listening, proactive learning, and time management skills, with readiness to be in a learning role;
• Previous experience and comfort with working for multiple stakeholders with competing priorities/interests; and
• Previous work in ECA region

Desirable:

• Skills, capabilities and experience of creating dashboards in R Shiny or other alternatives.
• Skills, capabilities and experience of developing visual outputs as suggested in the ToR

For every Child, you demonstrate…

UNICEF's values of Care, Respect, Integrity, Trust, and Accountability (CRITA) and core competencies in Communication, Working with People and Drive for Results.

To view our competency framework, please visit here.

Click here to learn more about UNICEF’s values and competencies.

UNICEF is committed to diversity and inclusion within its workforce, and encourages all candidates, irrespective of gender, nationality, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of the organization.

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.

To apply:

Interested applicants are invited to submit:

1. A technical proposal including their understanding of the assignment, its scope and potential risks and/or areas that may need further clarification during the inception phase; anticipated challenges related to this exercise and mitigation measures; proposed methodology and approach that are best ‘fit for purpose’ for the developmental evaluation; a tentative timeline; ideas to set up the developmental evaluation for the purposes described building on what is already in ToR; approaches to contextualise DE approaches, tools and materials for UNICEF use; list of past and ongoing developmental evaluations carried out by the bidder, accompanied by links to two sample reports.
2. A CV and a fully completed profile in the UNICEF e-recruitment system.
3. A financial proposal includign a daily fee to undertake the above TOR. Please note that applications without a proposed fee will not be accepted.

Remarks:

Only shortlisted candidates will be contacted and advance to the next stage of the selection process.

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.

[1] Jamie A.A. Gamble (2008) A developmental Evaluation Primer. untitled (mcconnellfoundation.ca)