Consultancy-Learning Specialist (Reading) - Developing a household survey tool to measure learning skills in primary school age children, Data & Analytics (D&A), DAPM NYHQ (remote based)

Contract

This is a Consultancy contract. More about Consultancy contracts.

Consultancy Title: Learning Specialist (Reading) - Developing a household survey tool to measure learning skills in primary school age children

Section/Division/Duty Station: Data and Analytics (D&A)/DAPM/New York

Duration: 11.5 months (127 working days)

About UNICEF

If you are a committed, creative professional and are passionate about making a lasting difference for children, the world's leading children's rights organization would like to hear from you. For 70 years, UNICEF has been working on the ground in 190 countries and territories to promote children's survival, protection and development. The world's largest provider of vaccines for developing countries, UNICEF supports child health and nutrition, good water and sanitation, quality basic education for all boys and girls, and the protection of children from violence, exploitation, and AIDS. UNICEF is funded entirely by the voluntary contributions of individuals, businesses, foundations and governments. UNICEF has over 12,000 staff in more than 145 countries.

BACKGROUND

UNICEF plays a critical role in monitoring the situation of children and women in the world and for reporting on progress toward internationally set goals such as the Sustainable Development Goals (SDGs), the World Fit for Children (WFFC) goals and UNICEF’s Strategic Plan (SP) targets. To support the global monitoring, UNICEF maintains a series of global databases on key indicators, assists countries in collecting necessary data through the UNICEF-supported Multiple Indicator Cluster Surveys (MICS), and conducts statistical analyses of the data for global reporting purposes.

The Foundational Learning Skills (FLS) module is a household-based assessment tool that was released as part of the MICS standard tools in 2017. The process to develop the FLS assessment began in 2014 and included a series of technical consultations with an advisory group composed of leading experts in early grade learning[1]. The objective of the FLS module was to generate comparable household survey data on children’s foundational learning skills in reading and math, thus contributing to SDG indicator 4.1.1(a) (“Proportion of children and young people in grades 2/3 achieving at least a minimum proficiency level in reading and mathematics, by sex”).

The MICS FLS module is the first assessment of its kind to be standardized and implemented by a global household survey program. As of October 2021, nearly 50 countries have either collected comparable data on foundational reading and math skills through MICS or are preparing for data collection. The data generated is helping bring increased stakeholders’ attention to early grade learning as the foundation for learning in later grades. It is also highlighting the need to address learning inequalities in early grades. As is the case with other MICS modules, the FLS module may be adopted by and integrated into other household surveys[2].

With learning loss during the school closures due to COVID-19, measuring learning outcomes in foundational learning skills has become an even more critical issue for the global community. This consultancy will contribute to strengthening the ability of countries to collect data on children’s learning. This will be achieved through the improvement of the existing FLS module and the development of a new assessment tool to measure minimum proficiency in reading at the end of primary school (SDG indicator 4.1.1(b): “Proportion of children and young people at the end of primary achieving at least a minimum proficiency level in reading and mathematics, by sex”).

[1] Details and methodological papers about this process can be found here: https://mics.unicef.org/methodological_work/2/ASSESSMENT-OF-LEARNING-OUTCOMES

[2] United Nations Children’s Fund, Guidelines for Adapting the Foundational Learning Module to Non-Multiple Indicator Cluster Household Surveys, UNICEF, New York, August 2019 (https://data.unicef.org/resources/guidelines-adapting-foundational-module-non-mics/)

Terms of Reference / Deliverables

In close collaboration and working with UNICEF’s project leads (householdsurvey specialist and the senior advisor in statistics and monitoring in Data and Analytics Section and the learning specialist in the Education Program Group), the incumbent is responsible for carrying out a series of studies and analyses to develop tools to assess learning skills among primary school children.

The incumbent is expected to contribute to the development of improved and new instruments for testing in two field locations, support the facilitation of an external expert review process of the instruments, and contribute to the preparation and finalization of a standard package that will guide the implementation of these assessments in the field.

Outcomes and specific deliverables of the consultancy are listed as follows:

Outcome 1: Improved alignment of the MICS Foundational Learning Skills (FLS) module to the Global Proficiency Framework (GPF) for reading in primary grades 2 and 3.

Under the supervision of UNICEF’s project team, the learning specialist will draft a technical report that presents various proposals to improve the alignment of the FLS assessment to the GPF. Working with the UNICEF project team, the specialist will:

• Propose modifications to the assessment instrument’s content that would be required to achieve alignment at different levels (i.e. minimally, additionally and strongly aligned) in terms of what additional domains, how many subconstructs per domain and how many items per subconstruct may need to be included in the improved assessment;
• Identify potential implications of each option for the administration of the assessment in a household setting (e.g. length and format of the assessment; see below for further details);
• Develop specific reading items that can be added to the FLS instrument to measure additional constructs or skills, as needed, based on the decision to achieve a given level of alignment;
• Support an expert review process by preparing documents for review and compiling external experts’ inputs (see below for details regarding the expert review);
• Evaluate and document how different modifications to the FLS instrument may affect data comparability with the original instrument released in 2017 and develop draft guidelines on how to interpret trends over time.

Outcome 2: Development of a new household-based assessment to measure minimum proficiency level in reading at the end of primary education.

In consultation with UNESCO, the UNICEF project team will explore the feasibility of adapting school-based assessment being developed by UNESCO’s Institute of Statistics (UIS) for administration in a household setting in the context of a multi-topic household survey like the MICS. As with the FLS assessment, the target is to develop an assessment that is adequately aligned with the GPF. The consultant will produce a proposal containing streamlined and adapted versions of the school-based assessment instrument’s content that would achieve alignment at different levels (i.e. minimally, additionally and strongly aligned). The proposal will identify what additional domains, how many subconstructs per domain and how many items per subconstruct may need to be included in the assessment.

The learning specialist will examine the UIS tools and, in consultation with the UNICEF project team, will make recommendations on item selection and overall design of the assessment. The following issues will be considered:

• Format and mode of the assessment: The administration of a learning assessment in a household setting differs from a school setting in various ways. For example, assessments are administered individually instead of in group settings. Assessments using only paper-and-pencil forms may not be possible. Likewise, even though MICS interviewers use tablets for data collection, computer-based testing may not be possible or advisable, and oral responses may be necessary. Children who have never been to school or who dropped out some time ago may not be familiar with test formats commonly administered in schools.
• Length of the assessment: A household-based assessment may need to be shorter than school-based assessments due to interview duration limitations. As a result, it may only be possible to adopt a subset of the UIS assessment items. Any necessary adjustments to the content and/or number of items will be in line with GPF expectations to ensure the instrument is adequate for SDG reporting.

The documentation and recommendations developed by the learning specialist will be reviewed by a combination of in-house and external experts in learning and household survey methodology. Recommendations and changes will be clearly documented, and the tools modified accordingly.

Outcome 3: Develop a household survey module that integrates both learning assessment tools (i.e. early grades and end of primary) and includes a screening mechanism for assignment of respondents to the appropriate assessment path.

Working closely with the UNICEF project team, the specialist will contribute to developing a draft household survey questionnaire that hosts both assessments and uses a screening mechanism to assign each respondent to the appropriate assessment module. The specialist will also provide inputs to the interviewer’s instructions and indicator tabulation plan, and will contribute to the preparations for two field tests that will provide evidence about potential issues with the tool and any concerns regarding the feasibility of administering the assessment in a household setting.

Working closely with the UNICEF project team, the learning specialist will develop the following materials for field trial and testing:

• Selected items for each learning assessment module.

• Recommendations for a screening mechanism to assign each respondent to the appropriate assessment module.

• Design of paper-based stimuli for each module.

• Guidelines for programming the screening mechanism in the CAPI entry application.

• Training guidelines for interviewers.

Qualifications

1) Education

• Advanced degree in Psychology, Education, Measurement or closely related areas.

2) Work experience

• A minimum of ten years of progressively responsible experience in relevant areas.
• A proved record of publications in peer-reviewed journals in the field of expertise in assessing learning outcomes in reading.
• Experience in working on household surveys in low- or middle-income countries is an asset.

3) Competencies

• Strong organizational, planning, and analytical skills
• Native or near-native proficiency in English (oral and written) is required. Knowledge of another UN language is an asset.
• Computer skills: Competent in advanced Excel, Word, PowerPoint.
• Other skills and attributes: excellent organizational skills and ability to prioritize and manage multiple takes; Strong writing and interpersonal communication skills.

Requirements****:

• Completed profile in UNICEF's e-Recruitment system and provide Personal History Form (P11) Upload copy of academic credentials
• Financial proposal that will include:
• your daily/monthly rate (in US$) to undertake the terms of reference (can be downloaded here: https://www.unicef.org/about/employ/index_consultancy_assignments.html
  • travel costs and daily subsistence allowance, if internationally recruited or travel is required as per TOR.
  • Any other estimated costs: visa, health insurance, and living costs as applicable.
  • Indicate your availability
• Any emergent / unforeseen duty travel and related expenses will be covered by UNICEF.
• At the time the contract is awarded, the selected candidate must have in place current health insurance coverage.
• Payment of professional fees will be based on submission of agreed satisfactory deliverables. UNICEF reserves the right to withhold payment in case the deliverables submitted are not up to the required standard or in case of delays in submitting the deliverables on the part of the consultant.

U.S. Visa information:

With the exception of the US Citizens, G4 Visa and Green Card holders, should the selected candidate and his/her household members reside in the United States under a different visa, the consultant and his/her household members are required to change their visa status to G4, and the consultant’s household members (spouse) will require an Employment Authorization Card (EAD) to be able to work, even if he/she was authorized to work under the visa held prior to switching to G4.

Only shortlisted candidates will be contacted and advance to the next stage of the selection process

For every Child, you demonstrate…

UNICEF's values of Care, Respect, Integrity, Trust, and Accountability (CRITA) and core competencies in Communication, Working with People and Drive for Results.

To view our competency framework, please visit here.

Click here to learn more about UNICEF’s values and competencies.

UNICEF is committed to diversity and inclusion within its workforce, and encourages all candidates, irrespective of gender, nationality, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of the organization.

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.

Remarks:

Mobility is a condition of international professional employment with UNICEF and an underlying premise of the international civil service.

Only shortlisted candidates will be contacted and advance to the next stage of the selection process.

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.

This is an extension of the vacancy in order to expand the candidate pool. Previous applicants need not reapply as their original application will be duly considered.