Local Consultancy - Team Leader for the Equity and Fairness-Oriented Analysis of School Classroom Assessment Practices to Identify Policy Recommendations for Improvement of Student Learning
Contract
This is a Consultancy contract. More about Consultancy contracts.
UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.
Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.
And we never give up.
For every child, a champion
Individual Consultant is expected to perform this assignment remotely (home based) with possible field missions as required.
Purpose of Activity/Assignment:
The aim of the assignment is to support improvement of Armenia’s classroom assessment policy and practices as well as national learning assessment system through comprehensive system analysis and policy recommendations that is systemic and would support implementing assessment for learning that is fair and equitable to all students. Under this assignment, the Team Leader, with a team of dedicated experts, will be responsible for the following two major components of the assessment, which envisage both secondary review of existing data, desk-research and primary data collection/analysis:
- analyzing school classroom assessment practices to generate evidence-based policy recommendations for improvement of student learning outcomes in Armenia;
- analysis of Armenia’s national learning assessment system.
These activities will contribute to the SDG (Sustainable Development Goal) Target 4.6. which requires states to “ensure that all youth and a substantial proportion of adults, both men and women, achieve literacy and numeracy.” At the global level, the agenda around the SDG 4 to ‘ensure inclusive and equitable quality education and promote lifelong learning opportunities for all’ by 2030, has reinforced a focus on learning assessment to gain data on learning that allow to establish and monitor education quality and learning progress.
Component One: Equity and Fairness-Oriented Analysis of School Classroom Assessment Practices to Identify Policy Recommendations for Improvement of Student Learning Outcomes
The specific objectives under this component are:
- to complete a robust and comprehensive literature review, followed by an overview of local legal framework and existing regulations, comprehensive desk-research and secondary analysis of existing raw data and relevant statistics;
- to design the methodology and the tools for data collection from the key stakeholders and beneficiaries, including local and regional actors, such as the education departments of regional authorities, teachers, students and other target groups;
- to manage the quantitative data collection and quality control of survey among teacher through the National Center of Educational Technologies (NCET) of the Ministry of Education, Science, Culture and Sports of Armenia (MESCS);
- to collect and analyze the qualitative data from students, teachers and school leadership, as well as analysis and reporting of primary data from the assessment through close collaboration with UNICEF and MESCS.
The analysis shall seek to explore:
- What would be the optimal theory and practice of classroom assessment as proposed by advanced international scholarly work and practice;
- To what extent is the current classroom assessment practice fair to and promotes equitable learning for all students;
- What are the key stakeholder views and understandings of classroom assessment, both in practice and in theory, and how they compare to the advanced international literature and practice reviewed;
- What are the key constraints and resources (current or required) to increase the quality of classroom assessment that is fair and promotes equitable learning for all students;
- The extent to which the contexts and assessment programmes are consistent with other elements of the wider education system, for example with learning standards and curriculum, the structure of the education system, national education priorities.
Component Two: Analysis of Armenia’s National Learning Assessment System
The specific objectives under this component are:
- comprehensive literature review of advanced international research and practice, document review and key stakeholder consultations focusing on the following three dimensions: (i) the context of the national assessment system, (ii) the quality of assessment and (iii) coherence with the wider educational system;
- comparative analysis of the methodologies, produced data and required implementational resources of ATC-organized large-scale assessments (e.g. HAAS, BAAS), privately organized large-scale contest-assessments (e.g. Kangaroo) and international large-scale assessments (TIMSS, PISA) to identify which of these assessments would comprise be the efficient policy of generating evidence to analyze equitable student learning.
The analysis shall aim to provide evidence-based policy recommendations to improve Armenia’s national learning assessment system.
Key stakeholders of the assignment are the government line ministries, namely the Ministries of Territorial Administration and Infrastructure (MoTAI), MESCS, sub-national and local authorities, UNICEF, EU and dedicated CSOs.The knowledge generated by the study should be used by UNICEF Armenia to inform and amplify its future activities and projects in the country and will also be used by other national stakeholders to further refine the quality, cohesiveness and timeliness of policies in the field of general education
How can you make a difference?
Scope of Work:
Context
National learning assessment systems are of particular importance in developing data-driven policies to ensure inclusive and equitable quality education, achieve universal literacy and numeracy, and to promote lifelong learning opportunities for every child. Such systems are broadly defined as sets of policies, practices, structures, organizations and tools used to generate data and improve learning. Generally, four types of learning assessment are used by countries to guide learning, and are complementary in terms of objectives, scopes and limitations: (i) national large-scale assessments, examinations and tests (i.e. assessment of learning); (ii) international large-scale standardized assessments (i.e. assessment of learning); (iii) classroom assessment (i.e. assessment as/for learning), (iv) household surveys. Fairness and equity in assessment are important in recognizing and supporting the needs of increasingly diverse student populations.
Armenia’s national learning assessment system is primarily defined by the Law on Education (1999), the Law on General Education (2009) and various normative documents prepared by the Ministry of Education and the Government. In 2004 Armenia established its National Assessment and Testing Centre (transferred from the Ministry’s subordination to that of the Government Cabinet in 2009) mandated to develop and implement end-of-cycle graduation examinations (covering primary, secondary and high school graduations) and university admissions unified examinations; to provide methodological support to teachers in classroom assessment (the latter is summative at large); to implement national and international assessments; and to research and analyze assessment and examinations data. Starting in 2009 unified examinations have been used for all subjects. In 2010 Armenia conducted its first domestically-developed national large-scale assessment (sample-based survey adopting certain technical procedures from TIMSS) in the Armenian language, literature, and history (HAAS 2010), in physics and chemistry (BAAS 2011), in geography and biology (BAAS) and ICT (Information Communication Technology) in 2012, in foreign languages (OLAAS) in 2013. Since 2010 the National Assessment and Testing Centre has been regularly conducting the above-mentioned national large-scale assessments and external subject-specific evaluations to provide student achievement data comparable to in-school summative assessments. Since 2008, most Armenian students have annually participated in privately developed and administered (i.e. Ayb Educational Foundation) large-scale subject contests measuring their knowledge and skills: Kangaroo in mathematics (about 40,000 participants nationally), Russian Bear assessment in Russian language and Bee in the Armenian language. In 2003, 2007, 2008 (TIMSS Advanced), 2011 and 2015 Armenia has also participated in TIMSS (i.e., The Trends in International Mathematics and Science Study) by The International Association for the Evaluation of Educational Achievement (IEA). In 2011 Armenia participated in the SABER (System Assessment and Benchmarking for Education Results) Student Assessment to benchmark its student assessment system, so far, the first and largest systematical examination of the national learning assessment system.
In 2021, the Government approved Armenia’s new National Curriculum, to implement the provisions of which in full, the Ministry of Education, Science, Culture and Sport (MESCS) joint with the National Assessment and Testing Centre (NATC) and its newly established National Center for Educational Development and Innovation (NCEDI) intends to review the national learning assessment system, design new classroom assessment guidelines and train teachers.
Responsibilities in the Team
The Team Leader will be responsible for developing, planning and leading the work of technical working groups for both components of the assignment, namely “Equity and Fairness-Oriented Analysis of School Classroom Assessment Practices to Identify Policy Recommendations for Improvement of Student Learning Outcomes” and “Analysis of Armenia’s National Learning Assessment System”, with full set of reporting documents adhering to UNICEF quality assurance standards being developed and submitted in a timely manner. The Consultant will be ensuring in-depth cooperation and communication with various stakeholders, particularly, members of the Local Education Group, focal points at MESCS, NCET, NATC, NCEDI, other agencies, as deemed necessary. Each technical working group will be composed of two Senior and two Junior Experts to be supervised by the Team Leader.
International expertise will be available for guidance and orientation to globally available resources and methodologies. UNICEF will organize thematic orientation sessions and consultations. This assignment is within the Global Partnership for Education (GPE) Education Sector Development Programme Grant-a joint UNICEF and the Ministry of Education, Science, Culture and Sports initiative to be implemented during 2021-2022.
Methodology
The study methodology shall follow a mixed-methods approach combining quantitative and qualitative methods of data collection to obtain rich insights, address all the objectives of the assignment and to assure triangulation of data sources and data collection methods. The detailed methodology allowing for collection of all required information with detailed sampling strategy, study instruments and analytical approach should be developed by the Team at the end of the inception phase and submitted to UNICEF in a form of a Study Protocol for approval and quality assurance.
The quantitative data should be utilized through secondary review of national external assessments aimed at monitoring classroom summative assessment, as well as other existing stakeholder surveys. Also, a survey among schoolteachers via standardized questionnaires should be developed and administered online in collaboration with NCET. Only teachers, that have received grading (first, second, third or fourth) should be targeted by the survey. Total number of the sample frame is around 500 teachers. Final decision on the sample design should be reached in the Inception Phase in close collaboration with UNICEF to either proceed with a census approach or with a stratified random probability sampling, specifically controlling for rural/urban, consolidated/non-consolidated, borderline/high-mountainous settlements and reserving a quota for the three large cities.
The qualitative data should be collected through remote stakeholder semi-structured interviews and focus groups discussions (FGDs) with the following target groups: students of 14-17 years old age group, teachers, principals, vice principals, school board members, school assessment guideline providers. Total number of key informant and key expert interviews (KIIs and KEIs) planned for the study is around 10-15 per each target group. Around total of 5-10 FGDs, with up to 8 participants each, should be administered. Details of sampling approach, sample sizes and recruitment of participants should be developed by the Consultant, presented in the Study Protocol and discussed with UNICEF Armenia team prior to approval.
The analysis should concentrate on a comprehensive and holistic overview of the assessment policy and practices in Armenia as well as the national learning assessment system as a whole, identify gaps within the system and propose targeted and both theory-driven and empirically proven solutions to each of these challenges. It should include the analysis of secondary data received from NCET and/or NATC, consultations with key stakeholders, including local and regional actors, such as the education departments of regional authorities adjacent to the MoTAI, as well as gathering, analysis and reporting of data according to the methodologies presented above.
The final reports should incorporate the main learnings from all stages and provide concrete and accurate policy solutions for the MESCS to pursue. More information on the technical requirements for each report will be provided to successful candidates.
The analysis of Armenia’s national learning assessment shall follow a comprehensive two-stage approach.
Firstly, Armenia’s national learning assessment system shall be benchmarked and evaluated against quality criteria based on the ANLAS methodology developed by the Global Partnership for Education. This analysis shall include comprehensive literature review of advanced international research and practice, document review and key stakeholder consultations focusing on the following three dimensions: (i) the context of the national assessment system, (ii) the quality of assessment and (iii) coherence with the wider educational system. Each dimension shall be evaluated based on dimensional key areas and defined quality objectives.
- The first dimension refers to the broader context in which the national learning assessment system is located and the extent to which this context is supportive of the assessment system. The key areas include: (i) legislation or policy, (ii) institutional arrangements and governance structures, (iii) funding, (iv) leadership.
- The second dimension refers to the overarching key quality concepts for learning assessments: clarity and consistency of purpose, fitness for purpose, technical rigor, objectivity and independence, transparency and accountability and ethicality and fairness. The key areas are defined per type of the assessment: (i) large-scale assessments and examinations, (ii) classroom assessments.
- The third dimension relates to the extent to which the assessment system is consistent with important aspects of the broader education system and other aspects within the assessment system. The key areas include: (i) learning standards and curriculum, (ii) education system structure, (iii) education priorities, (iv) system-level data use.
Secondly, comparative analysis of the methodologies, produced data and required implementational resources of ATC-organized large-scale assessments (e.g. HAAS, BAAS), privately organized large-scale contest-assessments (e.g. Kangaroo) and international large-scale assessments (TIMSS, PISA) to identify which of these assessments would comprise be the efficient policy of generating evidence to analyze equitable student learning.
To qualify as an advocate for every child you will have…
- Advanced university degree (PhD) and/or academic background in Education, Sociology, Economics, Public Policy, Social Policy or a related field.
- At least 8 years of documented experience in education (preferably at least 4 years in either research or development of large-scale assessments); with solid knowledge of international research practice in the field of national learning assessment systems; as well as of Armenia’s national learning assessment system (large-scale assessments, examinations, classroom assessments);
- Extensive experience in designing research projects, studies, evaluations and household surveys, data analysis and report writing (both in Armenian and in English; links to published research and analytical documents relevant to the TOR to be provided).
- Proven ability of leading large teams and expert groups as well as of working in international and multicultural environments, as well as with children and adolescents.
- Very good communication and presentation skills with government and international organizations.
- Fluency in written and spoken English.
- Previous experience in working with UN agencies will be an asset.
For more details please refer to the attached Terms of Reference. ToR (B2+B6-Team Leader)_VA.pdf
Terms of the application:
The deadline for the submission of applications is April 28. Applications should include:
- Cover letter;
- CV;
- Financial Proposal;
- Published research and analytical documents relevant to the TOR;
- Examples of published work (paper, abstract, proceedings) in top international conferences and SCOPUS-indexed high-quality journals will be an asset.
For every Child, you demonstrate…
UNICEF's values of Care, Respect, Integrity, Trust, and Accountability (CRITA) and core competencies in Communication, Working with People and Drive for Results.
To view our competency framework, please visit here.
Click here to learn more about UNICEF’s values and competencies.
UNICEF is committed to diversity and inclusion within its workforce, and encourages all candidates, irrespective of gender, nationality, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of the organization.
UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.
Remarks:
Mobility is a condition of international professional employment with UNICEF and an underlying premise of the international civil service.
Only shortlisted candidates will be contacted and advance to the next stage of the selection process.
Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.