Consultancy: UN Women Impact Evaluation Design and Guidance Note

This opening expired 3 years ago. Do not try to apply for this job.

Application deadline 3 years ago: Thursday 5 Nov 2020 at 23:59 UTC

Open application form

Contract

This is a International Consultant contract. More about International Consultant contracts.

Background

1.Purpose of the Consultancy

The purpose of this consultancy to provide guidance for evaluation managers on evaluating impact of their efforts towards gender equality and empowerment of women and to contribute to global knowledge and national evaluation capacity for evaluating gender equality and empowerment of women.

The objective of this consultancy is to adapt an existing impact evaluation framework (see annex 4 & 5) based on remote interviews and a desk-based review of a sample of 3 existing UN Women country or regional programmes or projects in the area of preventing and ending violence against women.[1] The framework was developed by the UN-Women Independent Evaluation Service as part of a working paper on approaches for measuring impact in Gender Equality and Empowerment of Women (GEEW) (details below). The consultant will develop a short 10 pages guidance note for evaluation and programme managers to carry this work forward, which is based on learnings from the adaptation of the framework to existing projects. The target audience is both UN Women staff and the global community of evaluators or programme managers. The consultant will report to the Regional Evaluation Specialist that is managing this work, but overall oversight of the consultant’s work is provided by the Chief of Independent Evaluation Service.

2.Background on Impact Evaluation at UN Women

The UN Evaluation Group (UNEG) 2013 Guidance on Impact Evaluation clearly outlines the different approaches to impact evaluation and their relative pros and cons. Most UN Agencies, including UN Women, have adopted the latest DAC definition of impact: “the extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects.”

Further to this, UN Women’s evaluation policy articulates the dual role that evaluations can play both in terms of providing evidence-based information about an intervention, as well as “a means to enhance gender equality and the empowerment of women through the incorporation of gender and women’s rights dimensions into evaluation approaches, methods, processes and use. Accordingly, not only does evaluation act as an important driver of positive change towards gender equality and the empowerment of women, but the way in which the evaluation process itself is undertaken empowers the stakeholders involved.” [2]

The UNEG guidance[3] lands on the following key principles for impact evaluation. A fundamental starting point of impact evaluation is establishing cause and effect chains to show if an intervention has worked and, if so, how. Different impact evaluation designs provide varying approaches to establishing how, and to what extent, interventions have caused anticipated and/or unanticipated effects. A “mixed method” approach utilizing quantitative, qualitative, participatory and blended (e.g. quantifying qualitative data) approaches is now widely accepted as advisable to address the types of interventions that are now predominant in international development. Impact Evaluations seek to identify the attributable change in an outcome, with one approach being through a counterfactual analysis. A traditional Impact Evaluation approach can be difficult to apply to UN Women’s normative and coordination work, as it is virtually impossible to construct a counterfactual for those.

3.UN Women’s working paper on evaluating impact of UN Women’s contributions to GEEW

The UN Women working paper[4] proposes a mapping of impact pathways for one to two of UN Women’s Thematic Priorities (TP), which are equivalent to UN Women’s Strategic Plan outputs. It was decided to focus on: Thematic Priority 8 (output 11 in the strategic plan):More countries and stakeholders are better able to prevent violence against women and girls and deliver essential services to survivors.”

TP8 was chosen for several reasons:

  • Outcome 4 of the Strategic Plan focused on Preventing and Ending Violence Against Women and girls receives the second largest amount of funding across the organization ($78 million covering 72 countries last year, and with $425 million allocated across all four years of the SP).
  • The causal evidence around the Elimination of Violence against Women and Girls (EVAWG) is well articulated more broadly in the literature and can be drawn upon to help inform UN Women’s impact evaluation work in this space; and
  • Several UN Women initiatives have looked at impact evaluation of related activities, for example the “Evaluation of the Port Moresby: A Safe City for Women and Girls” Program.

    Building block framework (‘the framework’)

The working paper identified enabling factors or building blocks that together provide the enabling environment or framework needed to secure a change in learning and action. The building block model[5] provides a useful framework for identifying the enabling factors (and by default the barriers) for change. The building blocks are the key enablers of change and by showing individual impact pathways for the building block relevant to each pillar of UN Women’s work, it is possible to: i) build a more comprehensive understanding of how interventions make a difference (and which are the key factors); ii) evaluate the individual and combined contributions of all three pillars of work; and iii) evaluate the impact and contribution of different stakeholders/initiatives working on different entry points (e.g. building blocks) and their individual and collective impact pathways.

As a result, 11 building blocks were identified providing the enabling environment underlying achievement of TP8 outputs and aligned with UN Women’s integrated mandate (See Annex 5).

4.Methodology

The existing guidance from UNEG on Impact Evaluation[6] cited previously, is already comprehensive in terms of outlining the various approaches to measuring impact. The guidance document outlines the definition and role of impact evaluation, impact evaluation design including a range of design approaches, development of a theory of change, and evaluability assessment design. It also includes a comprehensive description of a range of quantitative, qualitative and participatory methods available for IE, as well as guidance on how to choose a mixed methods approach. Much of this guidance will be relevant to designing IE for GEEW.

The UNEG Handbook on evaluating normative work[7] also provides a wealth of relevant information. It includes hands-on methodological guidance, concise practical examples and tools for conducting evaluations of normative work and is highly relevant to the Building Block IE Framework presented in the working paper.

Taking the above into consideration, the working paper proposed steps for planning an impact evaluation utilizing the building blocks framework.

[1] The projects are currently being selected – all documents will be provided to the consultant.

[2] Evaluation policy of the United Nations Entity for Gender Equality and the Empowerment of Women (2012)

[3] UNEG (2013). “Impact Evaluation in UN Agency Evaluation Systems: Guidance on Selection, Planning and Management.”

[4] The paper will be provided to the selected consultant as it is yet to be published.

[5] This analysis was informed by a UNDP Risk Governance Building Block Framework used in the Pacific for enabling transformation of the development agenda in support of resilient development. A review of the risk governance building blocks identified ten that are enshrined in the UN Women’s Strategic Plan (see Table 2). One additional building block – an enabling political framework (e.g. political commitment, political will, champions) was highlighted as a key enabling factor in UN Women’s Strategic Plan.

[6] United Nations Evaluation Group (UNEG) (2013). “Impact Evaluation in UN Agency Evaluation Systems: Guidance on Selection, Planning and Management.”

[7] United Nations Evaluation Guidance (UNEG) (2013) UNEG Handbook for Conducting Evaluations of Normative Work in the UN System.

Duties and Responsibilities

Consultancy Assignment

The adaption of an existing impact evaluation framework (see Annex 4 & 5) for 3 existing UN Women regional and/or country programmes or projects based on desk-based review and remote interviews; and an accompanying guidance note will be developed by a consultant The consultant will report to the Regional Evaluation Specialist that is managing this work, but overall oversight of the consultant is provided by the Chief of Independent Evaluation Service. UN Women project managers, thematic experts, HQ programme guidance unit, and others may be consulted to provide feedback during the process. The assignment is expected to initiate in November 2020 and must be completed by December 2020.

The consultant will conduct:

  • Desk review of the working paper and relevant literature and background documents (project documents, Strategic Note of the CO, relevant evaluations in the area, etc.).
  • Virtual Discussions/ interviews using an online platform (e.g. Skype, Teams, Zoom) with key UN Women staff from IES, project/programme managers, and Senior Management or relevant personnel at Country or regional level; and potentially with other international organizations with experiences in measuring impact.
  • Develop an impact evaluation design, including a framework that applies the building block model developed in the working paper using 3 existing regional and/or country programmes or projects (provided to the consultant by UN Women).
  • Develop a guidance note no more than 10 pages articulating the suggested steps for impact evaluation (building on the working paper) which incorporates the lessons from piloting the block framework and suggestions on how to adjust the framework, if necessary, to the context of UN Women to make it practical and to be able to apply it to other thematic priorities. (At least 1 round of revision based on feedback from UN Women).

The consultancy with have a duration of 20 working days. It will be home-based.

1.Deliverables

The activities, estimated working days and deliverables are as follows:

Activity

Working Days/Expected indicative deadline

Deliverable

Desk review of the working paper and relevant literature and background documents (project documents, Strategic Note of the CO, relevant evaluations in the area, etc.).

2 / 10 November 2020

Interviews/discussions using online platform with key UN Women staff from IES, EVAW team, Senior Management and relevant personnel at Country level; and potentially with other international organizations with substantive experiences in measuring impact.

1 / 16 November 2020

Interview Notes

Develop impact evaluation designs for 4 existing programmes/projects in different thematic areas, including a framework (that applies the building block framework developed in the working paper) using 3-4 existing projects (provided to the consultant by UN Women).

(At least 2 rounds of revisions based on feedback from UN Women)

12 / 4 December 2020

Draft and revised Impact Evaluation Design for 4 projects.

Develop a guidance note no more than 10 pages articulating the suggested steps for impact evaluation (building on the working paper) which incorporates the lessons from piloting/adaptation the building block framework and suggestions on how to adjust the framework, if necessary, to the context of UN Women to make it practical and to be able to apply it to other thematic priorities. (At least 1 rounds of revision based on feedback from UN Women). Provide feedback on the pilot/building block approach detailing challenges, opportunities and critical reflections.

4 / 11 December 2020

Draft and revised guidance note.

Contribution to communication and dissemination of the impact evaluation design and guidance note.

1 / 15 December 2020

PowerPoint presentation of impact evaluation design.

TOTAL

20

Competencies

  • Strong technical knowledge in evaluations
  • Strong knowledge in the field of Gender Equality and Women’s Empowerment; Substantive knowledge of gender-responsive evaluation; demonstrated knowledge of human rights-based approach;
  • Ability to design, manage, and conduct impact evaluations.
  • Ability to implement both quantitative and qualitative data collection methods and triangulation of data
  • Substantive knowledge in evaluating programmes or projects in the thematic area of ending violence against women and girls
  • Excellent ability to communicate with stakeholders

Required Skills and Experience

Education: Master’s degree in a field of relevance for the assignment: Gender, Sociology, Political Science, Anthropology, or other Social Science degree.

Professional Experience: The consultant must have at least 7 years of evaluation experience with minimum 3 years’ experience in designing, managing and undertaking impact evaluations and applying human rights and gender equality-based approaches to evaluation. Must have specific experience in the field of Gender Equality and Women’s Empowerment and Impact Evaluation. Must have demonstrated experience implementing both quantitative and qualitative data collection methods and triangulation of data. Must have evaluation experience in the thematic area of ending violence against women and girls.

  1. Proven work experience in designing and /or leading impact evaluation.
  2. Proven experience in contributing to gender-responsive evaluation or experience in gender analysis and human-rights based approaches.
  3. Proven experience in evaluating programmes or projects in the thematic area of ending violence against women and girls.
  4. Excellent ability to communicate with stakeholders.
  5. Added asset is knowledge of the role of UN Women or the UN system and its programming, coordination and normative roles at the regional and/or country level.
  6. Language proficiency in English.

Application procedure

Interested consultants can submit the following documents to be combined in one Word or PDF document:

  1. CV and UN Women P11
  2. Letter of motivation with professional fee (daily or lump sum) based on the deliverables listed above
  3. Sample evaluation report (with team leader role)
  4. List of 3 professional references that can be contacted

ANNEXES

Annex 1 UN Women Evaluation Consultants Agreement Form

Annex 2 UNEG Norms and Standards for evaluation

Annex 3 UN Women Evaluation Handbook

Annex 4 Building Block Framework (proposed in the working paper)

Annex 5

Indicative Impact Evaluation Framework: Thematic Priority 8 *

BUILDING BLOCKS

ACTIVITIES

OUTPUTS

OUTPUT/PROCESS INDICATORS

OUTCOMES (short/medium-term change in learning & action)

EVIDENCE OF CHANGE

IMPACT

NORMATIVE WORK – the supporting architecture for EVAWG

Political framework

No activities specified in ToC

No outputs specified in ToC

# of high-level commitments on EVAWG

Political will & commitment to EVAWG as a national priority (including prevention, coordinating quality essential services & addressing impunity)

Prioritisation of EVAWG in political agendas, national strategies, development plans, programmes & expenditure frameworks

EVAWG is prioritised

Legal & policy framework

(1.2) Technical assistance to review/reform legislation and policies

(1.2) Support oversight mechanisms to monitor implementation of EVAWG legislation & policy

(2.1) Global framework on the prevention of VAWG is developed, publicised and adapted for national context specific implementation

(1.2) Laws and policies regularly reviewed, reformed & resourced to confirm with international standards & evidence

(2.1) Prevention strategies and action plans in line with international human rights standards developed & implemented by UN and national actors in a coordinated manner

# of laws and policies reviewed, drafted, and reformed to conform with international human rights standards and evidence

# of national actors adapting global frameworks on the prevention of VAWG

Clear messaging that VAWG is not tolerated with increasing awareness & knowledge of legal obligations

Legal & policy reform e.g. objectives for EVAWG integrated as legislation, strategies & plans are updated or developed

EVAWG is increasingly integrated into decision-making

EVAWG including prevention is integrated into national, sector & subnational framework (e.g. strategies, development plans, preparedness plans, action plans)

Harmonisation & linkages of all laws and policies (e.g. family law, migration, disability & poverty)

Availability of normative products e.g. resources for implementation of EVAWG

EVAWG is integrated

Institutional frameworks (including norms & standards)

(1.3) Support to institutions, notably in the police, justice, health and education sectors to translate laws and policies into practice through codes of conduct, risk assessment processes and other tools, reporting mechanisms, awareness-raising programmes etc.

(3.1) Standards and guidelines on service provision across sectors developed and made available, including ensuring women have access to the range of comprehensive services that address physical, mental and sexual and reproductive health

(1.3) Capacity of institutions to implement legislation on EWA and other forms of discrimination strengthened

(3.1) Capacity of service providers to adapt & integrate global standards and guidelines for essential services is enhanced

# of national organisations that have adapted standards and guideline on service provision

# of national and regional human rights institutions provided with capacity development

#/% institutions that have procedures to act on reported cases of violence

EVAWG is Integrated into codes of conduct, institutional procedures & tools e.g. assessment, planning, training, delivery, monitoring and reporting

Changes in awareness, attitudes and knowledge towards EVAWG at the institutional level

Institutional behaviour change underpinned by standards & guidelines e.g. focal points for EVAWG, integration into employee induction programmes

Institutional frameworks include clear arrangements, roles and responsibilities for EVAWG e.g. ministry focal points

Institutional framework integrates EVAWG e.g. institutions have updated codes of practice, clearly defined roles, responsibilities & processes

Standards and guidelines integrating global standards in place e.g. service delivery guidelines

Employee training programmes integrate EVAWG

EVAWG is institutionalized

Financial framework

(1.2) Secure adequate budget allocations to implement laws and policies for EVAWG and to coordinate and improve essential service delivery

(1.2) Laws and policies regularly reviewed, reformed & resourced to confirm with international standards & evidence

% of recurrent & development budget allocated

% of donor & development partner funding allocated

Financial officers & development planners involved in decision-making about EVAWG (e.g. EVAWG included in budget circulars)

Changes to budget allocation & expenditure to ensure adequate, sustained & diverse funding sources/ resources to: i) implement laws, policies and programmes; ii) collect data; iii) build capacity; iv) coordinate & deliver services

Budget allocated for review/reform of policies, plans & essential services

Dedicated financial resources to fund interventions on EVAWG

Funding tracked for EVAWG e.g. budget coding

EVAWG is financed

COORDINATION WORK - the actors involved in EVAWG

Leadership & advocacy

No outputs specified in ToC

No outputs specified in ToC

# of advocacy campaigns

Leadership, advocacy and ownership for EVAWG is increasing within the UN Systems (e.g. UN Women) and nationally (e.g. beyond the ministry for women)

Women’s organisations are increasingly leading and coordinating advocacy & interventions

Leadership of policy dialogue, advocacy campaigns. programmes

Leadership by key stakeholders e.g. women’s organisations, ministry for women

Advocacy campaigns on EVAWG by women’s organisations

EVAWG has leadership at all levels

Partnerships & networks

No outputs specified in ToC

(2.1) Prevention strategies and action plans in line with international human rights standards developed and implemented by UN and national actors in a coordinated manner

# partnerships established for EVAWG

# coordination mechanisms established

Increase in collaboration for enhanced impact

System-wide decision-making, collective-action, coordination & collaboration promoting coherent EVAWG interventions

Enhanced coordination between line ministries, local government, CSOs, development partners, women’s organisations etc., instead of small scale, fragmented, standalone initiatives

Sustained partnerships & networks for EVAWG across countries, levels & sectors

Joint action plans

Peer-to-peer groups & exchanges across countries

Multi-sectoral and multi- stakeholder planning & action

Established networks (e.g. women’s, youth, faith) advocating for EVAWG

EVAWG is coordinated

Mobilisation & capacity of national actors

(1.1) Mobilisation of women’s organisations, parliamentarians, women’s health, education & justice ministries

(1.1) Women’s voice and agency strengthened to advocate for the development and implementation of laws and policies on EVAWG

# national stakeholders mobilised to advocate for EVAWG

# of organisations/ networks strengthened and mobilised to advocate against VAWG/harmful practices

Increase in awareness of EVAWG by decision makers & change in public discourse (e.g. attitudes, perceptions, aspirations, opinions, social norms)

Engagement of key/influential national actors for EVAWG including women’s organisations in decision making

Strengthened capacities of national actors e.g. increase in skills, personnel & focal points leading EVAWG in key institutions (e.g. ministry of education, NDMOs)

EVAWG capacity development strategies and plans

Key national stakeholders (e.g. representative organisations) involved in EVAWG decision making and implementation

Capacity development initiatives/training for national actors

Women’s organisations participate in senior decision-making forums

Women and their representatives are central to decision making

OPERATIONAL WORK – the processes, products & procedures guiding implementation & delivery of EVAWG

Mobilisation & capacity of subnational actors

(1.1) Capacity development, awareness raising and mobilization of women’s organizations…

(2.2) Community mobilisation targeting both men and women, boys and girls, and other stakeholders, including traditional and faith leaders

(2.3) Development of educational curricula and programmes that promote gender equality and human rights, respectful relationships

and non-violent communication skills

(1.1) Women’s voice and agency strengthened to advocate for the development and implementation of laws and policies on EVAWG

(2.2) Women, girls, men and boys at community and individual levels are mobilized in favour of respectful relationships and gender equality

(2.3) Educational curricula & programmes addressing gender equality & VAWG developed & integrated into formal and non-formal education

# women's organisations mobilised to advocate for EVAWG

# of women/girls with increased awareness around laws, etc.

# of educational curricula developed and integrated into institutions

**Strengthened community education****, dialogue & decision making on EVAWG

Increased awareness, knowledge & understanding of human rights, service availability & resources

Change in attitudes, beliefs & social norms at the community & individual level to prevent VAWG

Changes in individual demand for services

Engagement of men, women, boys, girls, traditional & faith leaders, CSOs etc.

VAWG capacity development strategies and plans

Outreach activities by key stakeholders

Stakeholders/communities mobilised and understand rights

EVAWG mainstreamed into employee training (e.g. government ministries)

Peer-to-peer groups established

EVAWG mainstreamed into all primary, secondary & tertiary (e.g. law, medicine) curricula & informal education

Social norms, attitudes & behaviour support EVAWG

Planning and design

No outputs specified in ToC

No outputs specified in ToC

# of interventions designed mainstreaming EVAWG objectives

# of EVAWG interventions designed

# of interventions engaging women in planning & design

EVAWG is integrated into decision making & planning processes and tools

Prevention is integrated into the design of comprehensive EVAWG programmes

Programmes designed with the participation of key stakeholder (e.g. women/children) & aligned with international EVAWG standards

Development/sectoral planning processes & tools integrate EVAWG (e.g. project risk assessments)

Joint planning of multi-sectoral programmes that prevent and respond to VAWG

Women are central to decision making & planning

Implementation

(2.2) Specific focus on groups of

women and girls who face multiple forms of discrimination, use of the media, theatre and other forms of entertainment, outreach by civil society, set of peer-to-peer groups, finding influential ‘champions’

(3.4) Awareness raising and outreach efforts on access to essential services and on laws & resource mechanisms through grassroots women and women’s organisations, media campaigns & public information available at police, justice, health social services & education facilities

(3.2) Training to service providers to meet level of quality service, and technical areas (including gender

equality and data collection and analysis), peer-to-peer learning, and exchanges across countries

(3.3) Geographic, including in urban and remote areas, linguistic and financial accessibility of essential services, particularly for vulnerable /marginalized women, including

indigenous, women living with HIV/AIDS etc.

(2.2) Women, girls, men and boys at community and individual levels are mobilized in favour of respectful relationships and gender equality

(3.4) Women understand and can exercise their rights to quality services

(****3.2) Capacity of service providers to deliver quality, coordinated essential services, to hold perpetrators to account, and the collection and use of data in an ethical manner are strengthened

(****3.3) Availability and accessibility of quality essential services for women and girls who have been subject to violence improved, and in the case of gender-related killings, support to victims’

families is strengthened

# of women and girls with access to acceptable, quality services

# of service providers with capacity to provide coordinated, quality services

# of programmes promoting EVAWG

% of prosecuted VAWG cases that result in conviction

Women and girls who experience violence are aware of existence, have access to the range of comprehensive services (e.g. addressing physical, mental, sexual & reproductive health) and are more confident using available services

Change in service provision including accessibility (geographically, linguistically and financially), multisectoral, coordinated, consistent and quality provision

Media campaigns

Information campaigns

Demand for support services including from women/girls

who experience multiple forms of discrimination & gender in-equality

Delivery of multi-sectoral/level/stakeholder programmes that prevent and respond to VAWG

Service providers institutionalise capacity development for employees

Provision in all geographic areas (e.g. rural & urban)

Behaviour & practice transformed

Data collection, monitoring, evaluation

* Capacity development of National Statistical Offices on collection, management and sharing/reporting of VAWG data

No activities specified for M & E in ToC

No outputs specified in ToC

# Initiatives informed by VAWG data

# of policies and programmes monitored for impact

NSOs/service providers are collecting & analysing VAWG data (e.g. prevalence surveys, analysis of administrative data)

**EVAWG data is centrally stored****, analysed & shared

Decision making & planning is informed by EVAWG data

Evidence-based programmes are implemented

Strengthened, quality & comparable data on VAWG/harmful practices

Increase in evidence base on what works for preventing VAWG

Budget allocated for data collection and information knowledge management

National indicators & targets agreed for EVAWG

Action on EVAWG is informed & monitored

*Action identified in narrative for TP8 rather than in the ToC

Note: wording in the table in italics are not directly from TP8 ToC.

Added 3 years ago - Updated 3 years ago - Source: jobs.undp.org