International Consultant for Mid-Term Project Evaluation

Consultant

Contract

This is a International Consultant contract. More about International Consultant contracts.

Background

Bosnia and Herzegovina (BiH) is a sovereign state with a decentralized political and administrative structure. The Constitution of BiH, an Annex to the General Framework Agreement for Peace in BiH (Dayton Peace Accords, 1995) confirmed the continuation of the legal existence of BiH as a country, while its internal structure was changed. In accordance with the Constitution, BiH comprises two entities: Republika Srpska (RS) and the Federation of Bosnia and Herzegovina (FBiH); and Brcko District as a separate administrative unit as decided by an arbitrary decision made on 5 March 1999 (Annex 2, Article 5 of the Dayton Peace Agreement), whereby it was formed under the exclusive sovereignty of the state.

The central government at the level of BiH received limited powers in accordance with the Dayton Peace Accords (1995), as all governmental functions and authorities that are not expressly assigned to the institutions of BiH in the Constitution, are those of entities.

FBiH is sub-divided into 10 Cantons with 79 local governments (municipalities and cities). While the RS administrative structure includes 64 municipalities and cities, more specifically – 57 municipalities and 7 cities. Municipalities and cities with local self-governance are the lowest level of the political and territorial structure of BiH.

When it comes to environmental authorities, the Ministry of Foreign Trade and Economic Relations of BiH (MoFTER) is tasked with defining policies, basic principles, coordinating activities and harmonizing plans of the entity authorities and institutions at the international level.

In FBiH, the Ministry of Environment and Tourism oversees strategy and policy of environment protection including air protection, water and soil conservation, waste management, and environmental standards. The Ministry of Spatial Planning and the Ministry of Agriculture, Water Management and Forestry have respective environmental competences.

In RS, the Ministry for Spatial Planning, Civil Engineering and Ecology is tasked with environmental protection (land, air and water) and waste management. The Ministry of Agriculture, Forestry and Water Management addresses related strategies, policies, standards and regulations.

BiH ratified the Stockholm Convention on 30 May 2010 and committed to meeting the requirements of the Convention, with the main objective to take measures for the elimination or restriction or prevention of the production, import, export and use of all manufactured POPs and the continuous reduction to minimize the occurrence of these pollutants in the environment, and the elimination of emissions of unintentionally produced POPs. These substances pose a global threat to the environment and human health due to their long persistence in the environment, their long term and cumulative toxic properties, and their capacity to bioaccumulate in living organisms, so the overall objective of the Project is to reduce risk for people’s health and the environment.

UNDP’s Environmentally Sound Management of Persistent Organic Pollutants (POPs) in industrial and hazardous waste sectors (POPs Project) intends to prevent the release of POPs in the environment through the implementation of green chemistry initiatives in the industry and agriculture along with destruction of identified POPs waste stockpiles.

About the Project

Project title

Environmentally Sound Management of Persistent Organic Pollutants (POPs) in industrial and hazardous waste sectors

Atlas ID

00118429

Corporate outcome and output

00115254 and 00121745

Country

Bosnia and Herzegovina

Date Project document signed

01/06/2019

Project End date

01/06/2024

Project budget

USD 6,539,562.17

Project expenditure at the time of evaluation

3,145,519.11 USD

2,371,453.16 USD - Output ID 0011525; Donor 00555

95,279.51 USD - Output ID 0011525; Donor 00067

678,786.44, USD - Output ID 0012174; Donor 00555

Funding source

Government of Sweden, Fund ID 30000

Fund for Environment Protection and Energy Efficiency of RS,

Fund ID 30071

Implementing party

UNDP

The five-year Project “Environmentally Sound Management of Persistent Organic Pollutants (POPs) in Industrial and Hazardous Waste Sectors” (POPs Project) is financed by the Government of Sweden and implemented by UNDP. The Project’s budget amounts to USD 6,539,562.17 including co-funding from the Fund for Environment Protection and Energy Efficiency of RS.

The main goal of the Project is to support elimination and reduction of releases of persistent organic pollutants (hereinafter POPs) into the environment, which is in line with requirements of the Stockholm Convention (hereinafter the Convention). BiH signed the Convention in 2001 and ratified it in 2010. By doing so, the country obliged to fulfil the requirements of the Convention which, inter alia, include avoidance of the use of hazardous POPs, shifting towards safer alternatives and removal of old supplies and equipment that contain these substances. The Convention lists 28 POPs in total, whereas five POPs had been added after the Seventh Conference of the Parties held in 2017.

The Project has the following components:

• Component 1: Capacity Building and mainstreaming of POPs related legislation into the process of harmonization of the BIH environmental legislation,
• Component 2: Prevention and monitoring of Unintentional POPs (U-POPs) generation and of release of POPs through minimization, segregation, and environmentally sound management of selected hazardous waste,
• Component 3: Implementation of green chemistry principles in plastic manufacturing to prevent the use of and release of new POPs including the candidate deca Polybrominated diphenyl ethers (PDBE) and Short-chain chlorinated paraffins (SCCP),
• Component 4: Management and disposal of PCBs and POPs from abandoned industrial premises,
• Component 5: Monitoring, Learning, adaptive feedback, outreach and evaluation,
• Component 6: Development of the inventory of cooling equipment which create ozone depleting substances -ODS), and
• Component 7: Responsible management of potentially infectious waste in relation to the COVID-19 public health crisis[1] .

The Outcomes of the Project are:

• Outcome 1.1 POPs related legislation mainstreamed into the process of inter-institutional and EU harmonization of the BIH environmental legislation and POPs inventory/registry developed
• Outcome 2.1. Around 2 Grams of the toxic equivalent I-TEQ per year (g TEq) of Dioxins (PCDD/F) release avoided through the establishment of capacity for the proper segregation and management of waste generating U-POPs.
• Outcome 2.2 Capacity for monitoring of POPs and U-POPs in the environment and at the originating sources established.
• Outcome 2.3. Prevention of illegal import or marketing of chemicals of goods containing or contaminated by POP substances.
• Outcome 3.1 Green chemistry principles adopted in the emerging plastic manufacturing sector with the avoidance of the use of at least 10 tons of PBDE, deca PBDE and short chain chlorinated paraffins.
• Outcome 4.1 Risk assessment conducted, and remediation designs developed for at least two contaminated sites.
• Outcome 5.1 Project results monitored, adaptive management applied, lessons-learned, experiences, and best practices extracted and disseminated at BiH wide and regional level.
• Outcome 6.1. Emissions of ozone depleting substances (ODSs) avoided through the establishment of capacity for proper segregation and management of appliances and equipment containing ODSs.
• Outcome 7.1 Authorities are supported to provide inclusive and multi-sectoral crisis management and response to COVID-19, particularly regarding medical waste management.

The Outputs of the Project are:

• Output 1.1.1 Coordination structure for the implementation of the convention (horizontal and vertical) established to ensure that POPs related legislation is mainstreamed into the process of inter-institutional and EU harmonization of the BIH’s environmental legislation.
• Output 1.1.2. Stockholm Convention mainstreamed in the environmental legislation of the BiH decision 2 entities and Brcko district. POPs related decrees drafted and approved.
• Output 1.1.3 Training on the integration of the Stockholm Convention with the EU and BiH wide legislation on chemical and waste for environmental decision makers carried out.
• Output 1.1.4 A database on POPs, containing also information on new POPs not fully addressed in the NIP is developed and made available to stakeholders and listed in database. At least three major abandoned industrial sites inspected and included in Inventory on POPs.
• Output 1.1.5 At least 50 tons of POPs containing waste or equipment disposed of.
• Output 2.1.1 Two (2) g/TeQ of PCDD/F releases avoided through the implementation of environmentally sound management of plastic waste contaminated by pesticides.
• Output 2.2.1 At least one laboratory trained on the sampling and analysis of POPs and U-POPs in the environment and at the stack of industrial sources.
• Output 2.3.1 Development of a manual for the Customs for the prevention of illegal import of POPs chemicals or POPs containing mixtures or goods.
• Output 3.1.1 Training on green chemistry in plastic manufacturing carried out for at least 50 participants
• Output 3.1.2 Non-POP alternative to flame retardants introduced in plastic manufacturing with the replacement of at least 5 t of C-PBDE and at least 5 t of SCCP yearly.
• Output 3.1.3 Development of incentive mechanisms to ensure sustainability and replicability of GC initiative in the manufacturing industry.
• Output 4.1.1. Risk assessment conducted, and remediation designs developed for at least 2 POPs contaminated sites.
• Output 5.1.1 Adaptive management applied, lessons-learned, best practices and experiences collected and disseminated at BiH wide and regional level to support replication.
• Output 6.1.1 Emissions of ODSs prevented and management of ODSs improved.
• Output 7.1.1: Households with COVID-19 positive members or those in self-isolation dispose of their waste responsibly and safely, while the general population is aware of the responsible behavior in terms of waste disposal during the public health crisis.
• Output 7.1.2: Current medical waste management practices and facilities in health care institutions in the most COVID-19 affected areas are rapidly assessed and capacitated to accommodate the newly generated waste, as well as prepare the system for events of similar nature in the future.
• Output 7.1.3: Public utilities for waste management are capable to effectively and safely provide waste services in the public health crisis.

Detailed outline of the Programme Result Framework is available in Annex 1

Partnerships:

The Project is implemented in partnership and collaboration with respective partner institutions from the Government: Ministry of Foreign Trade and Economic Relations of BiH (MoFTER); Ministry for Spatial Planning, Construction and Ecology of RS, Ministry of Environment and Tourism of the FBiH, Environment Protection in the Department for Spatial Planning and Property Affairs in Brcko District, Environmental Fund of the FBiH, and Environmental Protection and Energy Efficiency Fund of RS.

Overview of key stakeholders and partners and their roles in evaluation is provided in Annex 2.

Target groups and beneficiaries:

In addition to government institutions and civil servants, the beneficiaries of the Project are in both the public and private sectors, including the general public, vulnerable groups, workers, educators, NGOs. Target groups also include business entities and medium-sized enterprises in BiH dealing with waste management, export of hazardous waste and manufacturers of equipment that contains POPs.

Main achievements:

• Gap Analysis of Institutional and Legal Readiness to implement the Stockholm Convention in BiH,
• Project/Design of remediation and recultivation for one contaminated site in BiH,
• Procurement of most urgent equipment needed for proper and safe infectious waste management,
• Feasibility Study on Implementation of Container Management System in BiH,,
• Detailed Inventory of persistent organic pollutants (POPs) in BiH,
• Manual on POPs detection for customs officers.

Covid 19 context:

Starting from March 2020, the Project’s implementation was to some point negatively affected by the global outbreak of the COVID-19 pandemic. The COVID-19 imposed lockdown resulted in a temporary halt of the activities in the field, which caused delays in timely completion of some of the activities.

On the other side, the additional Project Component - Responsible management of potentially infectious waste in relation to the COVID-19 public health crisis, was approved and became operational as of mid-June 2020 due to sudden and emergency nature of crisis caused by COVID -19.

The intervention aimed to support authorities to offer an effective response to the mismanagement of medical waste in BiH, including potentially infectious one, during and immediately after the 2020 COVID-19 outbreak. The activities had been focused to: household waste that originated from infected or potentially infected persons as well as household waste from single-use masks and gloves disposal, and medical waste generated in medical care facilities during the public health crisis. The intervention targeted leverage points in the entire “life cycle” of above listed waste categories starting from: (a) addressing the two main sources of its generation including households (and individuals) and medical care facilities, further to (b) its collection and disposal by waste management public utilities and other waste management operators; and ultimately to (c) improving the overall capacities for coordination among actors in charge of the medical waste management system to better respond to the rising amounts of medical waste during crisis. The intervention concluded in 2021.

Project alignment and relevance:

The Project is in line with the requirements set by international agreements signed and ratified by BiH. Other than the Stockholm Convention, both the Rotterdam Convention on the prior notification procedure with approvals for trade of specific dangerous chemicals and pesticides in international trade, Basel Convention on the control of transboundary movements of hazardous wastes and their disposal and the Convention on Long-Range Transboundary Air Pollution (CLRTAP) and respective Protocol on Long Term Financing of the Co-operative Programme for Monitoring and Evaluation of the Long-range Transmission of Air Pollutants in Europe (EMEP), are relevant[2].

The implementation of Project’s activities also contributes to BiH’s implementation of the following Sustainable Development Goals:

• SDG 12 Responsible Consumption and production, Target 12.4 and Target 12.8
• SDG 3 Good health and well-being Target 3.9
• SDG 12 Responsible Consumption and production, Target 12.4 and Target 12.8
• SDG 17 Partnerships for the Goals, Target 17.7 and Target 17.9

Evaluation purpose, objectives and scope

Purpose

The main purpose of this Mid-term Evaluation (MTE) is to assess progress towards the achievement of the Project’s outputs/outcomes (as per the Project result framework) and identify potential challenges in Project implementation so far. It will assess intermediate signs of Project success or failure with an aim of recommending eventual course corrections in the second half of the Project lifetime and, if necessary, set the Project on-track in order to increase the probability for achieving its intended results by the end of its duration. The MTE will also review the Project’s implementation strategy, and its risks to sustainability. The MTE will assess the relevance, coherence, efficiency, effectiveness, sustainability and impact of the Project, and provide strategic recommendations for future decision-making in the sectors covered by the Stockholm Conventions, for institutional partners, UNDP, Government of Sweden and other relevant stakeholders.

[1] This Component was completed in 2021.

[2] The following conventions and protocols also bear relevance: Convention for the Protection of the Ozone Layer Vienna (1993), Protocol to the Vienna Convention on Substances that Deplete the Ozone Layer Montreal (1993); and Aarhus Protocol to CLRTAP on Persistent Organic Pollutants (POPs) and the Minamata Convention on Mercury.

Duties and Responsibilities

The MTE will focus on assessing the overall performance of the Project and its results generated from the beginning to the mid-point of the Project implementation, based on the scope and criteria set forth in this term of reference.

In a substantive review of the effectiveness of the Project approach, the MTE should assess cause and effect relations within the Project, identifying the extent to which the observed changes can be attributed to its interventions. The selected Evaluation Team will take a broad overview of the Project area by gathering perceptions, aspirations, feedback and data from relevant partners, stakeholders and beneficiaries for objective analysis and conduct of the evaluation.

Finally, this MTE’s objective is to provide forward-looking recommendations to the Government of Sweden and UNDP on the sustainability of the Project results and its scaling up potentials.

Scope

The evaluation will assess the extent to which the planned Project outcomes and outputs have been achieved since the beginning of the Project in June 2019 and likelihood for their full achievement by the end of the Project in June 2024 (based on the Project Document and its results framework).

The MTE will look into the Project’s processes and activities, strategic partnerships and linkages in the specific country’s context that proved critical in producing the intended outputs and the factors that facilitated and/or hindered the progress in achieving the outputs, both in terms of the external environment and risks, crisis caused by the pandemic, as well as internal, including weaknesses in programme design, management and implementation, human resource skills, and resources.

The evaluation will also assess the cross-cutting aspects of the Project, such as gender equality and human rights and innovativeness in result areas.

To the extent possible, the MTE will also consider the results of the Project’s contribution to address the effects of the COVID-19 pandemic.

I.Evaluation criteria and key questions

The MTE of the POPs Project will address the following questions, so as to determine the Project’s relevance, coherence, effectiveness, efficiency, impact and sustainability, including lessons learned and forward-looking recommendations:

Relevance and coherence

• Are the Project’s objectives relevant to the needs and priorities of the country and beneficiaries, having in mind political, social, legal and institutional context of the country, effective national policies and strategies?
• Have any changes been made to the Project’s design during the implementation? If yes, did they lead to significant design improvements? Were adequate steps taken by the Project to adjust its implementation strategy to the new circumstances and needs imposed by COVID-19 pandemic relevant?
• Are coordination, management and financing arrangements clearly defined and did they support institutional strengthening and local ownership?
• To what extent were human rights, gender equality and social inclusion mainstreamed within the Project? Has this mainstreaming been relevant to the needs of socially excluded groups and both women and men?
• To what extent has the Project been successful in ensuring complementarity, harmonisation and coordination with other relevant interventions of the governments in BiH and other donors, avoiding duplication of efforts and adding value?

Effectiveness

• To what extent have the intended results been achieved? What are the main Project’s accomplishments? Overview of the Project’s progress against the result framework indicators is to be provided in an Annex of the Evaluation Report.
• Briefly explain the reasons behind the success (or failure) of the Project in producing its different outputs and meeting expected quality standards? Were key stakeholders appropriately involved in producing the programmed outputs?
• To what extent has the Project supported effective capacity building for monitoring, management and disposal of POPs, including harmonization of relevant legislation?
• What good practices or successful experiences or transferable and innovative examples have been identified? Are there any course correction measures to be undertaken by the end of the project?
• To what extend has the Project outreached marginalized groups (i.e. youth, persons with disabilities, returnees, internally displaced, minorities…)? Have the interventions been implemented in accordance with a civic and human rights perspective: i.e. have target groups been participating in planning, implementation and follow up? Has anyone been discriminated by the Projects through the implementation? Have the Projects been implemented in a transparent fashion? What accountability mechanisms have been applied in the Projects?

Efficiency

• Have resources (financial, human, technical) been allocated strategically and economically to achieve the Project’s results?
• Has co-funding provided by additional donors ensured complementarity with the overall Project intervention? Were the Project’s activities implemented as scheduled and with the planned financial resources? Is the relationship between Project’s inputs and results achieved appropriate and justifiable?
• To what extent did the Project engage or coordinate with different beneficiaries (men and women), implementing partners and national counterparts to achieve outcome-level results? To what extent were the Project coordination approaches conducive to the delivery of the Project outputs?
• Has the communication and outreach of the Project been satisfactory?
• Did the Project have a sound M&E plan to monitor results and track progress towards achieving Project objectives?

Impact

• What is the Project impact in qualitative as well as quantitative terms from a broader development and system building perspective? What would the development have been like without the Project’s interventions in the area of concern?
• What are the positive or negative, intended or unintended, changes brought about by the Project’s interventions?
• To what extent the Project’s intervention may have led to adaptive change and paradigm shift towards resilient development pathways?

Sustainability

• To what extent are the achieved outputs sustainable? Will the outputs lead to benefits beyond the lifespan of the Project?
• To what extent has the Project contributed to sustainability of its achievements, by supporting nationally-owned strategic plans, legislative agendas and policies?
• To what extent do national partners have the institutional capacities, including sustainability strategies, in place to sustain the Project results? Are policy and regulatory frameworks in place to support the continuation of benefits for institutional beneficiaries and people in the future?
• Is the Project financially catalytic? To what extent have partners committed to providing continuing support?
• To what extent do partnerships exist with other national institutions, NGOs, United Nations agencies, the private sector and other development partners to sustain the attained results?
• What are the innovations/ best practices generated by the Project, that need to be further built upon?
• To what extent has the integration of human rights and gender led to an increase in the likelihood of sustainability of Project results?
• Does Project have well-planned exit strategies?

Human Rights and Gender Equality

• To what extent have the Project interventions been inclusive in supporting the most vulnerable and marginalized group in the implementing area?
• To what extent have gender equality and the empowerment of women been mainstreamed in the Project design and implementation? Has the Project had any positive or negative effects on gender equality?
• Could gender mainstreaming have been improved in planning, implementation or follow up?

Future-looking concept and recommendations

It is critical for the evaluation to balance its contribution to collective learning, with greater focus on adaptive management and systemic change, accountability over the use of public resources, considering the strategic context and the authorizing environment. With that view, in the forward-looking recommendations, the MTE will also consider:

• Have any good practices, success stories, lessons learned, or transferable examples been identified, that would eventually suggest any course correction in the future implementation? Please describe and document them.

The evaluation needs to assess the degree to which the Project supported or promoted gender equality, a rights-based approach, and human development. In this regard, United Nations Evaluation Group’s guidance on Integrating Human Rights and Gender Equality in Evaluation should be consulted.

II.Methodology

Based on the UNDP Evaluation Guidelines and UNEG Norms and Stand for Evaluations and in consultations with the UNDP, the Evaluation will be participatory, involving relevant stakeholders.

The Evaluation will be conducted by the Evaluation team composed of an International Evaluation Consultant (Evaluation Team Leader) and National Evaluation Consultant. The Evaluation Team Leader will lead the evaluation process and decide on planning and distribution of the evaluation workload and tasks. She/he will closely collaborate with the National Evaluation Consultant who will provide support throughout the evaluation process

The Evaluation Team Leader twill propose an adjusted evaluative methodology that may be needed to implement the evaluation effectively in the COVID – 19 pandemics circumstances, applying safety guidance and remote data collecting methods such as extended desk reviews, virtual stakeholder meetings and interviews by Evaluators[1]. A detailed plan for the Evaluation process will be proposed by the Evaluation Team Leader and agreed as a part of the Evaluation Inception Report.

The proposed methodology should employ innovating participatory approaches, relevant quantitative, qualitative or combined methods to conduct the Evaluation, based on diverse ecosystem of evidence, using gender sensitive data collection and analytical methods and tools applicable in the concrete case. The Evaluation team is expected to combine the standard and other evaluation tools and techniques to ensure maximum reliability of data and validity of the evaluation findings. These methods and approaches need to generate feedback loops and insights for transformational change. Stakeholder participation is an important source of data which can mitigate observational biases. The Evaluation recommendations will be forward looking and focused on adaptation in the changing system addressed by the project intervention.

Limitations to the chosen approach/methodology and methods shall be made explicit by the Evaluation team and the consequences of these limitations discussed in the proposed methodology. The Evaluation Team Leader shall, to the extent possible, present mitigation measures to address these limitations.

The Evaluation Team Leaderis expected to carry out the evaluation process with careful consideration of these Terms of References. In cases where sensitive or confidential issues are to be addressed in the evaluation, the Evaluation Team Leader should ensure an evaluation design that do not put informants and stakeholders at risk during the data collection phase or the dissemination phase.

Standard UNDP evaluation methodology would suggest the following data collecting methods:

• Desk review: The Evaluation Team Leader will conduct a detailed review of the Project materials and deliverables including but not limited to the Project Document and Addendums, theory of change and results framework, monitoring and Project quality assurance reports, annual workplans, consolidated progress reports etc. An extensive list of documents for desk review is provided in Annex 3.
• Key informant interviews: Using virtual technological solutions, the Evaluation Team Leader will remotely interview representatives of UNDP MoFTER; Ministry for Spatial Planning, Construction and Ecology of RS, Ministry of Environment and Tourism of the FBiH, Environment Protection in the Department for Spatial Planning and Property Affairs Brcko, Environmental Fund of the FBiH, and Environmental Protection and Energy Efficiency Fund of RS etc. Detailed list of main stakeholders that may be considered for meetings is provided in Annex 2.
• Other methodologies, as appropriate, such as case studies, statistical analysis, social network analysis, etc. online interviews, mobile questionnaires, online surveys, and collaboration platforms (slack or yammer) are recommended to be used to gather data. Stakeholders that are dealing with existing emergencies should be given advance notice.

As an integral part of the evaluation report and specifically under the impact criteria, the Evaluation Team Leader will review the Project’s effects and impact on the target groups. In this context and using the online tools, the consultancy is expected to gain insights from both the partners and the beneficiaries.

The expected duration of the assignment is up to 30 workdays in the period April – June 2022.

III.Evaluation tasks / deliverables

Following the initial briefing and a detailed desk review, the Evaluation Team Leader will be responsible for delivering the following products and tasks:

• Inception Report (10-15 pages) will be presented before the evaluation starts, showing how each evaluation question will be answered by proposing methods, sources of data and data collection procedures. The Inception Report should elaborate an evaluation matrix (provided in Annex 4) for the Project and propose a schedule of tasks, activities and evaluation deliverables. The Evaluation Inception Report should follow the structure proposed in the UNDP Evaluation Guidelines, p. 27

• Evaluation and data collection: Upon the approval of the Inception Report and the evaluation work plan by the UN team, the Evaluation Team Leader is expected to carry out the evaluation. D****ata collecting methodology presented in the Evaluation Inception Report should limit the exposure of any consultant, Project team member, beneficiary or stakeholder to the pandemic, therefore, strongly recommended is use of remote and virtual methodologies.

• Draft Evaluation Report: Based on the findings generated through desk review and data collection process, the Evaluation Team Leader will prepare and submit the Draft Evaluation Report to the UN team and key stakeholders for review. The Evaluation findings, lessons learned and forward-looking recommendations will be separately presented in distinct sections of the Evaluation Report. Structure of the Report is outlined in Annex 5.

• Evaluation review process (and eventual dispute settlement): Comments, questions, suggestions and requests for clarification on the evaluation draft will be submitted to the Evaluation Team Leader and addressed in the agreed timeframe. The Evaluation Team Leader should reply to the comments through the evaluation audit trail document[2]. If there is disagreement in findings, these should be documented through the evaluation audit trail, while effort should be made to come to an agreement.

• Evaluation debriefing: will be held with UNDP team, institutions’ representatives and other key stakeholders to present the main findings and recommendations in an online form (i.e. Skype/Zoom/Microsoft Teams briefing). In addition, short briefings on immediate findings with UNDP senior management and the Government will be considered after completion of the initial assessment.

• Evaluation Report (maximum 40 pages of the main body) should be logically structured (structure of the Evaluation Report is outlined in Annex 5 of the Terms of Reference), contain data and evidence-based findings, conclusions, lessons learnt and recommendations, and be presented in a way that makes the information accessible and comprehensible. Finally, based on the evaluation findings and in a distinct report section, the Evaluation Team Leader will provide a forward-looking actionable recommendations, outlining key strategic priorities to be addressed after completion of the Project in terms of policy dialogue and the work influenced by UNDP, Government of Sweden and follow-up activities by the governments and public institutions in BiH. The executive summary should be maximum 3 pages.

UNDP Evaluation Guidelines Note: As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. If it is not possible to travel to or within the country for the evaluation then the evaluation team should develop a methodology that takes this into account, conduct of the evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, surveys and evaluation questionnaires. This should be detailed in the Inception report and agreed with the Evaluation Manager.

If all or part of the evaluation is to be carried out virtually then consideration should be taken for stakeholder availability, ability or willingness to be interviewed remotely. In addition, their accessibility to the internet/ computer may be an issue as many government and national counterparts may be working from home. These limitations must be reflected in the evaluation report.

*If a data collection/field mission is not possible then remote interviews may be undertaken through telephone or online (skype, zoom etc.). International consultants can work remotely with national Evaluator support in the field if it is safe for them to operate and travel**. No stakeholders, consultants or UN staff should be put in harm and the safety is the key priority.***

IV.Evaluation timeframe

Deliverable

Anticipated timing

Number of days

Responsible party

Desk review and Inception Report

1 – 8 April, 2022

5

Evaluation Team

Field data collection

8 April – 22 April, 2022

12

Evaluation Team

Evaluation debriefing/presentation

5 May 2022

1

Evaluation Team

Draft Evaluation Report

6 – 16 May 2022

8

Evaluation Team

Report review (including several rounds of feedback)

16 May – 20 June, 2022

0

Evaluation Reference Group[3]

Final Report

30 June 2022

4

Evaluation Team

In line with the UNDP’s financial regulations, when determined by the Country Office and/or the consultant that a deliverable or service cannot be satisfactorily completed due to the impact of COVID-19 and limitations to the evaluation, that deliverable or service will not be paid. Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultant invested time towards the deliverable but was unable to complete to circumstances beyond his/her control.

V.Evaluation team composition and required competencies

The evaluation will be conducted by the Evaluation Team composed of an International Evaluation Consultant/Evaluation Team Leader and National Evaluation Consultant. The Evaluation Team Leader will lead the evaluation process and decide on planning and distribution of the evaluation workload and tasks. She/he will design and implement the evaluation process and will closely collaborate with the National Consultant who will provide support throughout the evaluation process.

The Evaluation Team Leader will report to the Evaluation Manager appointed by UNDP, who will oversee and support the overall evaluation process. In addition, an evaluation reference group will be formed to provide critical and objective inputs throughout the evaluation process to strengthen the quality of the evaluation. The Country Office Senior Management will take responsibility for the approval of the evaluation report. UNDP will support the implementation of remote/ virtual meetings. An updated stakeholder list with contact details (phone and email) will be provided by the Country office to the evaluation team.

[1] UNDP Evaluation Guidelines: Evaluation During COVID-19.

[2] Template available at http://web.undp.org/evaluation/guideline/documents/PDF/UNDP_Evaluation_Guidelines.pdf, p. 25

[3] Steering Committee members and Participating UN agencies, UNDP Evaluation Manager, UNDP EE Sector Leader, UNDP Project Coordinator.

Competencies

Core values:

• Demonstrates integrity and fairness by modelling UN values and ethical standards;
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability.

Core competencies:

• Demonstrates professional competence to meet responsibilities and post requirements and is conscientious and efficient in meeting commitments, observing deadlines and achieving results;
• Results-Orientation: Plans and produces quality results to meet established goals, generates innovative, practical solutions to challenging situations;
• Communication: Excellent communication skills, including the ability to convey complex concepts and recommendations, both orally and in writing, in a clear and persuasive style tailored to match different audiences;
• Team work: Ability to interact, establish and maintain effective working relations with a culturally diverse team;
• Client orientation: Ability to establish and maintain productive partnerships with national partners and stakeholders and pro-activeness in identifying of beneficiaries and partners’ needs and matching them to appropriate solutions.

Required Skills and Experience

Qualifications/Education:

• Minimum Master’s degree in environmental and ecological sciences (e.g. biochemistry, chemistry, technology) or other related disciplines;

Experience:

• Minimum 7 years of relevant experience Project and programme evaluations;
• Knowledge of UN/UNDP monitoring and evaluation policies and guidelines;
• Experience working in or closely with UN/UNDP is preferred;
• Sound knowledge of results-based management systems, and monitoring and evaluation methodologies;

Languages Requirements:

• Fluency in English language; knowledge of local languages of BiH will be taken as asset.

Other:

• Deep understanding of the development context in BiH and preferably understanding of environmental priorities and challenges, industrial waste management issues within the country context;
• Understanding and knowledge of the political and administrative context in BiH is an asset.

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The Evaluation Team Leader shall safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The Evaluation Team Leader must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners. The Evaluation Team Leader must be free from any conflict of interest related to this evaluation.[1]

Implementation arrangements and reporting relations

The Evaluation Team Leader will report to the Evaluation Manager appointed by UNDP, who will oversee and support the overall evaluation process. An evaluation reference group will be formed to provide critical and objective inputs throughout the evaluation process to strengthen the quality of the evaluation. The UNDP Senior Management will take responsibility for the approval of the evaluation report. UNDP team will support the implementation of remote/ virtual meetings.. An updated stakeholder list with contact details (phone and email) will be provided by the Project to the evaluation team.

TOR annexes

• Annex 1. Project Logical Framework and Theory of Change
• Annex 2. List of the main stakeholders and their roles in evaluation
• Annex 3. List of documents to be considered for the evaluation desk review
• Annex 4. Required Evaluation Matrix Template
• Annex 5. Standard outline for an evaluation report
• Annex 6. Code of Conduct
• Annex 7. Link to UNDP Evaluation Guidelines and Evaluation Quality Assessment Process

[1] UNDP Evaluation Guidelines, Box 7. Sources of conflict of interest in evaluation

Qualification Requirements

Criteria

Points

Max. Points

Relevant education

Max 25 points (20 points for MSc/MA

+ up to 5 points for PhD)

25

Relevant professional experience

Max 70 points

70

Knowledge of English

Max 5 points - will be assessed as:

5 points for fluency and the points decrease as per the level mentioned in the CV: good - 4 points;

fair/upper intermediate – 3 points; intermediate - 2 points; beginner - 1 point.

5

Total

100

Only candidates obtaining a minimum of 70 points would be considered for Technical Evaluation

Technical Evaluation

Criteria

Weight

Max. Point

Technical

Total technical 100%

Criterion A:

• Rating based on Qualifications

20%

20

Criterion B:

• Sound knowledge of results-based management systems, and gender-sensitive monitoring and evaluation methodologies;
• Excellent knowledge of monitoring and evaluation methodologies; sound judgment and ability to objectively evaluate Projects in terms of processes, as well as results achieved (evidenced through previously conducted evaluations and references).
• Sound knowledge of results-based management systems, and gender-sensitive monitoring and evaluation methodologies;
• General understanding and knowledge of the political and administrative context in BiH.

50%

50

Criterion C:

• Evaluation Methodology Proposal (outlining the specific design and methods for the evaluation):
  • Explaining why they are the most suitable for the work;
  • Providing a brief methodology on how they will approach and conduct the work;
  • Presenting the Consultant’s approach, proposed detailed methods and tools, scope and evaluation criteria and questions;

The methodology should apply a mixed-method approach collecting both quantitative and qualitative data to validate and triangulate data;

The methodology should include the filled in evaluation matrix (Annex 4);

30%

30

Only candidates obtaining a minimum of 70 points would be considered for the Financial Evaluation

Evaluation

Individual will be evaluated based on the following methodology:

Cumulative analysis

When using this weighted scoring method, the award of the contract should be made to the candidate whose offer has been evaluated and determined as: a) responsive/compliant/acceptable, and b) Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

• * Technical Criteria weight-70%
• * Financial Criteria weight- 30%

Interested candidated must submit the following documents/information to demonstrate their qualifications:

1. Most recent CV, including reference to similar evaluations conducted by the candidate;
2. Financial proposal (to be submitted separately);
3. Evaluation Methodology Proposal (outlining the specific design and methods for the evaluation):

• Explaining why they are the most suitable for the work;
• Providing a brief methodology on how they will approach and conduct the work;
  • the methodology should present the Consultant’s approach, proposed detailed methods and tools, scope and evaluation criteria and questions;
  • the methodology should apply a mixed-method approach collecting both quantitative and qualitative data to validate and triangulate data;
  • the methodology should include the filled in evaluation matrix (Annex 4).

Please scan all above mentioned documents and upload as one attachment only online through this website.

Note:

• For an assignment requiring travel, consultants of 65 years or more require full medical examination and statement of fitness to work to engage in the consultancy.
• Due to large number of potential applicants, only competitively selected candidates will be contacted for remaining steps of the service procurement process.

Apply now

Added 2 years ago - Updated 2 years ago - Source: jobs.undp.org