International Evaluator - Terminal Evaluation for UNDP-supported GEF-Financed Project

Consultant

Contract

This is a International Consultant contract. More about International Consultant contracts.

Background

Introduction

In accordance with UNDP and GEF M&E policies and procedures, all full- and medium-sized UNDP-supported GEF-financed projects are required to undergo a Terminal Evaluation (TE) at the end of the project. This Terms of Reference (ToR) sets out the expectations for the TE of the medium-sized project titled “Establishing Transparency Framework for the Republic of Serbia” (PIMS#6211) implemented by the Ministry of Environmental Protection. The project started on March 8th 2019 and is in its final year of implementation. The TE process must follow the guidance outlined in the document ‘Guidance for Conducting Terminal Evaluations of UNDP-Supported, GEF-Financed Projects (TE_ Guidance for UNDP-supported GEF-financed Projects.pdf).

Project Description

The United Nations Development Programme (UNDP), acting as an implementing agency of the Global Environment Facility (GEF), is providing assistance to the Serbian Government, namely Ministry of Environmental Protection, in the preparation and implementation of the GEF funded project “Establishing Transparency Framework for the Republic of Serbia” (CBIT project).

The project was designed to:

• Support the Government of Serbia in mainstreaming and integrating climate change considerations into development strategies and sector-based policy frameworks by strengthening and sustaining efforts to monitor, report, and verify activities to address climate change. Specifically, the project will assist the Government of Serbia with strengthening the methodologies and tools necessary to enhance transparency as described in Article 13 of the Paris Agreement;
• Accelerate Serbia’s EU accession process in the area of environment, energy and climate change, contributing to creation of enabling policy and institutional environment for effective implementation of relevant EU Acquis and related national legal act;
• To increase stakeholder engagement, inter-institutional collaboration and to enhance transparency in the climate change field.

The project resulted in the improved system of monitoring, reporting and verification of the data and information that will be used by the Serbian Government to implement climate & energy legislation and feed in the reporting processes and obligations arising out of various international treaties, such as UNFCCC and EU commitments.

The project supported finalization of monitoring, reporting, and verification (MRV) system that will provide more accurate information and analysis of the instruments that the country selects to mitigate and adapt to climate change. The MRV system also allows Serbia to define and implement climate change-related policies and measures as expressed in its Nationally Determined Contribution (NDC) effectively.

From the beginning of the project implementation inter-institutional, multisectoral working group (WG) was established, comprising of more than 60 representatives of all relevant institutions, and was actively involved during the national MRV system establishment.

This is an adjusted standard term of reference for evaluations in UNDP, considering the impact of COVID-19 on evaluations, including consideration for COVID-19 situation assessment within countries, impact and restrictions on evaluations, alternative approaches, methodologies and considerations to mitigate the impact of COVID-19 on evaluations.

As of 11 March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic as the new coronavirus rapidly spread to all regions of the world. The Government of Serbia declared the State of Emergency due to the COVID-19 outbreak on 14th March 2020. Consequently, number of restrictions were introduced related to movement of people and goods, working arrangements for public and private companies and state institutions. Daily Curfew restrictions were also introduced.

COVID-19 pandemic and the state of emergency declared by the Government in March 2020, caused a significant slowdown, even a deadlock in remaining project activities, which could not be resolved by the engagement of the project staff only. The state of emergency implied very strict measures including rigid travel restrictions (incl. public transport in the cities), as well as night and weekend curfews. Main project partners/beneficiaries are public institutions, which were heavily affected by the measures imposed to fight the COVID pandemic. Operating regime of all public institutions has been significantly changed and limited. Employees have been greatly focused on other urgent issues arising from the crisis. A significant number of employees in the ministries has temporarily been assigned to other duties or working remotely. Such measures have significantly impeded project activities, mainly capacity building and awareness raising. Consequently, the finalization of all expected project activities is delayed for three months, including the terminal project evaluation.

If it is not possible to travel to or within the country for the evaluation then the evaluation team should develop a methodology that takes this into account the conduct of the evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, surveys and evaluation questionnaires. This should be detailed in the Inception report and agreed with the Evaluation Manager.

If all or part of the evaluation is to be carried out virtually then consideration should be taken for stakeholder availability, ability or willingness to be interviewed remotely. In addition, their accessibility to the internet/ computer may be an issue, and these limitations must be reflected in the evaluation report. If a data collection/field mission is not possible then remote interviews may be undertaken through telephone or online (skype, zoom etc.). International consultants can work remotely with national evaluator support in the field if it is safe for them to operate and travel. No stakeholders, consultants or UNDP staff should be put in harm’s way and safety is the key priority.

A short validation mission may be considered if it is confirmed to be safe for staff, consultants, stakeholders and if such a mission is possible within the evaluation schedule. Equally, qualified and independent national consultants can be hired to undertake the evaluation and interviews in country as long as it is safe to do so.

TE Purpose

The TE report will assess the achievement of project results against what was expected to be achieved and draw lessons that can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP programming. The TE report promotes accountability and transparency and assesses the extent of project accomplishments.

Duties and Responsibilities

TE Approach & Methodology

The TE must provide evidence-based information that is credible, reliable and useful.

The TE team (evaluator and national consultant) will review all relevant sources of information including documents prepared during the preparation phase (i.e. PIF, UNDP Initiation Plan, UNDP Social and Environmental Screening Procedure/SESP) the Project Document, project reports including annual PIRs, project budget revisions, lesson learned reports, national strategic and legal documents, and any other materials that the team considers useful for this evidence-based evaluation. The TE team will review the baseline and midterm GEF focal area Core Indicators/Tracking Tools submitted to the GEF at the CEO endorsement and midterm stages and the terminal Core Indicators/Tracking Tools that must be completed before the TE field mission begins.

The TE team is expected to follow a participatory and consultative approach ensuring close engagement with the Project Team, government counterparts (the GEF Operational Focal Point), Implementing Partners, the UNDP Country Office(s), the Regional Technical Advisors, direct beneficiaries and other stakeholders.

Engagement of stakeholders is vital to a successful TE. Stakeholder involvement should include interviews with stakeholders who have project responsibilities, including but not limited to implementing agency, senior officials and task team/component leaders, key experts and consultants in the subject area, Project Board, project beneficiaries, academia, local government and CSOs, etc.

The specific design and methodology for the TE should emerge from consultations between the TE team and the above-mentioned parties regarding what is appropriate and feasible for meeting the TE purpose and objectives and answering the evaluation questions, given limitations of budget, time and data. The TE team must use gender-responsive methodologies and tools and ensure that gender equality and women’s empowerment, as well as other cross-cutting issues and SDGs are incorporated into the TE report.

The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the TE team. TE team should prepare and use questionnaires for broader stakeholder group and virtual interviews. The evaluation team can revise the approach in consultation with the evaluation manager and key stakeholders. These changes in approach should be agreed and reflected clearly in the TE Inception Report.

The final TE report should describe the full TE approach taken and the rationale for the approach making explicit the underlying assumptions, challenges, strengths and weaknesses about the methods and approach of the evaluation.

Detailed Scope of the TE

The TE will assess project performance against expectations set out in the project’s Logical Framework/Results Framework (see TOR Annex A). The TE will assess results according to the criteria outlined in the Guidance for TEs of UNDP-supported GEF-financed Projects, available at: TE_ Guidance for UNDP-supported GEF-financed Projects.pdf.

The Findings section of the TE report will cover the topics listed below. A full outline of the TE report’s content is provided in ToR Annex C.

The asterisk “(*)” indicates criteria for which a rating is required.

Findings

i. Project Design/Formulation

• National priorities and country driven-ness
• Theory of Change
• Gender equality and women’s empowerment
• Social and Environmental Safeguards
• Analysis of Results Framework: project logic and strategy, indicators
• Assumptions and Risks
• Lessons from other relevant projects (e.g. same focal area) incorporated into project design
• Planned stakeholder participation
• Linkages between project and other interventions within the sector
• Management arrangements

ii. Project Implementation

• Adaptive management (changes to the project design and project outputs during implementation)
• Actual stakeholder participation and partnership arrangements
• Project Finance and Co-finance
• Monitoring & Evaluation: design at entry (*), implementation (*), and overall assessment of M&E (*)
• Implementing Agency (UNDP) (*) and Executing Agency (*), overall project oversight/implementation and execution (*)
• Risk Management, including Social and Environmental Standards

iii. Project Results

• Assess the achievement of outcomes against indicators by reporting on the level of progress for each objective and outcome indicator at the time of the TE and noting final achievements
• Relevance (*), Effectiveness (*), Efficiency (*) and overall project outcome (*)
• Sustainability: financial (*), socio-political (*), institutional framework and governance (*), environmental (*), overall likelihood of sustainability (*)
• Country ownership
• Gender equality and women’s empowerment
• Cross-cutting issues (poverty alleviation, improved governance, climate change mitigation and adaptation, disaster prevention and recovery, human rights, capacity development, South-South cooperation, knowledge management, volunteerism, etc., as relevant)
• GEF Additionality
• Catalytic Role / Replication Effect
• Progress to impact

iv. Main Findings, Conclusions, Recommendations and Lessons Learned

• The TE team will include a summary of the main findings of the TE report. Findings should be presented as statements of fact that are based on analysis of the data.
• The section on conclusions will be written in light of the findings. Conclusions should be comprehensive and balanced statements that are well substantiated by evidence and logically connected to the TE findings. They should highlight the strengths, weaknesses and results of the project, respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to project beneficiaries, UNDP and the GEF, including issues in relation to gender equality and women’s empowerment.
• Recommendations should provide concrete, practical, feasible and targeted recommendations directed to the intended users of the evaluation about what actions to take and decisions to make. The recommendations should be specifically supported by the evidence and linked to the findings and conclusions around key questions addressed by the evaluation.
• The TE report should also include lessons that can be taken from the evaluation, including best practices in addressing issues relating to relevance, performance and success that can provide knowledge gained from the particular circumstance (programmatic and evaluation methods used, partnerships, financial leveraging, etc.) that are applicable to other GEF and UNDP interventions. When possible, the TE team should include examples of good practices in project design and implementation.
• It is important for the conclusions, recommendations and lessons learned of the TE report to include results related to gender equality and empowerment of women.

The TE report will include an Evaluation Ratings Table, as shown in the ToR Annex F.

Expected Outputs and Deliverables

The Evaluator shall prepare and submit:

Deliverables

Deadline

1. TE Inception Report, including the Evaluation Criteria Matrix template, prepared and accepted

4 March 2022

2. Presentation of Initial Findings to UNDP, Implementing partner and beneficiaries prepared and delivered

21 March 2022

3. Draft TE Report: Full draft report with annexes prepared and submitted

11 April 2022

4. Final TE Report* (up to 30 pages) and Audit Trail detailing how all received comments have (and have not) been addressed in the final TE report prepared and accepted

5 days upon received comments on the Draft TE, not later than 29 April 2022

*The final TE report must be in English.

All final TE reports will be quality assessed by the UNDP Independent Evaluation Office (IEO). Details of the IEO’s quality assessment of decentralized evaluations can be found in Section 6 of the UNDP Evaluation Guidelines.

TE Arrangements

The principal responsibility for managing the TE resides with the UNDP Country Office. The UNDP CO Serbia will contract the consultants and ensure the timely provision of per diems and travel arrangements within the country for the TE team, if necessary. Due to Covid-19 pandemic situation all meetings/interviews should be organized virtually. The Project Team will be responsible for liaising with the TE team to provide all relevant documents, set up stakeholder interviews, and arrange field visits. TE team will send deliverables to the Evaluation Manager.

Duration of the Work

The total duration of the TE will be approximately 30 working days within the period February - May 2022 and shall not exceed five months from when the TE team is hired. The tentative TE timeframe is as follows:

• 20 February 2022: Application closes
• 24 February 2022: Selection of TE Team
• 25 February 2022: Prep the TE team (handover of project documents)
• 4 March 2022: Document review and preparing TE Inception Report
• 10 calendar days: Finalization and Validation of TE Inception Report- latest start of TE mission
• 5 working days: TE mission: stakeholder meetings, interviews – virtually organized
• 21 March 2022: Mission wrap-up meeting & presentation of initial findings- earliest end of TE mission
• 15 calendar days: Preparation of draft TE report
• 11 April 2022: Circulation of draft TE report for comments
• 3 working days: Incorporation of comments on draft TE report into Audit Trail & finalization of TE report
• 20 April 2022: Preparation & Issue of Management Response
• 29 April 2022: Expected date of full TE completion

The expected date start date of contract is 25 February 2022.

Duty Station

Due to Covid-19 pandemic situation all meetings/interviews should be organized virtually.

Duty-station: home-based

Travel:

• International travel will not be required to Republic of Serbia during the TE mission;
• The BSAFE course must be successfully completed prior to commencement of travel;
• Individual Consultants are responsible for ensuring they have vaccinations/inoculations when travelling to certain countries, as designated by the UN Medical Director.
• Consultants are required to comply with the UN security directives set forth under: https://dss.un.org/dssweb/
• All related travel expenses will be covered and will be reimbursed as per UNDP rules and regulations upon submission of an F-10 claim form and supporting documents.

Competencies

TE Team Composition and Required Qualifications

The principal responsibility for managing TE resides with the UNDP Country Office. The UNDP CO Serbia will contract the consultants. A team of two independent evaluators will conduct the TE – one team leader - Evaluator, and one national consultant. The team leader- Evaluator will be responsible for the overall design and writing of the TE report and accompanying annexes. The national expert will support the organization on the interviews with key stakeholders and project beneficiaries; assess emerging trends with respect to regulatory frameworks, capacity building, work with the Project Team in developing the TE itinerary. The evaluator(s) cannot have participated in the project preparation, formulation and/or implementation (including the writing of the project document), must not have conducted this project’s Mid-Term Review and should not have a conflict of interest with the project’s related activities.

Skills and Competencies

• Excellent analytical skills
• Displays ability to synthesize research and reach empirically based conclusions on related subject
• Strong writing skills
• Proven capacity to produce reports
• Displays capacity to provide experienced advice on best practices
• Possesses knowledge of inter-disciplinary development issues
• Focuses on result for the client and responds positively to feedback
• Good application of Results-Based Management
• Good communication, coordination and facilitation skills
• Consistently ensures timeliness and quality of work
• Displays cultural, gender, religion, race, nationality and age sensitivity and adaptability
• Demonstrates integrity by modeling ethical standards

Required Skills and Experience

Education

• Master’s degree in the project related field (mechanical/ electrical/ agriculture/ forestry/ environment engineering or economy);
• Knowledge of the UNFCCC and Paris Agreement;

Experience

• Minimum 10 years of professional experience in relevant technical areas, preferably in energy/environmental protection sectors
• Relevant experience with results-based management evaluation methodologies;
• Experience applying SMART indicators and reconstructing or validating baseline scenarios;
• Competence in adaptive management, as applied to GEF Climate Change Focal Area;
• Experience in evaluating projects;
• Track record of professional international experience in project development/ management/ monitoring/ evaluation in the climate change field
• Experience working with the GEF or GEF-evaluations, means of verification: the list of evaluated GEF projects
• Good knowledge of international experiences, state of the art approaches and best practices in the specific areas the project and its subcomponents are dealing with
• Demonstrated understanding of issues related to gender and climate change and experience in gender sensitive evaluation and analysis
• Excellent communication skills;
• Demonstrable analytical skills;
• Project evaluation/review experience within United Nations system will be considered an asset
• Experience in working with wide range of stakeholders (private, governmental, etc.).

Language

• Fluency in written and spoken English.

Evaluator Ethics

The TE team will be held to the highest ethical standards and is required to sign a code of conduct upon acceptance of the assignment. This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The evaluator must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The evaluator must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of UNDP and partners.

Payment Schedule

• 20% payment upon satisfactory delivery of the final TE Inception Report and approval by the UNDP
• 40% payment upon satisfactory delivery of the draft TE report to the UNDP
• 40% payment upon satisfactory delivery of the final TE report and approval by the UNDP and RTA (via signatures on the TE Report Clearance Form) and delivery of completed TE Audit Trail

Criteria for issuing the final payment of 40%:

• The final TE report includes all requirements outlined in the TE TOR and is in accordance with the TE guidance.
• The final TE report is clearly written, logically organized, and is specific for this project (i.e. text has not been cut & pasted from other MTR reports).
• The Audit Trail includes responses to and justification for each comment listed.

APPLICATION PROCESS

DOCUMENTS TO BE INCLUDED WHEN SUBMITTING THE PROPOSALS

Application Procedure

Application should include:

• CV in English language containing date of birth, contact information (home address, phone number, e-mail) and timeline of work experience (including description of duties);
• Offeror’s Letter (only PDF format will be accepted) confirming Interest and availability for the Individual Contractor (IC) Assignment. Can be downloaded from the following link: http://www.undp.org.rs/download/ic/Confirmation.docx.
• The Offeror’s Letter should include financial proposal specifying a total lump sum amount for the tasks specified in this announcement with a breakdown of costs.
• Offeror’s Letter must also include the methodology concept containing a preliminary plan of work (no more than two pages).

In order to apply please merge above listed documents into a single PDF file. The system does not allow for more than one attachment to be uploaded.

Any request for clarification must be sent by standard electronic communication to the e-mail [email protected]. The procuring UNDP entity will respond by standard electronic mail and will send response, including an explanation of the query without identifying the source of inquiry, to all consultants.

Financial Proposal:

Lump sum contracts

The financial proposal shall specify a total lump sum amount, and payment terms around specific and measurable (qualitative and quantitative) deliverables (i.e. whether payments fall in installments or upon completion of the entire contract). Payments are based upon output, i.e. upon delivery of the services specified in the TOR. In order to assist the requesting unit in the comparison of financial proposals, the financial proposal will include a breakdown of this lump sum amount (including travel, per diems, and number of anticipated working days).

Travel

All envisaged travel costs must be included in the financial proposal. This includes all travel to join duty station/repatriation travel. In general, UNDP should not accept travel costs exceeding those of an economy class ticket. Should the IC wish to travel on a higher class he/she should do so using their own resources.

Evaluation

1. Cumulative analysis

When using this weighted scoring method, the award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:

a) responsive/compliant/acceptable, and

b) Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

* Technical Criteria weight; 70%

* Financial Criteria weight; 30%

Only candidates obtaining a minimum of 49 points would be considered for the Financial Evaluation

Criteria

Weight

Max. Points

Technical

70%

70 points

• Criteria A

Desk review of CVs based on relevant professional experience in relevant technical areas, preferably in energy/environmental protection sectors

30

• Criteria B

Desk Review of CVs based on experience in working with the GEF or GEF-evaluations

25

• Criteria C

Qualifications (Educational background and language requirements)

15

Financial

30%

30 points

Additional Information:

• Individual Contract (IC) will be applicable for individual consultants applying in their own capacity. If the applicant is employed by any legal entity, IC would be issued upon submission of Consent letter from the employer acknowledging the engagement with UNDP. Template of General Conditions on IC could be found on: https://rs.undp.org/content/dam/serbia/downloads/General%20Conditions%20for%20Individual%20Contracts.pdf
• Reimbursable Loan Agreement (RLA) will be applicable for applicants employed by any legal entity. Template of RLA with General Terms and Conditions could be found on: http://www.undp.org.rs/download/RLA%20with%20General%20Terms%20and%20Conditions.doc. In the case of engagement of Civil servants under IC contract modality a no-objection letter should be provided by the Government entity. The ‘no-objection’ letter must also state that the employer formally certifies that their employees are allowed to receive short-term consultancy assignment from another entity without being on “leave-without-pay” status (if applicable), and include any conditions and restrictions on granting such permission, if any. If the previous is not applicable ‘leave-without-pay’ confirmation should be submitted.

Engagement of Government Officials and Employees

• Government Officials or Employees are civil servants of UN Member States. As such, if they will be engaged by UNDP under an IC which they will be signing in their individual capacity (i.e., engagement is not done through RLA signed by their Government employer), the following conditions must be met prior to the award of contract:

(i) A “No-objection” letter in respect of the individual is received from the Government employing him/her, and;

(ii) The individual must provide an official documentation from his/her employer formally certifying his or her status as being on “official leave without pay” for the duration of the IC.

• The above requirements are also applicable to Government-owned and controlled enterprises and well as other semi/partially or fully owned Government entities, whether or not the Government ownership is of majority or minority status.

UNDP recognizes the possibility that there are situations when the Government entity employing the individual that UNDP wishes to engage is one that allows its employees to receive external short-term consultancy assignments (including but not limited to research institutions, state-owned colleges/universities, etc.), whereby a status of “on-leave-without-pay” is not required. Under such circumstance, the individual entering into an IC with UNDP must still provide a “No-objection” letter from the Government employing him/her. The “no objection” letter required under (i) above must also state that the employer formally certifies that their employees are allowed to receive short-term consultancy assignment from another entity without being on “leave-without-pay” status, and include any conditions and restrictions on granting such permission, if any. The said document may be obtained by, and put on record of, UNDP, in lieu of the document (ii) listed above.

Criteria for Selection of the Best Offer

Only those applications which are responsive and compliant will be evaluated. Offers will be evaluated according to the Combined Scoring method – where the educational background and experience on similar assignments will be weighted at 70% and the price proposal will weigh as 30% of the total scoring. The applicant receiving the Highest Combined Score that has also accepted UNDP’s General Terms and Conditions will be awarded the contract.

Annexes to the TE TOR:

• ToR Annex A: Project Logical/Results Framework
• ToR Annex B: Project Information Package to be reviewed by TE team
• ToR Annex C: Content of the TE report
• ToR Annex D: Evaluation Criteria Matrix template
• ToR Annex E: UNEG Code of Conduct for Evaluators
• ToR Annex F: TE Rating Scales and TE Ratings Table
• ToR Annex G: TE Report Clearance Form
• ToR Annex H: TE Audit Trail template
• Annex I: UNDP Evaluation dispute resolution process - handed over to evaluators when signing the contract

Annex A: Project Logical/Results Framework

available as well at: https://open.undp.org/projects/00114257 (Project Document, pages 30-37)

Project Results Framework

This project will contribute to the following Sustainable Development Goal (s): SDG 13: Take urgent action to combat climate change and its impacts

This project will contribute to the following country outcome included in the UNDAF/Country Programme Document: By 2020, there are improved capacities to combat climate change and manage natural resources and communities are more resilient to the effects natural and man-made disasters

This project will be linked to the following output of the UNDP Strategic Plan: 2.3.1 Data and risk-informed development policies, plans, systems and financing incorporate integrated and gender-responsive solutions to reduce disaster risks, enable climate change adaptation and mitigation, and prevent risk of conflict

Objective and Outcome Indicators

Baseline

Mid-term Target

End of Project Target

Data Collection Methods and Risks/Assumptions

Project Objective:

The development objective of the project is to shift Serbia towards a low-carbon and climate resilience development pathway by mainstreaming and integrating climate change considerations into development strategies and sector-based policy frameworks; ensuring continuity in institutional and technical capacity building; and sustaining these policies and measures with a routine mechanism for climate change monitoring, reporting and verification.

Objective Indictor 1 (CBIT TT Indicator 3): Quality of MRV Systems

Rubric based on CBIT tracking tool (10-point scale)*

5

6

8

Project documentation; structured interviews with project stakeholders

Risks: Lack of available data or access to data.

Assumptions: Existing data generation systems will remain functional and accessible

Objective indicator 2 (CBIT TT indicator 5):

Qualitative Assessment of Institutional Capacity for Transparency-Related Activities

Rubric based on CBIT tracking tool (4-point scale)**

2

3

4

Project documentation; structured interviews with project stakeholders

Risks: Insufficient attention to CC issues on the part of the Government due to other pressing concerns will hinder project implementation.

Assumptions: Continuous engagement with government partner and multi-stakeholder climate change via the Project Board and other meetings

Objective Indicator 3:

Number of direct project beneficiaries

Of that group, number of women

0

N.A.

50 direct beneficiaries, of whom 40 are women

150 direct beneficiaries, of whom 100 are women

Project documentation (training reports and participant evaluations)

Risks: If women are underrepresented in government positions, their participation in project activities may be lower.

Assumptions: Participatory approaches to project activities that support equal participation of men and women

Objective Indicator 4 (CBIT TT indicator 4):

Status of Convention obligations on reporting, including mitigation contribution

Currently, Serbia has submitted its Second NC and its First BUR

By the end of Q6, the Second BUR will have been submitted to the UNFCCC

By the end of Q12, the Third NC and an updated version of the NDC will have been submitted to the UNFCCC

TNC, SBUR, updated NDCs

Risks: Implementation delays could jeopardize the incorporation of information into upcoming UNFCCC reporting.

Assumptions: A combined management approach to the CBIT MSP and the Enabling Activity that allows the two projects to be coordinated.

Component/Outcome 1

National transparency capacity for tracking NDC progress from mitigation activities is strong

Output 1.1: Institutional and technical capacities for transparency of mitigation in relevant sectors improved

Number of users trained on mitigation-related aspects of the national MRV system

Level of participation in the MRV system

Degree to which domestic MRV system informs policies and reporting related to mitigation

18 gov. employees currently directly involved in the climate change mitigation related reporting (namely NCs/BuRs related reporting) and preparing for the EC related reporting obligations – e.g. under the MMR

Elements of the reporting system have been developed, but the system is not operational.

N/A

By the end of Q6, least 35 people have been trained on mitigation aspects of the new MRV system (and of those, at least 60 % are women).

By the end of Q6, database access and use of mitigation information is observed in at least 2 government agencies

By the end of Q6, mitigation information from the database to been used to inform national reporting, i.e. to be used for improvement of national CC related legislation

By the end of Q12, at least 60 people have been trained on mitigation aspects of the new MRV system (and of those, at least 60% are women).

By the end of Q12, database access and use of mitigation information is observed in at least 5 government agencies

By Q12, mitigation information from the database has been used to inform international reporting (e.g. the BUR and revised NDCs).

Registration sheets; training documentation; project reports; surveys; official reporting documents (NDCs, TNC, SBUR)

Risks: Project stakeholders may lack time or interest for meaningful participation in training.

Assumptions:

Government officials will be interested in participating actively in training on the system.

The BUR will be submitted on schedule.

The revision/update of the NDC will take place by Q12 of the project.

Component/ Outcome 2

National transparency capacity for tracking NDC progress from adaptation activities is strong

Output 2.1: institutional and technical capacities for transparency of adaptation in relevant sectors improved

Number of users trained on adaptation-related aspects of the national MRV system (gender-disaggregated)

Level of participation in the MRV system

Degree to which domestic MRV system informs policies and reporting related to CC adaptation

18 gov. employees currently directly involved in the climate change adaptation related reporting (namely NC/BuR-related reportings) and preparing for the EC related reporting obligations

N/A

N/A

By the end of Q6, at least 35 people have been trained on adaptation aspects of the new MRV system (and of those, at least 60% are women).

By the end of Q6, database access and use of mitigation information is observed in at least 2 government agencies

By the end of Q6, adaptation information from the database to been used to inform national reporting, i.e. to be used for improvement of national CC related legislation

By the end of Q12, at least 60 people have been trained on adaptation aspects of the new MRV system (and of those, at least 60% are women).

By the end of Q12, database access and use of adaptation information is observed in at least 5 government agencies

By Q12, adaptation information from the database has been used to inform national reporting (e.g. the 3NC and revised NDCs).

Registration sheets; training documentation; project reports; surveys; official reporting documents (NDCs, TNC, SBUR)

Risks: Project stakeholders may lack time or interest for meaningful participation in training.

Assumptions:

Government officials will be interested in participating actively in training on the system.

The BUR will be submitted on schedule.

The revision/update of the NDC will take place by Q12 of the project.

Component/ Outcome 3

An MRV system for the NDC, including financing for institutions, local communities and businesses, is in place

Output 3.1: Domestic MRV system for updating NDCs completed, including MRV to support stakeholders’ engagement.

Number of stakeholders in local governments and businesses trained on the national MRV system (gender-disaggregated)

Level of participation in the MRV system by local governments and businesses

Degree to which domestic MRV system informs policies and reporting related to capacity needs, technology transfer, and support received related to climate change and policy documents such as the NDCs

Local governments and businesses have not received training on climate change MRV

Local governments and businesses do not provide data on support received or climate change activities to the Government of Serbia.

No standardized system for continuous MRV for aspects reporting on capacity needs, technology transfer, and support received related to climate change.

By Q6, at least 25 local governments have been trained in MRV system and reporting on the CCM and CCA related data and activities

By Q6, at least 25 local governments are reporting data on CCM and CCA in energy, waste management, transport and water management sectors.

By Q6, at least 64 businesses have been trained on MRV requirements and reporting obligations.

All trainings will have taken active steps to ensure equal access to and participation in training by women

By the end of Q6, database access and use of information for transparency reporting other than mitigation and adaptation information is observed in at least 9 government agencies

By the end of Q6, information on capacity development and support received for climate change projects systematically collected and available in the database.

By Q12, at least 65 local governments have been trained in in MRV system and reporting on the CCM and CCA related data and activities

By Q12, at least 25 local governments are reporting data on CCM and CCA in majority of sectors, in accordance with the national MRV framework

By Q12, at least 128 businesses have been trained on MRV requirements and reporting obligations.

All trainings will have taken active steps to ensure equal access to and participation in training by women

By the end of Q12, database access and use of information for transparency reporting other than mitigation and adaptation information is observed in at least 18 government agencies

By Q12, information on capacity development and support received for climate change projects from the database has been used to inform national reporting (e.g. the SBUR and revised NDCs).

Registration sheets; training documentation; project reports; surveys; official reporting documents (NDCs, TNC, SBUR)

Risks: Project stakeholders may lack time or interest for meaningful participation in training.

Assumptions:

The BUR will be submitted on schedule.

The revision/update of the NDC will take place by Q12 of the project.

Output 3.2: National GHG inventories improved

Availability of country-specific emission factors

Scope of estimate of carbon sinks

Use of country-specific emission factors in reporting

Serbia lacks country-specific emission factors for thermal power plants, industry, agriculture, and LULUCF

Estimates of carbon sinks are not comprehensive

The SNC and FBUR use a limited number of country-specific emission factors

The SNC and FBUR make limited estimates of carbon sinks.

By Q6, at least 2 emission factors have been developed for Serbia’s GHG inventory

By Q6, estimates of carbon sinks in Serbia have been broadened to include soil carbon and litter

N.A.

N.A.

By Q12, at least 5 country-specific emission factors have been developed for thermal power plants, selected industrial sectors, agriculture, and LULUCF

By Q12, expanded estimates of carbon sinks have been incorporated into reporting under the TNC

By Q12, all of the country-specific emission factors developed have been incorporated into reporting under the TNC

Project reports; MRV system documentation and operational manuals; official reporting documents (NDCs, TNC, SBUR)

Risks: Sufficient data will be available and accessible to the experts developed emission factors to undertake the additional estimates.

Assumptions:

The emission factors and estimates used will be incorporated into UNFCCC reporting documents

The BUR will be submitted on schedule.

The revision/update of the NDC will take place by Q12 of the project.

Output 3.3: NDC Capacity Built through Peer Exchanges

Availability of peer exchanges

Degree to which peer exchange learning is applied

Support for peer exchanges on NDCs is ad hoc and limited.

N.A.

By Q6, at least 3 peer exchanges have taken place.

Most or nearly all participants report benefits from participation in post-exchange questionnaires (immediate ex post and three months later).

By Q9, at least 5 peer exchanges have taken place.

Nearly all or all participants report benefits from participation in follow-up questionnaires (immediate ex post and three months later.

Project documentation; participant evaluations and questionnaires / interviews as necessary.

Risks: Project stakeholders may lack time or interest for meaningful participation in peer exchanges

Assumptions:

Government partners will have sufficient time and interest for meaningful participation in the exchanges.

Component/ Outcome 4

Knowledge Management and M&E

Level of dissemination of knowledge products produced by the project

N.A.

Findings from the project have been presented internally within UNDP at the CO and regional level

Findings from the project have been presented internally at the country, regional, and global level and at a relevant international forum (e.g. COP side event, international conference).

Project documentation; presentations and publications

Risks: Insufficient involvement of decision makers in collecting sector-specific data necessary for preparing and disseminating knowledge products

Assumptions: Knowledge products will remain accessible to decision-makers and the broader public through Serbia’s national climate change website.

Level of compliance with project M&E plan

N.A.

M&E and adaptive management applied to project in response to needs, mid-term evaluation conducted, and its findings extracted

By the end of the project, a final evaluation has been conducted, and its results and lessons learned have been made available

External evaluations, Project Board Minutes

Risks: Frequent changes of the Government structures in the Country leading to permanent deviations from the project plan.

Assumptions: M&E and adaptive management resulting in improved quality of activities and adjustment of implementation plans to secure achievement of projected targets.

Absolute levels of awareness / capacity, and relative changes in awareness / capacity of project beneficiaries by gender

An initial CC & Gender study for Serbia was prepared within the SNC, as a general review, but it does not include deep gender-related analysis.

Absolute awareness levels and relative changes in awareness among project beneficiaries do not differ significantly between women and men participating in capacity strengthening activities

Absolute awareness levels and relative changes in awareness among project beneficiaries do not differ significantly between women and men participating in capacity strengthening activities

Structured interviews / questionnaires

Risks: Turnover of government staff could reduce the number of trained people in key positions

Assumptions: Training materials are available to new staff as needed; future staff members are trained by other employees at their agencies or by counterparts at other agencies.

* The rating for CBIT Indictor 3 is based on a 10-point scale specified by the GEFSec as follows:

1. Very little measurement is done, reporting is partial and irregular and verification is not there;
2. Measurement systems are in place, but data is of poor quality and/or methodologies are not very robust; reporting is done only on request or to limited audience or partially; verification is not there;
3. Measurement systems are in place for a few activities, improved data quality and methodologies, but not cost or time efficient; wider access to reporting is still limited and information is partial; verification is rudimentary/non-standardized;
4. Measurement systems are strong in a limited set of activities however, analyses still needs improvement; periodic monitoring and reporting although not yet cost/time efficient; verification is only upon specific request and limited;
5. Measurement systems are strong for a limited set of activities and periodically report on key GHG related indicators i.e. mainstreamed into the activity implementation; reporting is improved through few pathways but limited audience and formats; verification limited;
6. Measurement systems are strong and cover a greater percentage of activities – feedback loops exist even if they are not fully functioning; reporting is available through multiple pathways and formats but may not be complete/transparent; verification is done through standard methodologies but only partially (i.e. not all data is verifiable);
7. Measurement regarding GHG is broadly done (with widely acceptable methodologies), need for more sophisticated analyses to improve policy; Reporting is periodic with improvements in transparency; verification is done through more sophisticated methods even if partially;
8. Strong standardized measurements processes established for key indicators and mainstreamed into institutional policy implementation; reporting is widely available in multiple formats; verification is done for a larger set of information;
9. Strong Monitoring and Reporting systems – robust methodologies, cost effective and efficient, periodic; verification done to a significant degree;
10. Strong MRV systems that provide quality GHG-related information in a transparent, accurate and accessible to a wide audience, with feedback of information from MRV flowing into policy design and implementation.

** The rating for CBIT Indictor 5 is based on a 4-point scale specified by the GEFSec as follows:

1. No designated transparency institution to support and coordinate the planning and implementation of transparency activities under Article 13 of the Paris Agreement exists.
2. Designated transparency institution exists, but with limited staff and capacity to support and coordinate implementation of transparency activities under Article 13 of Paris Agreement. Institution lacks authority or mandate to coordinate transparency activities under Article 13.
3. Designated transparency institution has an organizational unit with standing staff with some capacity to coordinate and implement transparency activities under Article 13 of the Paris Agreement. Institution has authority or mandate to coordinate transparency activities under Article 13. Activities are not integrated into national planning or budgeting activities.
4. Designated transparency institution(s) has an organizational unit with standing staff with some capacity to coordinate and implement transparency activities. Institution(s) has clear mandate or authority to coordinate activities under Article 13 of the Paris Agreement, and activities are integrated into national planning and budgeting activities.

Annex B: Project Information Package to be reviewed by TE team

#

Item (electronic versions preferred if available)

1.

Project Identification Form (PIF)

2.

UNDP Initiation Plan

3.

Final UNDP-GEF Project Document with all annexes

4.

CEO Endorsement Request

5.

UNDP Social and Environmental Screening Procedure (SESP) and associated management plans (if any)

6.

Inception Workshop Report

7.

Mid-Term Review report and management response to MTR recommendations

8.

All Project Implementation Reports (PIRs)

9.

Progress reports (quarterly, semi-annual or annual, with associated workplans and financial reports)

10.

Oversight mission reports

11.

Minutes of Project Board Meetings and of other meetings (i.e. Project Appraisal Committee meetings)

12.

GEF Tracking Tools (from CEO Endorsement, midterm and terminal stages)

13.

GEF/LDCF/SCCF Core Indicators (from PIF, CEO Endorsement, midterm and terminal stages); for GEF-6 and GEF-7 projects only

14.

Financial data, including actual expenditures by project outcome, including management costs, and including documentation of any significant budget revisions

15.

Co-financing data with expected and actual contributions broken down by type of co-financing, source, and whether the contribution is considered as investment mobilized or recurring expenditures

16.

Audit reports

17.

Electronic copies of project outputs (booklets, manuals, technical reports, articles, etc.)

18.

Sample of project communications materials

19.

Summary list of formal meetings, workshops, etc. held, with date, location, topic, and number of participants

20.

Any relevant socio-economic monitoring data, such as average incomes / employment levels of stakeholders in the target area, change in revenue related to project activities

21.

List of contracts and procurement items over ~US$5,000 (i.e. organizations or companies contracted for project outputs, etc., except in cases of confidential information)

22.

List of related projects/initiatives contributing to project objectives approved/started after GEF project approval (i.e. any leveraged or “catalytic” results)

23.

Data on relevant project website activity – e.g. number of unique visitors per month, number of page views, etc. over relevant time period, if available

24.

UNDP Country Programme Document (CPD)

25.

List/map of project sites, highlighting suggested visits

26.

List and contact details for project staff, key project stakeholders, including Project Board members, RTA, Project Team members, and other partners to be consulted

27.

Project deliverables that provide documentary evidence of achievement towards project outcomes

Annex C: Content of the TE report

i. Title page

• Tile of UNDP-supported GEF-financed project
• UNDP PIMS ID and GEF ID
• TE timeframe and date of final TE report
• Region and countries included in the project
• GEF Focal Area/Strategic Program
• Executing Agency, Implementing partner and other project partners
• TE Team members

ii. Acknowledgements

iii. Table of Contents

iv. Acronyms and Abbreviations

1.Executive Summary (3-4 pages)

• Project Information Table
• Project Description (brief)
• Evaluation Ratings Table
• Concise summary of findings, conclusions and lessons learned
• Recommendations summary table

2.Introduction (2-3 pages)

• Purpose and objective of the TE
• Scope
• Methodology
• Data Collection & Analysis
• Ethics
• Limitations to the evaluation
• Structure of the TE report

3.Project Description (3-5 pages)

• Project start and duration, including milestones
• Development context: environmental, socio-economic, institutional, and policy factors relevant to the project objective and scope
• Problems that the project sought to address: threats and barriers targeted
• Immediate and development objectives of the project
• Expected results
• Main stakeholders: summary list
• Theory of Change

4.Findings

(in addition to a descriptive assessment, all criteria marked with (*) must be given a rating )

4.1 Project Design/Formulation

• Analysis of Results Framework: project logic and strategy, indicators
• Assumptions and Risks
• Lessons from other relevant projects (e.g. same focal area) incorporated into project design
• Planned stakeholder participation
• Linkages between project and other interventions within the sector

4.1 Project Implementation

• Adaptive management (changes to the project design and project outputs during implementation)
• Actual stakeholder participation and partnership arrangements
• Project Finance and Co-finance
• Monitoring & Evaluation: design at entry (*), implementation (*), and overall assessment of M&E (*)
• UNDP implementation/oversight (*) and Implementing Partner execution (*), overall project implementation/execution (*), coordination, and operational issues
• Risk Management incl. Social and Environmental Standards (Safeguards)

4.2 Project Results

• Progress towards objective and expected outcomes (*)
• Relevance (*)
• Effectiveness (*)
• Efficiency (*)
• Overall Outcome (*)
• Country ownership
• Gender
• Other Cross-cutting Issues
• Sustainability: financial (*), socio-economic (*), institutional framework and governance (*), environmental (*), and overall likelihood (*)
• Country Ownership
• Gender equality and women’s empowerment
• Cross-cutting Issues
• GEF Additionality
• Catalytic Role / Replication Effect
• Progress to Impact

5. Main Findings, Conclusions, Recommendations & Lessons

• Main Findings
• Conclusions
• Recommendations
• Lessons Learned

6. Annexes

• TE ToR (excluding ToR annexes)
• TE Mission itinerary
• List of persons interviewed
• List of documents reviewed
• Summary of field visits
• Evaluation Question Matrix (evaluation criteria with key questions, indicators, sources of data, and methodology)
• Questionnaire used and summary of results
• Co-financing tables (if not include in body of report)
• TE Rating scales
• Signed Evaluation Consultant Agreement form
• Signed UNEG Code of Conduct form
• Signed TE Report Clearance form
• Annexed in a separate file: TE Audit Trail
• Annexed in a separate file: relevant terminal GEF/LDCF/SCCF Core Indicators or Tracking Tools, as applicable

Annex D: Evaluation Criteria Matrix template

Evaluative Criteria Questions

Indicators

Sources

Methodology

Relevance: How does the project relate to the main objectives of the GEF Focal area, and to the environment and development priorities a the local, regional and national level?

(include evaluative questions)

(i.e. relationships established, level of coherence between project design and implementation approach, specific activities conducted, quality of risk mitigation strategies, etc.)

(i.e. project documentation, national policies or strategies, websites, project staff, project partners, data collected throughout the TE mission, etc.)

(i.e. document analysis, data analysis, interviews with project staff, interviews with stakeholders, etc.)

Effectiveness: To what extent have the expected outcomes and objectives of the project been achieved?

Efficiency: Was the project implemented efficiently, in line with international and national norms and standards?

Sustainability: To what extent are there financial, institutional, socio-political, and/or environmental risks to sustaining long-term project results?

Gender equality and women’s empowerment: How did the project contribute to gender equality and women’s empowerment?

Impact: Are there indications that the project has contributed to, or enabled progress toward reduced environmental stress and/or improved ecological status?

(Expand the table to include questions for all criteria being assessed: Monitoring & Evaluation, UNDP oversight/implementation, Implementing Partner Execution, cross-cutting issues, etc.)

Annex E: UNEG Code of Conduct for Evaluators

Independence entails the ability to evaluate without undue influence or pressure by any party (including the hiring unit) and providing evaluators with free access to information on the evaluation subject. Independence provides legitimacy to and ensures an objective perspective on evaluations. An independent evaluation reduces the potential for conflicts of interest which might arise with self-reported ratings by those involved in the management of the project being evaluated. Independence is one of ten general principles for evaluations (together with internationally agreed principles, goals and targets: utility, credibility, impartiality, ethics, transparency, human rights and gender equality, national evaluation capacities, and professionalism).

Evaluators/Consultants:

1. Must present information that is complete and fair in its assessment of strengths and weaknesses so that decisions or actions taken are well founded.
2. Must disclose the full set of evaluation findings along with information on their limitations and have this accessible to all affected by the evaluation with expressed legal rights to receive results.
3. Should protect the anonymity and confidentiality of individual informants. They should provide maximum notice, minimize demands on time, and respect people’s right not to engage. Evaluators must respect people’s right to provide information in confidence, and must ensure that sensitive information cannot be traced to its source. Evaluators are not expected to evaluate individuals, and must balance an evaluation of management functions with this general principle.
4. Sometimes uncover evidence of wrongdoing while conducting evaluations. Such cases must be reported discreetly to the appropriate investigative body. Evaluators should consult with other relevant oversight entities when there is any doubt about if and how issues should be reported.
5. Should be sensitive to beliefs, manners and customs and act with integrity and honesty in their relations with all stakeholders. In line with the UN Universal Declaration of Human Rights, evaluators must be sensitive to and address issues of discrimination and gender equality. They should avoid offending the dignity and self-respect of those persons with whom they come in contact in the course of the evaluation. Knowing that evaluation might negatively affect the interests of some stakeholders, evaluators should conduct the evaluation and communicate its purpose and results in a way that clearly respects the stakeholders’ dignity and self-worth.
6. Are responsible for their performance and their product(s). They are responsible for the clear, accurate and fair written and/or oral presentation of study imitations, findings and recommendations.
7. Should reflect sound accounting procedures and be prudent in using the resources of the evaluation.
8. Must ensure that independence of judgement is maintained, and that evaluation findings and recommendations are independently presented.
9. Must confirm that they have not been involved in designing, executing or advising on the project being evaluated and did not carry out the project’s Mid-Term Review.

Evaluation Consultant Agreement Form

Agreement to abide by the Code of Conduct for Evaluation in the UN System:

Name of Evaluator: ______________________________________________________________

Name of Consultancy Organization (where relevant): ____________________________________

I confirm that I have received and understood and will abide by the United Nations Code of Conduct for Evaluation.

Signed at __________________________________ (Place) on ______________________ (Date)

Signature: _____________________________________________________________________

Annex F: TE Rating Scales & Evaluation Ratings Table

TE Rating Scales

Ratings for Outcomes, Effectiveness, Efficiency, M&E, Implementation/Oversight, Execution, Relevance

Sustainability ratings:

6 = Highly Satisfactory (HS): exceeds expectations and/or no shortcomings

5 = Satisfactory (S): meets expectations and/or no or minor shortcomings

4 = Moderately Satisfactory (MS): more or less meets expectations and/or some shortcomings

3 = Moderately Unsatisfactory (MU): somewhat below expectations and/or significant shortcomings

2 = Unsatisfactory (U): substantially below expectations and/or major shortcomings

1 = Highly Unsatisfactory (HU): severe shortcomings

Unable to Assess (U/A): available information does not allow an assessment

4 = Likely (L): negligible risks to sustainability

3 = Moderately Likely (ML): moderate risks to sustainability

2 = Moderately Unlikely (MU): significant risks to sustainability

1 = Unlikely (U): severe risks to sustainability

Unable to Assess (U/A): Unable to assess the expected incidence and magnitude of risks to sustainability

Evaluation Ratings Table

Monitoring & Evaluation (M&E)

Rating (1)

M&E design at entry

M&E Plan Implementation

Overall Quality of M&E

Implementation & Execution

Rating

Quality of UNDP Implementation/Oversight

Quality of Implementing Partner Execution

Overall quality of Implementation/Execution

Assessment of Outcomes

Rating

Relevance

Effectiveness

Efficiency

Overall Project Outcome Rating

Sustainability

Rating

Financial resources

Socio-political/economic

Institutional framework and governance

Environmental

Overall Likelihood of Sustainability

(1) Outcomes, Effectiveness, Efficiency, M&E, I&E Execution, Relevance are rated on a 6-point rating scale: 6 = Highly Satisfactory (HS), 5 = Satisfactory (S), 4 = Moderately Satisfactory (MS), 3 = Moderately Unsatisfactory (MU), 2 = Unsatisfactory (U), 1 = Highly Unsatisfactory (HU). Sustainability is rated on a 4-point scale: 4 = Likely (L), 3 = Moderately Likely (ML), 2 = Moderately Unlikely (MU), 1 = Unlikely (U)

Annex G: TE Report Clearance Form

Terminal Evaluation Report for (Project Title & UNDP PIMS ID) Reviewed and Cleared By:

Commissioning Unit (M&E Focal Point)

Name: _____________________________________________

Signature: __________________________________________ Date: _______________________________

Regional Technical Advisor (Nature, Climate and Energy)

Name: _____________________________________________

Signature: __________________________________________ Date: ______________________________

Apply now

Added 2 years ago - Updated 2 years ago - Source: jobs.undp.org