The National Transport Strategy (NTS) for 2030 was developed to help fulfil Kingdom Vision 2030 by identifying the challenges faced by government entities and establishing targets and initiatives to overcome those challenges and the Project is participating in achieving the NTS, set targets and Kingdom Vision 2030 strategic objectives which will lead to achieve the Sustainable Development Goals especially Goal (9) ‘‘Industry, Innovation and Infrastructure’’ and Goal (13) ‘‘Climate Action’’ where these two goals are critical drivers of economic growth and development of KSA road network.

Therefore, the tasks related to both the Vision 2030 and the National Transportation Strategy (NTS2030) initiatives require a high level of coordination among Ministry of Transport (MOT) departments and with other government agencies and demand additional specialized knowledge to support the Strategic Planning Department and other concerned MOT entities. Due to the new context of the Vision 2030 and the current need for MOT to focus on the implementation of the new initiatives such as coordination with Ministry of Municipal Rural Affairs MOMRA, therefore, the duration of the Project needs to be extended beyond the original to 2022.

Basic Project information can also be included in table format as follows:

PROJECT/OUTCOME INFORMATION

Project/outcome title

Sustainable Road and Transport Management

Atlas ID

SAU10/79238

Corporate outcome and output

Public sector strengthened through improved efficiency, effectiveness, equity and accountability

Country

Saudi Arabia

Region

RBAS

Date project document signed

Project dates

Start

Planned end

01/01/2012

31/12/2022

Project budget

16.2M

Estimated Project expenditure at the time of evaluation

13M

Funding source

Government

Implementing party

Ministry of Transport

Duties and Responsibilities

2. Evaluation purpose, scope and objectives

The project has been ongoing since 2011 and has, thus far, never been evaluated. Drastic changes have been taking place in the country and the project has had to adapt to the changes over recent years, this included changes in Ministers, Deputy Ministers and Project staff, resulting in changing project directions. In order to ensure the project has delivered its intended objectives and to provide recommendations for the way forward, it was imperative to conduct a final evaluation. This evaluation will benefit the Ministry of Transport in their planning for future years to meet Saudi Vision 2030 and highlight the impacts this project has had on the transport sector over the past few years.

This evaluation should cover all components of the project (such as road safety initiatives, trade facilitation and etc), those relevant before Saudi Vision 2030 and those after and its impact on the sector in the whole country. However, the evaluator shall consider the Saudi Vision 2030 objective’s and other MOT Initiatives in formulating the new Project Document which can both improve the sustainability of benefits from this project, and aid in the overall enhancement of UNDP Goals as mentioned above.

Evaluation criteria and key guiding questions

Evaluation questions define the information that this evaluation will generate. Questions should be grouped according to the four OECD-DAC evaluation criteria: (a) relevance; (b) effectiveness; (c) efficiency (d) impact; and (e) sustainability (and/or other criteria used).

The mainstream definitions of the OECD-DAC criteria are neutral in terms of human rights and gender dimensions and these dimensions need to be added into the evaluation criteria chosen (see page 77, table 10 of Integrating Human Rights and Gender Equality in Evaluations).

Ratings must be provided on the following performance criteria. The completed table must be included in the evaluation executive summary.

Evaluation Ratings:

1. Monitoring and Evaluation

rating

2. IA& EA Execution

rating

M&E design at entry

Quality of UNDP Implementation

M&E Plan Implementation

Quality of Execution - Implementing Partner

Overall quality of M&E

Overall quality of Implementation

3. Assessment of Outcomes

rating

4. Sustainability

rating

Relevance

Financial resources:

Effectiveness

Socio-political:

Efficiency

Institutional framework and governance:

Impact

Sustainability

Overall Project Outcome Rating

Environmental :

Overall likelihood of sustainability:

Relevance

To what extent was the project in line with the national development priorities, the country programme’s outputs and outcomes, the UNDP Strategic Plan and the SDGs?
To what extent does the project contribute to the theory of change for the relevant country programme outcome especially in addressing the transport sector in Saudi Arabia?
To what extent were lessons learned from other relevant projects considered in the project’s design?
To what extent were perspectives of those who could affect the outcomes, and those who could contribute information or other resources to the attainment of stated results, taken into account during the project design processes?
To what extent has the project enhanced knowledge on transportation behaviour change, particularly on sustainable transportation
To what extent does the project contribute to gender equality, the empowerment of women and the human rights-based approach?
To what extent has the project been appropriately responsive to political, legal, economic, environmental, institutional, etc., changes in the country?

Effectiveness

To what extent did the project contribute to the country programme outcomes and outputs, the SDGs, the UNDP Strategic Plan and national development priorities?
To what extent were the project outputs achieved especially in achieving sustainable transport?
What factors have contributed to achieving or not achieving intended country programme outputs and outcomes?
To what extent has the UNDP partnership strategy been appropriate and effective?
What factors contributed to effectiveness or ineffectiveness?
In which areas does the project have the greatest achievements? Why and what have been the supporting factors? How can the project build on or expand these achievements?
In which areas does the project have the fewest achievements? What have been the constraining factors and why? How can or could they be overcome?
What, if any, alternative strategies would have been more effective in achieving the project’s objectives?
Were the project’s objectives and outputs clear, practical and feasible within its frame?
To what extent have stakeholders been involved in project implementation? Or to what extent do they feel they own the actions they are taking.
To what extent are project management and implementation participatory and is this participation contributing towards achievement of the project objectives?
To what extent has the project been appropriately responsive to the needs of the national constituents and changing partner priorities?
To what extent has the project contributed to gender equality, the empowerment of women and the realization of human rights? To what extent women were involved in the implementation of the actions indirectly or directly.

Efficiency

To what extent was the project management structure as outlined in the project document efficient in generating the expected results?
To what extent have the UNDP project implementation strategy and execution been efficient and cost-effective?
To what extent has there been an economical use of financial and human resources? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes?
To what extent have resources been used efficiently? Have activities supporting the strategy been cost-effective?
To what extent have project funds and activities been delivered in a timely manner?
To what extent do the M&E systems utilized by UNDP ensure effective and efficient project management?

Impact:

What are the major impacts this project has made on the project outcomes
To what extent has this project affected positive change

Sustainability

Are there any financial risks that may jeopardize the sustainability of project outputs?
To what extent will financial and economic resources be available to sustain the benefits achieved by the project?
Are there any social, economic, environmental or political risks that may jeopardize sustainability of project outputs and the project’s contributions to country programme outputs and outcomes?
Do the institutional and legal frameworks, policies and governance structures and processes within which the project operates pose risks that may jeopardize sustainability of project benefits?
To what extent are institutional and human resource capacities strengthened to provide effective technical support to national partners and stakeholders for energy efficiency actions
What is the risk that the level of stakeholders’ ownership will be sufficient to allow for the project benefits to be sustained? To what extent the project was effective to enhance integration of sustainable transport in public and private sector actions
To what extent do mechanisms, procedures and policies exist to allow primary stakeholders to carry forward the results attained on gender equality, empowerment of women, human rights and human development?
To what extent do stakeholders support the project’s long-term objectives?
To what extent are lessons learned being documented by the project team on a continual basis and shared with appropriate parties who could learn from the project?
To what extent do UNDP interventions have well-designed and well-planned exit strategies?
What could be done to strengthen exit strategies and sustainability?

Evaluation cross-cutting issues sample questions

Human rights

To what extent have poor, indigenous and physically challenged, women and other disadvantaged and marginalized groups benefited from the work of UNDP in the country?
To what extent does the project ensure that no one is left behind in regards to project benefits?

Gender equality

To what extent have gender equality and the empowerment of women been addressed in the design, implementation and monitoring of the project?
Is the gender marker data assigned to this project representative of reality?
To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any positive or negative unintended effects?

Guiding evaluation questions can be further refined by the evaluator and agreed with UNDP/Project management and the- evaluation stakeholders in the inception report.

Evaluation Approach and Methodology

An overall approach and method[1] for conducting project terminal evaluations of UNDP. The methodology of work will consist of desk review of relevant project documentation and direct consultations with the project management, staff and other key local stakeholders during two weeks site visit to Riyadh in September2021.

The overall duration of the assignment is expected to consist of a site visit of 15 days includes a corresponding amount of desk work to pre-review the required project documentation and to finalize the reporting. This makes the total working days to be 21working days, the timeframe detailed shall be as stipulated below.

In carrying out the evaluation task, the consultant will pay particular attention to the following:

Evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments;
Document review of all relevant documentation. This would include a review of inter alia;
Project document (contribution agreement);
Theory of change and results framework;
Programme and project quality assurance reports;
Annual workplans;
Activity designs;
Consolidated quarterly and annual reports;
Results-oriented monitoring report;
Highlights of project board meetings;
Technical/financial monitoring reports;
The National Transport Strategy (NTS);
Vision 2030 and corresponding transformation plans related to the transportation sector (add links)
Semi-structured interviews with key stakeholders including key government counterparts, donor community members, representatives of key civil society organizations, UNCT members and implementing partners: All interviews should be undertaken in full confidence and anonymity under the support of the Project Management. The final evaluation report should not assign specific comments to individuals.
Development of evaluation questions around relevance, effectiveness, efficiency and sustainability and designed for different stakeholders to be interviewed.
Surveys and questionnaires including participants in development programmes, UNCT members and/or surveys and questionnaires involving other stakeholders at strategic and programmatic levels.
Field visits and on-site validation of key tangible outputs and interventions.
The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries including industry partners and general public on awareness.
Other methods such as outcome mapping, observational visits, group discussions, etc.
Data review and analysis of monitoring and other data sources and methods.
Ensure maximum validity, reliability of data (quality) and promote use; the evaluator will ensure triangulation of the various data sources.

The evaluator must use gender-based methodology and tools and ensure that gender equality and women's empowerment, inclusion of vulnerable groups as well as other cross-cutting issues and the SDGs, are included in the final evaluation report and the new project document.

The evaluation must provide factual information that is credible, reliable and useful. The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP, stakeholders and the evaluators.

Evaluation products (deliverables)

An evaluation report and an associated power point presentation summarizing the findings of the evaluation and the proposed follow-up actions in a new UNDP Project Document format.

The consultant will be expected to deliver the following:

Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
Evaluation findings debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
Draft evaluation report (60 pages including executive summary). The programme unit and key stakeholders in the evaluation should review the draft evaluation report and provide a set of comments to the evaluator within an agreed period of time, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in the evaluation guidelines.
Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how he/she has addressed comments.
Final evaluation report:
- Executive summary;
- Introduction, including description of the work conducted;
- Findings and conclusions;
- Recommendations, including, as applicable, a revised work plan to address the pending tasks and eventual corrective action as well as an improved system for measuring the impact of the project in terms of achieved energy savings;
- Annexes providing a brief summary of the documents reviewed and persons interviewed with the description of the key content / conclusions drawn and any other relevant materials.
Validation workshop for presentations to stakeholders and/or the evaluation reference group including MOT;
Evaluation brief and other knowledge products or participation in knowledge-sharing events, if relevant.
A comprehensive UNDP new Project Document

The consultant should present three hard copies of the report as well as an electronic copy. The draft final report should be submitted not later than three weeks after the end of the on-site mission and the final report and the Project document within two weeks from receiving the comments of the project management and UNDP on the draft reports.

The Project document will be formulated with technical assistance provided by the Ministry and the Project Manager.

Standard templates that need to be followed are provided in the Annexes section. It is expected that the evaluator will follow the UNDP evaluation guidelines and UNEG quality check list and ensure all the quality criteria are met in the evaluation report.

In line with UNDP’s financial regulations, when determined by the Country Office and/or the consultant that a deliverable or service cannot be satisfactory completed due to impact of COVID-19 and limitations to the evaluation, that deliverable or service will not be paid. Due to the current COVID-19 situation and its implications, a partial payment may be considered if the consultant invested time towards the deliverable but was unable to complete to circumstances beyond his/her/their control.

[1] For additional information on methods, see the Handbook on Planning, Monitoring and Evaluating for Development Results, Chapter 7, pg. 163

Competencies

Evaluation consultant required competencies

The consultancy will be conducted by one individual evaluator (national or international).

The evaluator shall have prior experience in evaluating similar projects and have sufficient experience in transport sector planning related to initiatives and main national and international development operations related to road sector specifically.

The evaluator selected should have not participate in the project implementation in order to avoid any conflict of interest with the project related activities.

The evaluator must present the following qualifications:

Advanced university degree in transport planning and engineering and obtain Professional Transport Planner (PTP) certificate is preferable;
Minimum 15 years of relevant professional experience in the area of transport planning and initiatives;
Familiar with UNDP approaches to development assistance;
Demonstrated understanding of issues related to human rights and gender; experience in gender sensitive evaluation and analysis in a development project; Previous experience with results-based formulating, monitoring and evaluation methodologies;
Strong communication and analytical skills;
Strong command of English language, both written and spoken;
Previous work experience in the region is an asset.

Evaluation ethics

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.”

Implementation arrangements

UNDP Saudi Arabia Country Office will select the consultant through a transparent process in consultation with MOT. UNDP will be responsible for the management of the consultant and will in this regard designate an evaluation manager. The project management unit will assist in facilitating the process (e.g., providing relevant documentation, arranging visits/interviews with key informants, participate in reviewing the evaluation deliverables and arrange for the consultant all necessary site visits and meetings in Saudi Arabia according to the ToR). UNDP country office in coordination with the project management unit shall arrange logistics for the mission including hotel reservation and transportation during the mission.

The evaluation manager will convene an evaluation reference group comprising of technical experts from UNDP, donors and implementing partners. This reference group will review the inception report and the draft review report to provide detailed comments related to the quality of methodology, evidence collected, analysis and reporting. The reference group will also advise on the conformity of processes to the UNDP evaluation guidelines. The evaluator needs to show how he/she addressed the comments.

The consultant will take responsibility, with assistance from the project manager, for conducting the meetings and the review, subject to advanced approval of the methodology submitted in the inception report. Project staff will not participate in the meetings between the consultant and the evaluation participants.

The consultant will report directly to the designated evaluation manager and work closely with the project manager.

The final report and project document will be approved by the evaluation commissioner.

UNDP with support of relevant stakeholders will develop a management response to the evaluation within 2 weeks of report finalization.

Required Skills and Experience

Time frame for the evaluation process

Activity

Timing/working days

Estimated Completion Date

Note

Desk Review

15 September 2021

In home country

Evaluation Mission

20 September 2021

In KSA

Draft Evaluation Report

30 September 2021

In home country

Final Report

10 October 2021

Draft Project Document

17 October 2021

In home country

Final Project Document

24 October 2021

Total

September – October 2021

EVALUATION DELIVERABLES

The Evaluator is expected to deliver the following:

Deliverable

Content

Timing

Responsibilities

Inception Report

Evaluator provides clarifications on timing and method

On arrival Day (15 September 2021)

Evaluator submits to Project Management and UNDP CO.

Debriefing presentations

Initial Findings

End of evaluation mission (19 September 2021)

To Project Management and UNDP CO.

Draft Final Report

Full report, (per annexed template) with annexes

Within 2 weeks of the evaluation mission (30 September 2021)

Sent to Project Management and UNDP CO, reviewed by MOT.

Final Report*

Revised report

Within 1 week of receiving UNDP comments on draft (10 October 2021)

To Project Management and UNDP CO.

Draft Project Document

Full Project Document

Within 4 weeks of the evaluation mission (17 October 2021)

Sent to Project Management and UNDP CO, reviewed by MOT.

Final Project* Document

Revised Project Document

Within 1 week of receiving UNDP comments on draft (24 October 2021)

To Project Management and UNDP CO.

*When submitting the final evaluation report and final Project Document, the evaluator is required also to provide an 'audit trail', detailing how all received comments have (and have not) been addressed in the final evaluation report and the Project Document.

Payment modalities and specifications

Milestone

20%

Acceptance of Inception Report prior to the field visit.

30%

Following approval of the final Evaluation Report.

50%

Following approval of the final Project Document.

In accordance with UNDP's financial regulations, where the country office and/or the consultant determine that a deliverable or service cannot be provided satisfactorily due to the impact of COVID-19 and limitations on evaluation, that deliverable or service will not be paid.

Due to the current situation and the implications of COVID-19, a partial payment may be considered if the consultant has invested time in the production of the deliverable but has not been able to ensure its full supply due to circumstances beyond his control.

Estimated working day allocation and schedule for this evaluation

ACTIVITY

ESTIMATED # OF WORKING DAYS

DATE OF COMPLETION

PLACE

RESPONSIBLE PARTY

Phase One: Desk review and inception report

Meeting briefing with UNDP (programme managers and project staff as needed)

At the time of the Mission Started 15 September 2021

UNDP or remote

Evaluation manager and commissioner

Sharing of the relevant documentation with the evaluation team

At the time of contract signing

8 September 2021

Via email

Evaluation manager and commissioner

Desk review, Evaluation design, methodology and updated workplan including the list of stakeholders to be interviewed

3 days

Within two weeks of contract signing

15 September 2021

Home- based

Evaluation Consultant

Submission of the inception report

(15 pages maximum)

By 15 September 2021

Evaluation Consultant

Comments and approval of inception report

By 17 September 2021

UNDP

Evaluation manager

Phase Two: Data-collection mission

Consultations and field visits, in-depth interviews and focus groups

19 September 2021

In country

With field visits

UNDP to organize with local project partners, project staff, local authorities, etc.

Debriefing to UNDP and key stakeholders

19 September 2021

In country

Evaluation Consultant

Phase Three: Evaluation report writing

Preparation of draft evaluation report (50 pages maximum excluding annexes), executive summary (5 pages) and Draft Report submission

Within Two weeks of the completion of the field mission

End of September 2021

Home- based

Evaluation Consultant

Consolidated UNDP and stakeholder comments to the draft report

By 6 October 2021

UNDP

Evaluation manager and evaluation reference group

Finalization and submission of the evaluation report incorporating additions and comments provided by project staff and UNDP country office

by 10 October 2021

Home- based

Evaluation Consultant

Preparation of the draft Project Document

Within one week of final evaluation report acceptance

17 October 2021

Home- based

Evaluation Consultant

Consolidated UNDP and stakeholder Comments on Project Document

Within one week from draft PD submission

23 October 2021

UNDP and National Partner

Finalization of PD and submission

Within 5 days from receiving comments

30 October 2021

Evaluation Consultant

Estimated total days for the evaluation

Application submission process and criteria for selection

As required by the programme unit.

TOR annexes

Annexes can be used to provide additional detail about evaluation background and requirements to facilitate the work of evaluators. Some examples include:

Intervention results framework and theory of change. Provides more detailed information on the intervention being evaluated.
Key stakeholders and partners. A list of key stakeholders and other individuals who should be consulted, together with an indication of their affiliation and relevance for the evaluation and their contact information. This annex can also suggest sites to be visited.
Documents to be consulted. A list of important documents and web pages that the evaluators should read at the outset of the evaluation and before finalizing the evaluation design and the inception report. This should be limited to the critical information that the evaluation team needs. Data sources and documents may include:
- Relevant national strategy documents,
- Strategic and other planning documents (e.g., programme and project documents).
- Monitoring plans and indicators.
- Partnership arrangements (e.g., agreements of cooperation with Governments or partners).
- Previous evaluations and assessments.
- UNDP evaluation policy, UNEG norms and standards and other policy documents.
Evaluation matrix (suggested as a deliverable to be included in the inception report). The evaluation matrix is a tool that evaluators create as map and reference in planning and conducting an evaluation. It also serves as a useful tool for summarizing and visually presenting the evaluation design and methodology for discussions with stakeholders. It details evaluation questions that the evaluation will answer, data sources, data collection, analysis tools or methods appropriate for each data source, and the standard or measure by which each question will be evaluated.

Table 1****. Sample evaluation matrix

Relevant evaluation criteria

Key questions

Specific sub questions

Data sources

Data-collection methods/tools

Indicators/ success standard

Methods for data analysis

Schedule of tasks, milestones and deliverables. Based on the time frame specified in the TOR, the evaluators present the detailed schedule.

Required format for the evaluation report. The final report must include, but not necessarily be limited to, the elements outlined in the quality criteria for evaluation reports (). The standard template can be found in the following link (Evaluation report).

Code of conduct. UNDP programme units require each member of the evaluation team to read carefully, understand and sign the ‘Code of Conduct for Evaluators in the United Nations system’, it should be made available as an attachment to the evaluation report.
Inception report
Audit trail
UNDP Evaluation Guidelines
Evaluation Quality Assessment (the final report should address all the questions in the checklist pages 8- 11)

Interested individual consultants must submit the following documents/information to demonstrate their qualifications in one single PDF document:

Technical Methodology/Proposal
Duly accomplished Confirmation of Interest and Submission of Financial Proposal Template using the template provided by UNDP (Annex II);
Personal CV or P11, indicating all past experience from similar projects, as well as the contact details (email and telephone number) of the Candidate and at least three (3) professional references.

The interested offeror must read the Individual Consultant (IC) Procurement Notice, which can be viewed at http://procurement-notices.undp.org/view_notice.cfm?notice_id=62795 for more detail information about term of references, instructions to the offeror, and to download the documents to be submitted in the offer through online.

UNDP reserves rights to reject any applications that are incomplete.

Please be informed that we don’t accept application submitted via email.

Interested Offerors are required to submit an application via UNDP Jobsite system as the application screening and evaluation will be done through UNDP Jobsite system. Please note that UNDP Jobsite system allows only one uploading of application document, so please make sure that you merge all your documents into a single file. Your on¬line application submission will be acknowledged where an email address has been provided. If you do not receive an e¬mail acknowledgement within 24 hours of submission, your application may not have been received. In such cases, please resubmit the application, if necessary. Please combine all your documents into one (1) single PDF document as the system only allows to upload maximum one document.

Any request for clarification/additional information on this procurement notice shall be communicated in writing to UNDP office or send to email mohammed.abbas@undp.org. While the Procurement Unit would endeavour to provide information expeditiously, only requests receiving at least 3 working days prior to the submission deadline will be entertained. Any delay in providing such information will not be considered as a reason for extending the submission deadline. The UNDP's response (including an explanation of the query but without identifying the source of inquiry) will be posted in the Individual Consultant (IC) Procurement Notice page as provided above. Therefore, all prospective Offerors are advised to visit the page regularly to make obtain update related to this Individual Consultant (IC) Procurement Notice

UNDP is committed to achieving workforce diversity in terms of gender, nationality and culture. Individuals from minority groups, indigenous groups and persons with disabilities are equally encouraged to apply. All applications will be treated with the strictest confidence.

UNDP does not tolerate sexual exploitation and abuse, any kind of harassment, including sexual harassment, and discrimination. All selected candidates will, therefore, undergo rigorous reference and background checks.

Annexes:

Annex-1 – Template Confirmation of Interest and Submission of Financial Proposal.

Annex-2 – Individual Consultant General Terms and Conditions.

Annex 3 : TOR

Forms are available on the following link

https://www.dropbox.com/sh/zwnm3ns28iavyeh/AAA7bhxAjH5lxAWmiNnJBjksa?dl=0

Apply now

Added 2 years ago - Updated 2 years ago - Source: jobs.undp.org