Activity 5 Task 2 of the EU EIP project required the specification of a definitive list of practical ITS Deployment and Benefit KPIs for the ex-post evaluation of road-based ITS services. A KPI taskforce was therefore set up in 2016, including representatives of some of the ITS corridors. Active contributors to the KPI definition report included EU EIP representatives from Finland, Germany, Italy, Netherlands, Sweden and the United Kingdom; however, the wider EU EIP project team was also involved in the review of the proposed KPIs list. The objective of such co-operation was twofold:
- To ensure that knowledge matured from previous studies (EasyWay and EIP+) was translated in this latest piece of work; and
- The ensure that the developed list of KPIs could benefit from a wide consensus base from the start – hopefully resulting in a limited number of future revisions.
The EU EIP deployment and benefit KPIs have been presented in a technical report which was circulated for the first time to all ITS Corridors in February 2017. This document has been produced with the intent of providing a single convenient and practical reference point to evaluators, including KPI definitions and suggested estimation methods. The content of this document draws significantly from various sources and work on the subject, in particular: the ITS KPI definitions developed by DG MOVE as well as the body of work relating to the EasyWay and EIP+ studies. For ease of reference, a shortlist of DG MOVE’s KPIs has been included in the EU EIP KPI report. The report also provides a comparison of related (if not directly corresponding) EU/EIP and DG MOVE KPIs. It is important to note that whilst some differences in the KPI lists have been acknowledged, these do not represent ‘mistakes or omissions’ but should be considered within the context of applicability of said KPIs; specifically:
- EU/EIP KPIs are, generally speaking, more detailed and should be used primarily for evaluation purposes;
- DG MOVE KPIs are more ‘generic’ and should continue being employed by Member States as part their reporting duties to the Commission.
The KPI report, whilst providing an ‘how to’ guide relating to the estimation of KPIs, recognises the fact that there exist country-specific data gathering/quality consistency issues. As such, the KPI evaluation guidelines do not represent mandatory requirements but, for the sake of consistency, it is strongly advised that evaluators adopt such methodologies wherever possible and that consider opportunities for ‘process-change’ if this means a better alignment with the evaluation guidelines. The EU EIP KPI description document does not intend to preclude the use of National standards/estimation methodologies but promotes the principles of ‘Record and Explain’ key kepartures from the suggested guidelines. This approach can also be used to develop a ‘knowledge database’ (Values of Time, Value of Emissions, Value of a Life, Societal Discount rates, etc). In summary, where alternative methodologies are adopted by evaluators, it is required that any significant divergence/departure from the guidelines remains well documented and that impacts are, at least, qualitatively described.
As mentioned previously, the EU EIP KPIs Technical Note intends to be a “practical document”, developed through consensus. In this instance on-the-field implementation will tell us whether this objective has been met! Furthermore, the EU EIP project team recognises the fact that, with time, additional KPIs may be added while existing KPIs amy be modified (or even dropped), as technology evolves. For this reason, this technical document should be considered as a LIVE DOCUMENT.
It is also important to note that the Benefit KPIs have been developed with the intention to facilitate the undertaking of consistent Cost Benefit Analyses(CBAs) across all Corridors, in line with relevant CBA guidance developed as part of the EU/EIP+ project.
Finally, it is important to note that the definition of the KPIs will inform the structure of the future Evaluation Reports as well as other A5 work packages relating to ITS Directive Priority Action Evaluation Plan and the development of the Evaluation Toolkit which will be aligned to the KPI Definitions where applicable.
A copy of the ITS Deployment and Benefit KPI Definitions can be found here.