1 Scope
This document defines metrics for the charging performance of electronic fee collection
(EFC) systems in terms of the level of errors associated with charging computation.
This document is a toolbox standard of metrics. The detailed choice of metrics depends
on the application and the respective context.
This document describes a set of metrics with appropriate definitions, principles
and formulations, which together make up a reference framework for the establishment
of requirements for EFC systems and their later examination of the charging performance.
The charging performance metrics defined in this document are intended for use with any Charging Scheme, regardless
of its technical underpinnings, system architecture, tariff structure, geographical
coverage, or organizational model. They are defined to treat technical details that
can be different among technologies and vendors or vary over time as a “black box”.
They focus solely on the outcome of the charging process, i.e. the amount charged
in relation to a pre-measured or theoretically correct amount, rather than intermediate
variables from various components as sensors, such as positioning accuracy, signal
range, or optical resolution. This approach ensures comparable results for each metric
in all relevant situations.
The metrics are designed to cover the information exchanged on the Front End interface and the interoperability interfaces between Toll Service Providers, Toll Chargers and Road Users as well as on the End-to-End level.
Metrics on the following information exchanges are defined:
-
—
Charge Reports;
-
—
Toll Declarations;
-
—
Billing Details and associated event data;
-
—
Payment Claims on the level of toll service user accounts;
-
—
User Accounts;
-
— End-to-End Metrics which assess the overall performance of the charging process.
The details on the rationale of this choice are described in 5.1.
The proposed metrics are specifically addressed to protect the interests of the actors
in a toll system, such as Toll Service Providers, Toll Chargers and Road Users. The metrics can be used to define requirements (e.g. for requests
for proposals) and for performance assessment.
This document recognises two types of situations where a performance assessment is
necessary:
-
a) when an assessment is carried out during a limited time span, such as when formulating
requirements and assessing systems for acquisition purposes, conducting acceptance
testing as part of the commissioning process, or as part of a certification procedure.
Any one of these types of assessment is referred to as an evaluation;
-
b) when an assessment is needed as an ongoing supervision process, throughout the lifetime
of a system, in order to validate contracted service levels, to identify fraud or
malfunction, or to support ongoing maintenance and performance improvement processes.
This type of assessment is referred to as monitoring.
NOTE 1 Definitions and metrics proposed in this document are intended for both situations.
The following are not covered by this document.
-
— This document does not propose specific numeric performance bounds, or average or
worst-case error bounds in percentage or monetary units. Those decisions are left
to the Toll Charger (or to agreements between Toll Charger and Service Provider), while providing a way to be sure that there is a consistent
framework for describing system requirements when writing Request for proposals, for
system comparisons during acquisition, for test results, for Service Level Agreements,
and ongoing (post-deployment) performance monitoring.
-
— This document does not consider the evaluation of the expected performance of a system based on modelling and measured data from
a trial at another place.
-
— This document does not consider the specification of a common reference system which
would be required for comparison of performance between systems.
-
— This document does not specify metrics on parts of tolling systems other than the
charging process chain, such as:
-
— enforcement system;
-
— security measures.
-
— This document does not cover metrics on parts of the charging processing chain which
are considered an internal matter of one of the interoperability partners:
-
— equipment performance, e.g. for on-board equipment, road-side equipment or data centres
such as signal range, optical resolution or computing system availability;
-
— position performance metrics: The quality of data generated by position sensors is
considered as an internal aspect of the Front End. It is masked by correction algorithms, filtering, inferring of data and the robustness
of the Charge Object recognition algorithms.
Even though some of these aspects have a direct impact on charging performance, they
are not considered explicitly in this document.
NOTE 2 While the Front End interface is considered as internal to the Toll Service Provider domain of responsibility, it is still covered by metrics. There are two reasons for
this exception: firstly, a set of standards [ISO 17575 (all parts)] exists on this interface, and secondly, the information exchanged on this interface
is also part on the TSP-TC interface (ISO 12855) and therefore metrics are needed.