Introduction

0.1 General

A long established and commonly held view is that the single most significant factor in road traffic safety are the actions of the driver. However, the road transport system is a complex socio-technical system which places high demands on humans to negotiate. Crashes occur since human beings, due to finite cognitive capacity and physiological limitations, cannot always cope with these demands. To increase road safety, the road transport system therefore should be designed to support the road user to cope with this complexity and to mitigate the effects of crashes when this is not possible.

Autonomous vehicles (AVs) have the potential to replace the human driver and to increase road safety by reducing the risk of human error in daily operations. This will probably take a long time and in the meantime the capabilities of technology and humans should be combined and integrated in such a way that the strengths of both are utilized efficiently. Important safety improvements were made over the last century, but automated driving technology provides new opportunities to improve safety even further.

AVs are unlikely to gain widespread acceptance until the travelling public feels assured of their safety and security, not only of passengers but also other vehicles and vulnerable road users. This includes a behaviour of the AV according to the desires and requirements of society. However, despite contemporary expectations and a very optimistic view on technology, humans have an extremely valuable faculty that machines will probably never possess—ethical decision making and judgement. For real traffic situations the human does not always have the prerequisites to take a rational decision based on ethics since the time frame is often too narrow. For that reason, many “decisions” are made instinctively without the possibility of making a well-reasoned and balanced ethical decision. Defining this element and imparting it on machines is critical for the success of AVs. This can only be achieved by ensuring that AVs are equipped with driving action policies that align with the general ethical beliefs, needs, and desires of humanity on a global level, subject to local specific nuances.

To achieve the imprint of global and local ethical considerations in AV design, there is a need for a framework of ethics involving the necessary stakeholders of different areas. To that objective’s end, it is important to develop ethical standards for AV behaviour. While there are few standards available, or under development, that address the engineering and technological aspects of AVs, there are no International Standards that address aspects concerning the general topics of driving policy and ethical behaviour, which are also important. Driving policy means a general approach of how an AV makes a decision and performs manoeuvres. Ethical relevant behaviour represents positive or potentially negative impact on road users and especially the vulnerable ones as well as the public space at large.

The objective of this document is to lay out a framework for the development of a standard for ethical and societally accepted driving policy.

0.2 The concept of autonomous vehicles

AVs have the objective to substitute driver, including tasks, decisions, and responsibilities. Hence the driver behaviour model proposed by Michon (1985) and applied to AVs in Reference [26] (Figure 1) can be a start to design and operate AVs. The driving task consists of three levels: the strategic level concerned with the higher-level trip goals (e.g. route choice), the tactical level concerned with the manoeuvring decisions, including negotiations and interactions with other vehicles, and operational level concerned with the execution of these tactical and operational behaviours at the level of vehicle control. There is no strictly hierarchical relationship among these levels.

Figure 1 — Hierarchical model of driving task (source: Reference [26])

Beside this consideration which has a timely sequence, the AV will ideally be designed thoroughly to ensure safe and secure operation, and the decision base for the desired behaviour established. Therefore, all necessary information would be available, and the situation will be captured and understood while different perspectives will support the final decision.

Figure 1 is a generic decomposition of how an AV works. Figure 2 is a more focused and detailed description of the decision making module (where the ethical considerations resides) Hence a functional decomposition into the following six layers (Reference [27]) (Figure 2) can provide the first step into solution space.

Figure 2 — Decomposition layers (source: Reference [27])

— Information access

Information available and accessible.
— Information reception

All necessary information can be captured.
— Information processing

Captured information contain all required classification and identification for further process.

— Situational understanding

Based on information, situation is captured and understood.

— Behavioural decision

Based on designed or trained situational awareness, the desired behaviour is chosen.
— Action

Vehicle transforms whished behaviour into action.

Within this framework, technical realization can differ while the focus of the situation decision will be comprehensible, accountable, and comparable among different designs. A thorough development will benefit transparency for action and to take decisions. The work described in this document focuses primarily on Layer 4 – behavioural decision.

0.3 The concept of ethical considerations for autonomous vehicles

Philosophy helps people question, understand and make sense of the world so they can act properly in it. This means that decisions and actions are not only intrinsically fair, but that they are also performed in a way that is balanced with respect with other’s needs, the societal needs and humanistic values, as well as with respect to the physical world. It is widely accepted that philosophy can be divided into theoretical philosophy and practical philosophy. Theoretical philosophy consists of subdisciplines such as ontology, epistemology, logic, and philosophy of mind, amongst others. Practical philosophy consists of subdisciplines like ethics, political and social philosophy, and aesthetics.

Within ethics there are three main branches: normative ethics, applied ethics as well as meta-ethics. Normative ethics is the study of ethical action and determining standards for decision making and conduct (e.g. consequentialism, deontology, virtue ethics, see Annex A). Applied ethics is the application of standards of ethics to real life situations (e.g. biomedical ethics, AI ethics, political ethics, see Annex B). In this document, ethics refers to both normative and applied ethics but not to meta-ethics (which is more concerned with the meaning of ethical concepts such as ‘good’ or ‘bad’ or the nature of moral judgments. Applied ethics is a flexible and practical way to address ethical considerations in the development of new technologies because an applied ethics field, such as AV ethics, can borrow from more than one normative school of ethics. Therefore, the framework offered in this document, although based on a principle-based approach, can use ideas from deontology and virtue ethics, for example, to help solve problems. This is a balanced approach; it does not condone one type of normative ethics over another, it offers a range of perspectives that will help the designer/developer in choosing the best (or better) decision possible for specific situations.

In summary, ethics is the study of how to choose to act in situations in order to make good and rationally justifiable decisions. To make decisions, it is necessary to clarify what matters and to understand what is considered good, bad, right and wrong. Therefore, ethics may be viewed as a tool that helps us create the difference between a “good” decision and a “bad” one, but also to justify this decision on rational and intersubjectively acceptable grounds. This is of great importance in the development of AVs because choices made during the design and development of AV systems determine its “driving behaviour” and how it caters to its passengers and interacts with other road users (e.g. vulnerable road users). “Driving behaviour” is what was designed for and programmed into the machine, “conduct” is what actually transpires as a consequence of applying the driving behaviour to the real world.

This document offers a particular framework for AV ethics which is intended to support the practical integration of ethics into the AV development process. The framework suggested here builds upon Socrates conception of the hierarchical nature of philosophy for practical use which has three levels (see Figure 3). The base level is the conduct that transpires as a consequence of decision and actions (behaviour), the second level is how these behaviours are governed. Namely what kind of policy and arrangements are in place to make decisions. The top level is purpose, which behaviour and conduct will normally align with, in order to achieve the goals and objectives of the entire endeavour. Finally, knowledge is the “tool” for making sure that the purpose is reasonable and balanced and that the governance and behaviour/conduct are indeed feasible and appropriate.

Figure 3 — Socrates’ hierarchical nature of philosophy

Figure 4 is a graphical depiction of the framework advocated here for AV ethics. It shares similarities with other ethical approaches for emerging technologies, it is also unique as it has been created specifically for the AV context. Essentially, the AV ethics framework provides guidance for reflective and critical decision making and is composed of four main elements: purpose, values, principles, and process. The framework approach provides structure and guidance, yet it is flexible enough to accommodate multicultural perspectives.

Figure 4 — The AV ethics framework for the integration of ethical considerations into the design and development of AVs

The purpose element refers to the overall goal of integrating ethics into AV design and development. The purpose is the “why” aspect of the framework or the reason for the existence of this document. Purpose offers high level (or meta) guidance when decision making, i.e. will a particular decision lead to an increase in road traffic safety or not. As this is an international standard, the values element provides guidance on what is important to the world’s population. Therefore, universal values such as the UN universal values (e.g. human dignity) are recommended in this framework. The principles element gives more detailed guidance on decision making by identifying the boundaries of a good decision and a bad one as well as directing the attention to specific areas (fairness in AV behaviour and conduct in the context of other vehicles). The process element refers to two levels of process activities, governance, and daily operations, which are required to integrate ethics into AV development. Table 1 summarises these four elements.

Specific examples for the values, principles, and process elements are provided in the AV ethics framework section. The purpose is not an example but a strong recommendation as it contributes to the overall goal of the ISO traffic management safety series of standards. The values, principles and processes presented here are relevant and universally acceptable examples but may be changed according to the requirements and needs of specific countries, societies and organizations. The users of this document may decide on which values, principles and processes they will use for their work. Finally, it is acknowledged that an ethics framework is only useful if it is adhered to by all involved and is well integrated into the AV system development process.

Table 1 — Framework elements and recommended designations

Purpose	Overarching goal: increase safety in road traffic systems for all traffic participants (ISO TC 241)
Values	Value set: UN universal values recommended
Principles	Principle set: artificial intelligence for people principles recommended (with other options listed)
Process	Governance (e.g. integration of AV ethics into existing governance measures)
Process	Daily operations (e.g. ethical evaluations, operationalization of principles)

BS ISO 39003:2023

Road traffic safety (RTS). Guidance on ethical considerations relating to safety for autonomous vehicles

Current

Published:

31 Aug 2023