National foreword

This Published Document is the UK implementation of ISO/PAS 8800:2024.

The UK participation in its preparation was entrusted to Technical Committee AUE/32/1.

A list of organizations represented on this committee can be obtained on request to its committee manager.

Contractual and legal considerations

This publication has been prepared in good faith, however no representation, warranty, assurance or undertaking (express or implied) is or will be made, and no responsibility or liability is or will be accepted by BSI in relation to the adequacy, accuracy, completeness or reasonableness of this publication. All and any such responsibility and liability is expressly disclaimed to the full extent permitted by the law.

This publication is provided as is, and is to be used at the recipient’s own risk.

The recipient is advised to consider seeking professional guidance with respect to its use of this publication.

This publication is not intended to constitute a contract. Users are responsible for its correct application.

This publication is not to be regarded as a British Standard.

Compliance with a Published Document cannot confer immunity from legal obligations.

This Published Document was published under the authority of the Standards Policy and Strategy Committee on 31 December 2024.

Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO member bodies). The work of preparing International Standards is normally carried out through ISO technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. International organizations, governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of ISO document should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).

ISO draws attention to the possibility that the implementation of this document may involve the use of (a) patent(s). ISO takes no position concerning the evidence, validity or applicability of any claimed patent rights in respect thereof. As of the date of publication of this document, ISO had not received notice of (a) patent(s) which may be required to implement this document. However, implementers are cautioned that this may not represent the latest information, which may be obtained from the patent database available at www.iso.org/patents. ISO shall not be held responsible for identifying any or all such patent rights.

Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.

This document was prepared by Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 32, Electrical and electronic components and general system aspects.

Any feedback or questions on this document should be directed to the user’s national standards body. A complete listing of these bodies can be found at www.iso.org/members.html.

Introduction

The purpose of this document is to provide industry-specific guidance on the use of AI systems in safety-related functions. It is not restricted to specific AI methods or specific vehicle functions.

This document defines a framework for managing AI safety that tailors or extends existing approaches currently defined in the ISO 26262 series and in ISO 21448.

Functional safety-related risks associated with malfunctioning behaviour of an AI system are addressed by tailoring or extending relevant clauses from ISO 26262‑series.

The risks related to functional insufficiencies in the AI system are addressed by extending the concepts and guidance provided by ISO 21448. A causal model for understanding the sources of functional insufficiencies in the AI system is proposed. The model is used to derive a set of safety requirements on the AI system as well as a set of risk reduction measures.

NOTE 1 ISO 21448 is applicable to intended functionalities where proper situational awareness is essential to safety and where such situational awareness is derived from sensors and processing algorithms, especially functionalities of emergency intervention systems and systems with ISO/SAE PAS 22736 levels 1 to 5 for driving automation. It is therefore possible that systems utilize AI technologies that do not fall within the scope of ISO 21448.

EXAMPLE 1 ISO 21448 does not apply to the development of an engine control unit that uses AI to optimize its performance whereas this document does.

This document recognizes that due to the wide range of applications of AI and associated safety requirements, as well as the rapidly evolving state-of-the-art, it is not possible to provide detailed requirements on the process or product characteristics required to achieve an acceptably low level of residual risk associated with the use of AI systems. Therefore, in addition to providing guidance for tailoring or extending the ISO 26262 series and ISO 21448, this document focuses on the principles that support the creation of a project-specific assurance argument for the safety of the AI elements within on-board vehicle systems. This includes proposing risk reduction measures during the design and operation phases using an iterative approach to reducing risk as outlined in ISO/IEC Guide 51.

Hazard analysis and risk analysis are beyond the scope of this document. These are considered a part of the vehicle level systems safety engineering activities described in the ISO 26262 series and ISO 21448, or in application of specific standards such as ISO TS 5083.

ISO/IEC TR 5469 provides generic guidance for the application of AI technologies as part of safety functions, independent of specific industry sectors. Many of the concepts outlined in ISO/IEC TR 5469 can be applied in the context of road vehicles. There is therefore a close relationship to concepts described within this document and ISO/IEC TR 5469.

ISO/IEC TR 5469 provides classification schemes to determine the safety requirements on the AI/ML function. These include the usage level and AI technology class.

The usage level is related to the nature of the task being performed by the engineered AI system.

NOTE 2 The usage levels are described in ISO/IEC TR 5469:2024, 6.2.

The technology class is related to the problem complexity and the transferability of existing standards to demonstrating an adequate level of safety based on properties of the target function and the AI technology used.

NOTE 3 For the technology classes, see ISO/IEC TR 5469:2024, 6.2.

This document does not explicitly call out the classes and usage levels of ISO/IEC TR 5469.

EXAMPLE 2 For some AI technology, the application of ISO 26262 is deemed to be sufficient. This corresponds to Class I of ISO/IEC TR 5469.

The guidance outlined within this document is relevant for all usage of AI for which safety requirements can foreseeably be allocated either through:

the use of AI for the functionality itself;
the use of AI as a safety mechanism.

NOTE 4 These usages correspond to the usage levels A1, A2, C of ISO/IEC TR 5469. In all cases, the applicability of the guidance provided within this document can be determined by the allocation of safety requirements to the AI technology, whereas the usage levels of ISO/IEC TR 5469 can be used to support the requirements elicitation process.

This document is aligned with standards and documents developed by ISO/IEC JTC1/SC42. AI-specific definitions are used from ISO/IEC 22989, unless in conflict with safety-specific definitions.

Other documents developed within ISO/IEC JTC1/SC42 can be used to provide additional guidance on specific aspects of AI that are relevant to safety-related properties. Examples of such documents include ISO/IEC TR 24027 and ISO/IEC TR 24029‑1.

This document harmonizes the concepts already described in ISO 21448:2022, Annex D.2 and ISO/TS 5083:20‑1, Annex B whilst extending these with specific guidance regarding the definition of safety requirements of machine learning (ML), ML safety analyses and the creation of associated safety evidence during the development and deployment lifecycle.

ISO/TS 5083:20‑, Annex B is an application of this document to automated driving systems (ADS).

The relationship with the above-mentioned documents is summarized in Table 1–1.

This document adds the following contents with respect to the documents listed in Table 1-1:

tailoring or extensions of ISO 26262 and ISO 21448 required specifically for AI elements of the system (referred to as AI systems);
a conceptual model for reasoning about errors and their causes specific to AI systems;
a reference AI safety lifecycle;
the safety assurance argument for AI systems;
a method for deriving AI safety requirements for AI systems;
considerations for the design of safe AI systems;
considerations on data management for the AI systems;
a verification and validation strategy for AI systems;
a safety analysis approach for AI systems (focused on insufficiencies);
activities during operation required to ensure the continuous AI safety.

1 Scope

This document applies to safety-related systems that include one or more electrical and/or electronic (E/E) systems that use AI technology and that is installed in series production road vehicles, excluding mopeds. It does not address unique E/E systems in special vehicles, such as E/E systems designed for drivers with disabilities.

This document addresses the risk of undesired safety-related behaviour at the vehicle level due to output insufficiencies, systematic errors and random hardware errors of AI elements within the vehicle. This includes interactions with AI elements that are not part of the vehicle itself but that can have a direct or indirect impact on vehicle safety.

EXAMPLE 1 Examples of AI elements within the vehicle include the trained AI model and AI system.

EXAMPLE 2 Direct impact on safety can be due to object detection by elements external to the vehicle.

EXAMPLE 3 Indirect impact on safety can be due to field monitoring by elements external to the vehicle.

The development of AI elements that are not part of the vehicle is not within the scope of this document. These elements can conform to domain-specific safety guidance. This document can be used as a reference where such domain-specific guidance does not exist.

This document describes safety-related properties of AI systems that can be used to construct a convincing safety assurance claim for the absence of unreasonable risk.

This document does not provide specific guidelines for software tools that use AI methods.

This document focuses primarily on a subclass of AI methods defined as machine learning (ML). Although it covers the principles of established and well-understood classes of ML, it does not focus on the details of any specific AI methods e.g. deep neural networks.

2 Normative references

3 Terms and definitions

3.1 General AI-related definitions

3.2 Data-related definitions

3.3 General safety-related definitions

3.4 Safety: Root cause-, error-and failure-related definitions

3.5 Miscellaneous definitions

4 Abbreviated terms

5 Requirements for conformity

5.1 Purpose

5.2 General requirements

6 AI within the context of road vehicles system safety engineering and basic concepts

6.1 Application of the ISO 26262 series for the development of AI systems

6.2 Interactions with encompassing system-level safety activities

6.3 Mapping of abstraction layers between the ISO 26262 series, ISO/IEC 22989 and this document

6.4 Example architecture for an AI system

6.5 Types of AI models

6.6 AI technologies of a ML model

6.7 Error concepts, fault models and causal models

7 AI safety management

7.1 Objectives

7.2 Prerequisites and supporting information

7.3 General requirements

7.4 Reference AI safety life cycle

7.5 Iterative development paradigms for AI systems

7.6 Work products

8 Assurance arguments for AI systems

8.1 Objectives

8.2 Prerequisites and supporting information

8.3 General requirements

8.4 AI system-specific considerations in assurance arguments

8.5 Structuring assurance arguments for AI systems

8.6 The role of quantitative targets and qualitative arguments

8.7 Evaluation of the assurance argument

8.8 Work products

9 Derivation of AI safety requirements

9.1 Objectives

9.2 Prerequisites and supporting information

9.3 General requirements

9.4 General workflow for deriving safety requirements

9.5 Deriving AI safety requirements on supervised machine learning

9.6 Work products

10 Selection of AI technologies, architectural and development measures

10.1 Objectives

10.2 Prerequisites

10.3 General requirements

10.4 Architecture and development process design or refinement

10.5 Examples of architectural and development measures for AI systems

10.6 Work products

11 Data-related considerations

11.1 Objectives

11.2 Prerequisites and supporting information

11.3 General requirements

11.4 Dataset life cycle

11.5 Work products

12 Verification and validation of the AI system

12.1 Objectives

12.2 Prerequisites and supporting information

12.3 General requirements

12.4 AI/ML specific challenges to verification and validation

12.5 Verification and validation of the AI system

12.6 Work products

13 Safety analysis of AI systems

13.1 Objectives

13.2 Prerequisites and supporting information

13.3 General requirements

13.4 Safety analysis of the AI system

13.5 Work products

14 Measures during operation

14.1 Objectives

14.2 Prerequisites and supporting information

14.3 General requirements

14.4 Planning for operation and continuous assurance

14.5 Continual, periodic re-evaluation of the assurance argument

14.6 Measures to assure safety of the AI system during operation

14.7 Field data collection

14.8 Evaluation and continuous development

14.9 Work products

15 Confidence in use of AI development frameworks and software tools used for AI model development

15.1 Objectives

15.2 Prerequisites and supporting information

15.3 General requirements

15.4 Confidence in the use of AI development frameworks

15.5 Confidence in the use of tools used to support the AI-safety lifecycle

15.6 Principles for data-driven AI model training and evaluation

15.7 Work products

Bibliography

PD ISO/PAS 8800:2024

Road vehicles. Safety and artificial intelligence

Current

Published:

31 Dec 2024