National foreword

This British Standard is the UK implementation of EN ISO/IEC 25059:2024. It is identical to ISO/IEC 25059:2023. It supersedes BS ISO/IEC 25059:2023, which is withdrawn.

The UK participation in its preparation was entrusted to Technical Committee ART/1, Artificial Intelligence.

A list of organizations represented on this committee can be obtained on request to its committee manager.

Contractual and legal considerations

This publication has been prepared in good faith, however no representation, warranty, assurance or undertaking (express or implied) is or will be made, and no responsibility or liability is or will be accepted by BSI in relation to the adequacy, accuracy, completeness or reasonableness of this publication. All and any such responsibility and liability is expressly disclaimed to the full extent permitted by the law.

This publication is provided as is, and is to be used at the recipient’s own risk.

The recipient is advised to consider seeking professional guidance with respect to its use of this publication.

This publication is not intended to constitute a contract. Users are responsible for its correct application.

Compliance with a British Standard cannot confer immunity from legal obligations.

This British Standard was published under the authority of the Standards Policy and Strategy Committee on 31 July 2023.

Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.

The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).

ISO and IEC draw attention to the possibility that the implementation of this document may involve the use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not received notice of (a) patent(s) which may be required to implement this document. However, implementers are cautioned that this may not represent the latest information, which may be obtained from the patent database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held responsible for identifying any or all such patent rights.

Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 42, Artificial intelligence.

Any feedback or questions on this document should be directed to the user’s national standards body. A complete listing of these bodies can be found at www.iso.org/members.html and www.iec.ch/national-committees.

Introduction

High-quality software products and computer systems are crucial to stakeholders. Quality models, quality requirements, quality measurement, and quality evaluation are standardized within the International Standards on SQuaRE, see Annex A for further information.

AI systems require additional properties and characteristics of systems to be considered, and stakeholders have varied needs. AI systems have different properties and characteristics. For example, AI systems can:

replace human decision-making;
be based on noisy, or incomplete data;
be probabilistic;
adapt during operation.

According to ISO/IEC TR 24028,^[²^] trustworthiness has been understood and treated as both an ongoing organizational process as well as a non-functional requirement specifying emergent properties of a system — that is, a set of inherent characteristics with their attributes — within the context of quality of use as indicated in ISO/IEC 25010.

ISO/IEC TR 24028 discusses the applicability to AI systems of that have been developed for conventional software. According to ISO/IEC TR 24028, does not sufficiently address the data-driven unpredictable nature of AI systems. While considering the existing body of work, ISO/IEC TR 24028 identifies the need for developing new International Standards for AI systems that can go beyond the characteristics and requirements of conventional software development.

ISO/IEC TR 24028 contains a related discussion on different approaches to testing and evaluation of AI systems. It states that for testing of an AI system, modified versions of existing software and hardware verification and validation techniques are needed. It identifies several conceptual differences between many AI systems and conventional systems and concludes that “the ability of the [AI] system to achieve the planned and desired result … may not always be measurable by conventional approaches to software testing”. Testing of AI systems is addressed in ISO/IEC TR 29119‑11:2020.^[³^]

This document outlines an application-specific AI system extension to the SQuaRE quality model specified in ISO/IEC 25010.

AI systems perform tasks. One or more tasks can be defined for an AI system. Quality requirements can be specified for the evaluation of task fulfilment.

The quality model is considered from two perspectives, product quality as described in Clause 5 and quality in use in Clause 6. The relevance of these terms is explained, and links to other standardization deliverables (e.g. the ISO/IEC 24029 series^[⁴^][⁵^]) are highlighted.

ISO/IEC 25012:2008^[⁶^] contains a model for data quality that is complementary to the model defined in this document. ISO/IEC 25012:2008 is being extended for AI systems by the ISO/IEC 5259 series.^[⁷^]

This European Standard was approved by CEN on 1 August 2024.

CEN and CENELEC members are bound to comply with the CEN/CENELEC Internal Regulations which stipulate the conditions for giving this European Standard the status of a national standard without any alteration. Up-to-date lists and bibliographical references concerning such national standards may be obtained on application to the CEN-CENELEC Management Centre or to any CEN and CENELEC member.

This European Standard exists in three official versions (English, French, German). A version in any other language made by translation under the responsibility of a CEN and CENELEC member into its own language and notified to the CEN-CENELEC Management Centre has the same status as the official versions.

CEN and CENELEC members are the national standards bodies and national electrotechnical committees of Austria, Belgium, Bulgaria, Croatia, Cyprus, Czechia, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Türkiye and United Kingdom.

European foreword

The text of ISO/IEC 25059:2023 has been prepared by Technical Committee ISO/IEC JTC 1 "Information technology” of the International Organization for Standardization (ISO) and has been taken over as EN ISO/IEC 25059:2024 by Technical Committee CEN-CENELEC/ JTC 21 “Artificial Intelligence” the secretariat of which is held by DS.

This European Standard shall be given the status of a national standard, either by publication of an identical text or by endorsement, at the latest by February 2025, and conflicting national standards shall be withdrawn at the latest by February 2025.

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. CEN-CENELEC shall not be held responsible for identifying any or all such patent rights.

Any feedback and questions on this document should be directed to the users’ national standards body. A complete listing of these bodies can be found on the CEN and CENELEC websites.

According to the CEN-CENELEC Internal Regulations, the national standards organizations of the following countries are bound to implement this European Standard: Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Republic of North Macedonia, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Türkiye and the United Kingdom.

Endorsement notice

The text of ISO/IEC 25059:2023 has been approved by CEN-CENELEC as EN ISO/IEC 25059:2024 without any modification.

1 Scope

This document outlines a quality model for AI systems and is an application-specific extension to the standards on SQuaRE. The characteristics and sub-characteristics detailed in the model provide consistent terminology for specifying, measuring and evaluating AI system quality. The characteristics and sub-characteristics detailed in the model also provide a set of quality characteristics against which stated quality requirements can be compared for completeness.

2 Normative references

3 Terms and definitions

3.1 General

3.2 Product quality

3.3 Quality in use

4 Abbreviated terms

5 Product quality model

5.1 General

5.2 User controllability

5.3 Functional adaptability

5.4 Functional correctness

5.5 Robustness

5.6 Transparency

5.7 Intervenability

6 Quality in use model

6.1 General

6.2 Societal and ethical risk mitigation

6.3 Transparency

Bibliography

ISO/IEC 25059:2023

Software engineering. Systems and software Quality Requirements and Evaluation (SQuaRE). Quality model for AI systems

Current, Under Review

Published:

28 Jun 2023