Introduction

High-quality software products and computer systems are crucial to stakeholders. Quality models, quality requirements, quality measurement, and quality evaluation are standardized within the International Standards on SQuaRE, see Annex A for further information.

AI systems require additional properties and characteristics of systems to be considered, and stakeholders have varied needs. AI systems have different properties and characteristics. For example, AI systems can:

— replace human decision-making;
— be based on noisy, or incomplete data;
— be probabilistic;
— adapt during operation.

According to ISO/IEC TR 24028,^[²^] trustworthiness has been understood and treated as both an ongoing organizational process as well as a non-functional requirement specifying emergent properties of a system — that is, a set of inherent characteristics with their attributes — within the context of quality of use as indicated in ISO/IEC 25010.

ISO/IEC TR 24028 discusses the applicability to AI systems of that have been developed for conventional software. According to ISO/IEC TR 24028, does not sufficiently address the data-driven unpredictable nature of AI systems. While considering the existing body of work, ISO/IEC TR 24028 identifies the need for developing new International Standards for AI systems that can go beyond the characteristics and requirements of conventional software development.

ISO/IEC TR 24028 contains a related discussion on different approaches to testing and evaluation of AI systems. It states that for testing of an AI system, modified versions of existing software and hardware verification and validation techniques are needed. It identifies several conceptual differences between many AI systems and conventional systems and concludes that “the ability of the [AI] system to achieve the planned and desired result … may not always be measurable by conventional approaches to software testing”. Testing of AI systems is addressed in ISO/IEC TR 29119‑11:2020.^[³^]

This document outlines an application-specific AI system extension to the SQuaRE quality model specified in ISO/IEC 25010.

AI systems perform tasks. One or more tasks can be defined for an AI system. Quality requirements can be specified for the evaluation of task fulfilment.

The quality model is considered from two perspectives, product quality as described in Clause 5 and quality in use in Clause 6. The relevance of these terms is explained, and links to other standardization deliverables (e.g. the ISO/IEC 24029 series^[⁴^][⁵^]) are highlighted.

ISO/IEC 25012:2008 ^[ ⁶ ^] contains a model for data quality that is complementary to the model defined in this document. ISO/IEC 25012:2008 is being extended for AI systems by the ISO/IEC 5259 series.^[⁷^]

BS ISO/IEC 25059:2023

Software engineering. Systems and software Quality Requirements and Evaluation (SQuaRE). Quality model for AI systems

Current, Under Review

Published:

31 Jul 2023