National foreword

This British Standard is the UK implementation of ISO/IEC 4944:2024.

The UK participation in its preparation was entrusted to Technical Committee ICT/2, ICT Accessibility.

A list of organizations represented on this committee can be obtained on request to its committee manager.

Contractual and legal considerations

This publication has been prepared in good faith, however no representation, warranty, assurance or undertaking (express or implied) is or will be made, and no responsibility or liability is or will be accepted by BSI in relation to the adequacy, accuracy, completeness or reasonableness of this publication. All and any such responsibility and liability is expressly disclaimed to the full extent permitted by the law.

This publication is provided as is, and is to be used at the recipient’s own risk.

The recipient is advised to consider seeking professional guidance with respect to its use of this publication.

This publication is not intended to constitute a contract. Users are responsible for its correct application.

Compliance with a British Standard cannot confer immunity from legal obligations.

This British Standard was published under the authority of the Standards Policy and Strategy Committee on 31 October 2024.

Foreword

ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.

The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).

ISO and IEC draw attention to the possibility that the implementation of this document may involve the use of (a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any claimed patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not received notice of (a) patent(s) which may be required to implement this document. However, implementers are cautioned that this may not represent the latest information, which may be obtained from the patent database available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held responsible for identifying any or all such patent rights.

Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.

This document was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 35, User interfaces.

Any feedback or questions on this document should be directed to the user’s national standards body. A complete listing of these bodies can be found at www.iso.org/members.html and www.iec.ch/national-committees.

Introduction

As smart devices such as smart phones and smart speakers have become popular, new forms of user interfaces have been introduced. One aspect of the new interfaces is mimicking natural ways of interacting with the devices and utilizing speech, gestures, body language, eye gaze and other natural features of human communication through natural user interfaces (NUIs), which allow for faster, smoother, intuitive, and smarter interaction. However, while the concept of NUI is well defined, practical applications of NUI still need to be developed. This is strongly related to the lack of usability of NUI in smart devices.

When specified users of systems, products or services achieve their specified goals with effectiveness, efficiency and satisfaction in a specified context of use (as defined in ISO/IEC 9241‑11), usability can be evaluated. Effectiveness is considered as the ability of users to complete tasks with the system (e.g. the NUI). Efficiency measures resources taken by users to complete specified tasks. Satisfaction measures comfort levels of user experience along with acceptance.

It is important that the usability evaluation of NUI concentrates on recognizing the user's intention and focuses on multiple modalities of information flows between the users and the devices.

This document describes a framework for evaluating the usability of NUI. Although the framework can be applied in most general contexts, it is specifically focused on optimizing user experiences in terms of NUI.

1 Scope

This document provides a framework, requirements and recommendations for evaluating the usability of a natural user interface (NUI) for systems, products or services. Measuring and reporting the usability of the NUI are also described. The usability evaluation focuses on the efficiency, effectiveness and satisfaction of the NUI. This document can also be applied to the usability evaluation of other user interfaces (UIs) for emerging technologies.

2 Normative references

3 Terms and definitions

3.1 Terms related to natural user

3.2 Terms related to usability

4 Natural user interfaces (NUI)

5 Evaluating usability of a natural user interface (NUI)

5.1 Framework

5.2 NUI as a system, product or service being evaluated

5.3 Evaluating usability for diverse NUI users

5.4 Evaluating usability for goals, tasks and interactions of NUI

5.5 Evaluating usability for diverse contexts of use of NUI

5.6 Components of usability

5.7 Evaluation based on use

6 Measuring usability of a NUI

6.1 Purposes of and measures for evaluation

6.2 Measures for evaluating the user interface of NUI

6.3 Measures for evaluating naturalness of interactions with NUI

7 Reporting usability evaluation results

7.1 Content elements for reporting usability evaluations

7.2 Format for reporting usability evaluations

Bibliography

BS ISO/IEC 4944:2024

Information technology. User interfaces. Evaluating usability of natural user interfaces

Current

Published:

31 Oct 2024