Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO member bodies). The work of preparing International Standards is normally carried out through ISO technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. International organizations, governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of ISO documents should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of any patent rights identified during the development of the document will be in the Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).

Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.

This document was prepared by Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 42, Artificial intelligence.

This document is intended to be used in conjunction with all parts of the ISO/IEC 5259 series.

A list of all parts in the ISO/IEC 5259 series can be found on the ISO website.

Any feedback or questions on this document should be directed to the user’s national standards body. A complete listing of these bodies can be found at www.iso.org/members.html.

Introduction

Visualization can be used to augment data quality management by displaying a data quality measure generated by the measurement function in a tangible and meaningful manner for assessment by the stakeholder. Visualization can be used in any data quality management process in a data quality management life cycle as part of the development and making of the artificial intelligence (AI) system. For example, it is useful as part of data quality reporting for documenting the data quality management process. It can also stimulate cognitive responses from the stakeholder in exploratory data analysis which can lead to more insights (e.g. detection of missing data, outliers, anomalies, deviations, errors, making comparisons and potential relationships among the observations). On the other hand, visualization also has its pitfalls that stem from cognitive biases such as pareidolia and apophenia.

Visualization can also help in explaining to stakeholders how the application built from the data makes its predictions by providing some transparency to the choice of and input to machine learning (ML) algorithms. This can contribute to the trustworthiness of an AI system by stakeholders who use the AI system and have different expectations.

The background of data quality management is described in Clause 5. A visualization framework for data quality based on data quality management concepts is described in Clause 6. Illustration of the application of the visualization framework with practical use cases is presented in Clause 7. Annex A provides information on AI stakeholders’ perspectives and Annex B provides information on database properties.

1 Scope

This document describes a visualization framework for data quality in analytics and machine learning (ML). The aim is to enable stakeholders using visualization methods to assess the results of data quality measures. This visualization framework supports data quality goals.

2 Normative references

3 Terms and definitions

4 Symbols and abbreviated terms

5 Data quality management

5.1 General

5.2 Data life cycle stages

5.3 Data quality management life cycle (DQMLC)

5.4 Data quality concept framework

5.5 Data quality management and visualization

6 Visualization framework for data quality

6.1 General

6.2 Stakeholders and their perspectives

6.3 Dataset properties

6.4 Data quality management life cycle stages and processes

6.5 Data quality model

6.6 Data quality measures

6.7 Data quality assessment

6.8 Applying the visualization framework

7 Data visualization

7.1 General

7.2 Visualization considerations

7.3 Visualization examples

7.4 Summary

Bibliography