Introduction

Digital data delivers value by enhancing all aspects of organizational performance including:

— operational effectiveness and efficiency;
— safety;
— reputation with customers and the wider public;
— compliance with statutory regulations;
— consumer costs, revenues and stock prices.

The influence on performance originates from data being the formalized representation of information; this information enables organizations to make reliable decisions. This decision making can be performed by human beings directly and also by automated data processing including artificial intelligence systems.

Through widespread adoption of digital computing and associated communication technologies, organizations become dependent on digital data. This dependency amplifies the negative consequences of lack of quality in this data. These consequences are the decrease of organizational performance.

The biggest impact of digital data comes from the data having a structure that reflects the nature of the subject matter and from the data also being computer processable (machine readable) rather than just being for a person to read and understand.

The content of ISO 9000 explains that quality is not an abstract concept of absolute perfection. Quality is actually the conformance of characteristics to requirements and, thus, any item of data can be of high quality for one use but not for another use that has differing requirements.

EXAMPLE 1

When storing start times for meetings, a calendar application requires less precision than a control system would for storing the times at which to activate a propulsion unit during a spaceflight.

The nature of digital data is fundamental to establishing requirements that are relevant to the specific decisions that are made by each organization.

EXAMPLE 2

ISO/TS 8000‑1 identifies that data has syntactic (format), semantic (meaning) and pragmatic (usefulness) characteristics.

To support the delivery of high‑quality data, the ISO 8000 series addresses:

— data governance, data quality management and maturity assessment;

EXAMPLE 3

ISO 8000‑61 specifies a process reference model for data quality management.
— creating and applying requirements for data and information;

EXAMPLE 4

ISO 8000‑110 specifies how to exchange characteristic data that is master data.
— monitoring and measuring data and information quality;

EXAMPLE 5

ISO 8000‑8 specifies approaches to measuring data and information quality.
— improving data and, consequently, information quality;

EXAMPLE 6

This document specifies an approach to data profiling, which identifies opportunities to improve data quality.
— issues that are specific to the type of content in a data set.

EXAMPLE 7

ISO/TS 8000‑311 specifies how to address quality considerations for product shape data.

Data quality management covers all aspects of data processing, including creating, collecting, storing, maintaining, transferring, exploiting and presenting data to deliver information.

Effective data quality management is systemic and systematic, requiring an understanding of the root causes of data quality issues. This understanding is the basis for not just correcting existing nonconformities but also implementing solutions that prevent future reoccurrence of those nonconformities.

EXAMPLE 8

If a data set includes dates in multiple formats including “yyyy‑mm‑dd”, “mm‑dd‑yy” and “dd‑mm‑yy”, then data cleansing can correct the consistency of the values. However, such cleansing requires additional information to resolve ambiguous entries (e.g. “04‑05‑20”) and cannot address any process issues and people issues, including training, that have caused the inconsistency.

As a contribution to this overall capability of the ISO 8000 series, this document specifies an approach to data profiling, which involves applying analysis techniques to data in actual use. This analysis generates a profile consisting of the structure, columns and relationships of the data. The profile provides the basis for identifying opportunities to improve data quality by establishing new explicit rules for the data. The approach also typically produces greater effect from repeated application to uncover issues progressively.

Organizations can use this document on its own or in conjunction with other parts of the ISO 8000 series.

This document supports activities that affect:

— one or more information systems;
— data flows within the organization and with external organizations;
— any phase of the data life cycle.

By implementing parts of the ISO 8000 series, an organization achieves the following benefits:

— establishing reliable foundations for digital transformation;
— recognizing how data in digital form has become a fundamental asset class that organizations rely on to deliver value;
— securing evidence‑based trustworthiness of data and information for all stakeholders;
— creating portable data that protects against the loss of intellectual property and that is reusable across the organization and applications;
— achieving traceability of data back to original sources;
— ensuring all stakeholders work with common understanding of explicit data requirements.

ISO/TS 8000‑1 provides a detailed explanation of the structure and scope of the ISO 8000 series.

Annex A contains an identifier that unambiguously identifies this document in an open information system.

PD ISO/TS 8000-81:2021

Data quality - Data quality assessment. Profiling

Current

Published:

31 May 2021