Foreword

ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO member bodies). The work of preparing International Standards is normally carried out through ISO technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. International organizations, governmental and non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.

The procedures used to develop this document and those intended for its further maintenance are described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of ISO documents should be noted. This document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).

ISO draws attention to the possibility that the implementation of this document may involve the use of (a) patent(s). ISO takes no position concerning the evidence, validity or applicability of any claimed patent rights in respect thereof. As of the date of publication of this document, ISO had not received notice of (a) patent(s) which may be required to implement this document. However, implementers are cautioned that this may not represent the latest information, which may be obtained from the patent database available at www.iso.org/patents. ISO shall not be held responsible for identifying any or all such patent rights.

Any trade name used in this document is information given for the convenience of users and does not constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions related to conformity assessment, as well as information about ISO's adherence to the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www.iso.org/iso/foreword.html.

This document was prepared by Technical Committee ISO/TC 46, Information and documentation, Subcommittee SC 4, Technical interoperability.

Any feedback or questions on this document should be directed to the user’s national standards body. A complete listing of these bodies can be found at www.iso.org/members.html.

Introduction

The Data Documentation Initiative (DDI) is an ongoing program within the social, behavioural and economic (SBE) data community for documenting SBE data. The program is managed by a consortium, the DDI Alliance^[6^], that comprises university data libraries, university and national data archives, research centres, national statistical offices, consultancies, and software development organizations. The suite of products under DDI include several standards and other products designed to describe data and the processes used to produce them.

DDI first began in 1995 to build a framework for describing a codebook, the document describing the variables, questions, code lists, classifications, methodologies, and usage guidance used to collect, organize, and delimit the data in research studies. This document was built using XML (eXtensible Markup Language) and first released in 2001 as DDI-Codebook (DDI-C)^[8^]. It has been updated periodically since and the current version is numbered 2.5.

In 2003, the DDI community realized an expanded version of DDI-C that addressed the descriptive needs of data producers, such as national statistical offices, was needed. These requirements included descriptions of the methodologies used in production, which include questionnaires, sampling, and weighting. These are used in probability based statistical sample surveys, the standard design for surveys conducted by national statistical offices. . Further, the ability to reuse descriptions of variables, questions, and other artefacts was considered essential for efficiently documenting ongoing statistical surveys and for supporting comparability, interoperability, and integration over time. This effort, also developed using XML, was called DDI-Lifecycle (DDI-L).^[11^] Its current version is numbered 3.3.

Reacting to a dramatic decline in response rates to their surveys, survey organizations began looking into enhancing their data by integrating them with data from outside sources. These data come in a variety of formats, with inconsistent quality and coverage, and often aren’t based on a probability sample. A framework of descriptions, intended to be independent of SBE and to support data integration, was designed to address this problem. The result was the DDI Alliance developed the Cross-Domain Integration (DDI-CDI) standard^[9^]. It was released in early 2025.

Along with the standards, the DDI Alliance advocated the development of several supporting products, which are:

XKOS^[13^] – eXtended Knowledge Organization System is an extension of the W3C SKOS (Simple Knowledge Organization System) used to render concept systems in RDF (Resource Description Framework). XKOS extends SKOS in two major ways, it
- supports levels in hierarchies, which are used to organize statistical classifications
- includes the semantics for some temporal and sequential relations;
SDTL^[12^] – Structured Data Transformation Language is a mid-level language for documenting the processing steps in a data production environment;
Controlled Vocabularies^[9^] – A series of category sets and code lists for use as common ways to populate some descriptors in the DDI standards. An example is the names for the kinds of sampling schemes available.

Beginning around 1970, several researchers independently coined the term metadata. This loosely meant “data about data”, and it referred to descriptions of data. Since then, the concept of metadata has expanded, and the term is now applied to descriptions of any object or resource, not just to data. This expanded notion of metadata is how the term is understood. Each of the DDI standards describes more than data.

Given this understanding of metadata, all the DDI standards are metadata standards. They address the organization and formats of the metadata needed for describing data and related artefacts. This mostly applies to SBE data, but the DDI-CDI standard addresses data independent of the source.

This document describes several common aspects of all the DDI standards and other products. The DDI standards all use the notion of a variable, and these roughly correspond to a column of data organized in the rectangular format. The focus is on the common notion of a variable and the organization of its description.

Each DDI standard and the other products are used to describe parts of the data lifecycle. The phases of this lifecycle are defined, and which phases are addressed by the DDI standards and products are identified.

Of the DDI standards managed using a UML (Unified Modelling Language) model now, those models are independent of each syntax representation (for example, XML, JSON, RDF, etc.) and are called Platform Independent Models (PIM). Each syntax representation uses its own model, and each is an approximation (as close as possible) to the PIM. They are called Platform Specific Models (PSM).

Using the terminology and ideas in ISO/IEC Guide 2, each PSM conforms to its PIM.

1 Scope

This document describes the shared and interoperable features of the standards and other products under the Data Documentation Initiative (DDI).

2 Normative references

3 Terms and definitions

4 Metadata

5 Variables

6 Variable cascade

7 Unit cascade

8 Value domains

8.1 General

8.2 Value domain structures

8.3 Value domain usages

9 Data lifecycle

10 Model independence

11 Semantic interoperability

Bibliography