Introduction

Artificial neural networks have been adopted for a broad range of tasks in multimedia analysis and processing, media coding, data analytics and many other fields. Their recent success is based on the feasibility of processing much larger and complex neural networks (deep neural networks, DNNs) than in the past, and the availability of large-scale training data sets. As a consequence, trained neural networks contain a large number of parameters and weights, resulting in a quite large size (e.g. several hundred MBs). Many applications require the deployment of a particular trained network instance, potentially to a larger number of devices, which may have limitations in terms of processing power and memory (e.g. mobile devices or smart cameras), and also in terms of communication bandwidth. Any use case, in which a trained neural network (or its updates) needs to be deployed to a number of devices thus benefits from a standard for the compressed representation of neural networks.

Considering the fact that compression of neural networks is likely to have a hardware dependent and hardware independent component, this document is designed as a toolbox of compression technologies. Some of these technologies require specific representations in an exchange format (i.e. sparse representations, adaptive quantization), and thus a normative specification for representing outputs of these technologies is defined. Others do not at all materialize in a serialized representation (e.g. pruning), however, also for the latter ones required metadata is specified. This document is independent of a particular neural network exchange format, and interoperability with common formats is described in the annexes.

This document thus defines a high-level syntax that specifies required metadata elements and related semantics. In cases where the structure of binary data is to be specified (e.g. decomposed matrices) this document also specifies the actual bitstream syntax of the respective block. Annexes to the document specify the requirements and constraints of compressed neural network representations; as defined in this document; and how they are applied.

— Annex A specifies the implementation of this document with the Neural Network Exchange Format (NNEF¹), defining the use of NNEF to represent network topologies in a compressed neural network bitstream.
— Annex B provides recommendations for the implementation of this document with the Open Neural Network Exchange Format (ONNX®)², defining the use of ONNX to represent network topologies in a compressed neural network bitstream.
— Annex C provides recommendations for the implementation of this document with the PyTorch®³ format, defining the reference to PyTorch elements in the network topology description of a compressed neural network bitstream.
— Annex D provides recommendations for the implementation of this document with the Tensorflow®⁴ format, defining the reference to Tensorflow elements in the network topology description of a compressed neural network bitstream.
— Annex E provides recommendations for the carriage of tensors compressed according to this document in third party container formats.
— Annex F provides recommendations for the naming of common performance metrics to specify the metric that was used for validation.
— Annex G provides recommendations for implementing the encoding side of some of the compression tools.

The compression tools described in this document have been selected and evaluated for neural networks used in applications for multimedia description, analysis and processing. However, they may be useful for the compression of neural networks used in other applications and applied to other types of data.

ISO/IEC 15938-17:2024

Information technology. Multimedia content description interface. - Part 17: Compression of neural networks for multimedia content description and analysis

Current

Published:

10 Jan 2024