Networked Media Open Specifications
HOME OVERVIEW MODEL GUIDANCE GITHUB INFO... TOOLS... IS-XX... MORE... SEARCH

Explanation – Flow Representation

←Explanation - Flow · Index↑ · Explanation - Timing→

Please ensure you have read the Summary and Definitions before considering the further detail covered by this page.

Understanding the Flow Representation Entity

Setting the level of a Flow for a scenario defines the interface that will be used for the data – it defines the data type to be used when handling the data (that is, how the data is encoded and how to interpret it). This interface might be implemented in different ways: for example, using different containers or transports for the interchange of the data. These implementations are known as Flow Representations and can take different approaches to managing these practicalities. For example:

A Flow might be requested from a system using a Flow ID. The expectation is often that a single Flow Representation will be delivered throughout the resulting communication / transportation session even though there might be other Flow Representations associated with the same Flow.

In general, the Flow Representations associated with a Flow will not be identical. Some of the differences might be signalled by technical metadata whereas others might not. Note that no Flow Representation is considered to be canonical.

Notes on the Formal Definition of Flow Representation

For a Flow, any differences in the data communicated by its Flow Representations must be considered negligible for the intended purpose. For example, in a media scenario the Flow Representations must all represent baseband signals that are considered “the same” for practical purposes such that they all result in “the same” presentation being perceived (for example, the differences in video pictures are visually imperceptible).

This constraint allows Flow Representations with very small differences in baseband signal to be members of the same Flow. For example, if a lossy video compression process with a fixed configuration processes a video twice then the Flow Representations produced might not decode to identical baseband video signals for various reasons (such as random processes used in the video encoder). However, as long as these differences are considered negligible (they will not be visually noticeable or affect further graphics processing) then these Flow Representations can still be members of the same Flow. An alternative way to view this is that each Data Object might have a range of values associated with it: values in this range are all regarded as having a variation in quality / accuracy that is negligible for the scenario (in other words, the range of values: results in practically the same presentation; represents practically the same baseband signal).

The Flow Representations belonging to a Flow are constrained to all use the same set of Time Values (because these Time Values are defined by the Flow). However, these Flow Representations are not required to all populate the same set of Time Values with data. For example, one Flow Representation might contain data for a one minute portion of the Flow whereas another Flow Representation might contain data for a three minute portion of the Flow.

A Flow Representation is not given an ID in the model because identifying (in an interchangeable or persistent way) individual Flow Representations is generally not considered to be an important requirement.

Examples of Flow Representations

Example 1: Mono PCM Audio in Different Packaging Formats

In this example, we use a Flow ID of X to represent this audio encoded as PCM. In this scenario the Flow is set at the level of the PCM-coded audio samples (these are endianness-independent).

The same Flow ID is used across a variety of packaging / delivery mechanisms (different Flow Representations).

RTP:

WAV:

Store:

HTTP API:

C++ function:

Flow technical metadata would include properties such as the audio codec (PCM), sampling rate, bit-depth, but not the endianness.

Example 2: Uncompressed Video

In this example, we use a Flow ID of X to represent this uncompressed HD video. In this scenario the Flow is set at the level of the uncompressed video frames (these are byte-packing independent). Note: in this example, if the video were losslessly compressed (for example using VC-2 in a lossless mode) then a different Flow ID would be used.

The same Flow ID is used across a variety of packaging / delivery mechanisms (different Flow Representations).

RTP (RFC 4175 or SMPTE ST 2110):

HD SDI (video carriage):

C++ function:

HTTP API:

Store, MXF file, or similar:

Flow technical metadata would include properties such as: 1920x1080, YUV, 10-bit, 4:2:0, colour space but not the byte packing.

Example 3: AVC Intra

In this example, we use a Flow ID of X to represent this AVC Intra coded video. In this example, we assume that each video frame is encoded only once by one encoder (with one set of settings) across all the Flow Representations with a Flow ID of X.

In this scenario the Flow is set at the level of these compressed video frames. More specifically, all the member Flow Representations are such that only bitstream assembly operations need to be carried out to create an AVC Intra compliant bitstream (or the bitstream is already compliant).

The same Flow ID is used across a variety of packaging / delivery mechanisms (different Flow Representations).

RTP (RFC 6184):

MXF – example 1:

MXF – example 2:

HTTP API, Store, etc:

MP4 file (H.264 mapped as per ISO/IEC 14496-15):

Example 4: Lossy Compression in a Cloud Environment (Distributed Encoders etc)

In this example, we use a Flow ID of X to identify the Flow Representations that can be used interchangeably subject to a set of defined rules.

For example, this could be useful for:

The assumption is that the baseband signal is virtually identical in each case so the Flow Representations can belong to the same Flow, but the properties of the encoding itself might be quite different. However, in this example the encoders are configured and operated to ensure that the encodings can be used interchangeably subject to a set of defined rules. For example, GOP placements might be restricted so that Flow Representations can be safely switched between at known intervals. In general, these rules should enable the client to answer questions such as:

Example 5: Temperature Readings

In this example, a temperature sensor generates readings regularly. Each reading is a 32-bit signed-integer. We use a Flow ID of X to identify these 32-bit signed-integer readings. In this scenario the Flow is set at the “level” of the 32-bit signed-integers (and so there are no restrictions on the serialisation etc of the data).

The same Flow ID is used across a variety of packaging / delivery mechanisms (different Flow Representations).

C++ function:

HTTP API:

File:

Example 6: Weather Observations (Alternative 1)

In this example, a weather station makes weather observations regularly (from a number of sensors). The details of each observation are stored in a JSON data structure using JSON Schema Y. We use a Flow ID of X to identify these JSON-formatted weather observations. In this scenario the Flow is set at the level of the JSON-formatted (using Schema Y) weather observations (these are independent of details of the serialisation).

The following Flow Representations would all have the same Flow ID:

Note:

Example 7: Weather Observations (Alternative 2)

In this alternative we define Flow at a higher level – we say that Flow is at the level of the weather observations themselves and so the same Flow ID is used regardless of serialisation (XML, different JSON Schemas, etc).

The data contained within each observation can be expressed as:

WeatherObservation {
  int16 temperature;
  uint16 humidity;
}

The same Flow ID is used across different implementations of this interface (different Flow Representations).

C++ function:

HTTP API:

←Explanation - Flow · Index↑ · Explanation - Timing→