RTP Payload Format for Visual Volumetric Video-based Coding (V3C)

3.1. Definitions

3.2. Abbreviations

ACL atlas coding layer¶

AP aggregation packet¶

AU aggregation unit¶

CVS coded V3C sequence¶

DON decoding order number¶

IRAP intra random access point¶

MTU maximum transmission unit¶

NAL network abstraction layer¶

NALU NAL unit¶

RBSP raw byte sequence payload¶

V3C visual volumetric video-based coding¶

VPS V3C parameter set¶

4.1. Overview of the V3C codec (informative)

V3C encoding of a volumetric frame is achieved through a conversion of the volumetric frame from its 3D representation into multiple 2D representations and a generation of associated data documenting such conversions and transformations. The associated data, also known as the atlas data, provides information on how to reproject the 2D representations back into the 3D volumetric frame.¶

2D representations, known as V3C video components, of volumetric frame are encoded using traditional 2D video codecs. V3C video component may, for example, include occupancy, geometry, or attribute data. The occupancy data informs a V3C decoder which pixels in other V3C video components contribute to reconstructed 3D representation. The geometry data describes information on the position of the reconstructed voxels, while attribute data provides additional properties for the voxels, e.g., colour or material information.¶

Atlas data, known as V3C atlas component, provides information to interpret V3C video components and enables the reconstruction from a 2D representation back into a 3D representation of volumetric frame. Atlas data is composed of a collection of patches. Each patch identifies a region in the V3C video components and provides information necessary to perform the appropriate inverse projection of the indicated region back into 3D space. The shape of the patch region is determined by a 2D bounding box associated with each patch as well as their coding order. The shape of these patches is also further refined based on occupancy data.¶

To enable parallelization, random access, as well as a variety of other functionalities, an atlas frame can be divided into one or more rectangular partitions referred to as tiles. Tiles are not allowed to overlap and should be independently decodable. An atlas frame may contain regions that are not associated with any tile or patch.¶

The binary form of V3C video components, i.e., video bitstream, and V3C atlas components, i.e., atlas bitstream, can be grouped and represented by a single V3C bitstream. The V3C bitstream is composed of a set of V3C units. Each V3C unit has a V3C unit header and a V3C unit payload. The V3C unit header describes the V3C unit type for the payload. V3C unit payload contains V3C video components, V3C atlas components or a V3C parameter set. V3C video components, i.e., occupancy, geometry, or attribute components, correspond to video data units (e.g., NAL units defined in [ISO.IEC.23008-2]) that could be decoded by an appropriate video decoder. An example of V3C bitstream consisting of a V3C parameter set, atlas bitstream and three video component bitsreams (geometry, occupancy, attribute) is provided in Figure 1.¶

  +-------------------+------------------+-------------------+
  | V3C Unit(V3C_VPS) | V3C Unit(V3C_AD) | V3C Unit(V3C_GVD) |
  +------------------++------------------+-----------------+-+---
  |V3C Unit(V3C_OVD) | V3C Unit(V3C_AVD) | V3C Unit(V3C_AD)| ...
  +------------------+-------------------+-----------------+-----

4.2. V3C parameter set (informative)

While this memo intends to describe encapsulation of V3C atlas data, aspects related to signalling of V3C parameter set are also considered. V3C parameter set is encapsulated in its own V3C unit, which allows decoupling the transmission of V3C parameter set from the V3C video and atlas components. V3C parameter set may be transmitted by external means (e.g., as a result of the capability exchange) or through a (reliable or unreliable) control protocol. This memo provides information on how a V3C parameter set may be signalled as part of session description protocol, see Section 9.¶

Generally, it is useful to signal V3C parameter set out-of-band, because it describes what overall resources are needed to decode and reconstruct the associated V3C bitstream. Signalling it dynamically as part of an RTP stream might result in undefined behaviour when receiver does not have the required capabilities to decode the received V3C video component sub-bitstreams or when reconstruction process relies on information that the receiver does not support.¶

4.3. V3C atlas and video components (informative)

v3c_unit_header( ) {
 unsigned int(5) vuh_unit_type;
 if( vuh_unit_type == V3C_AVD || vuh_unit_type == V3C_GVD ||
   vuh_unit_type == V3C_OVD || vuh_unit_type == V3C_AD ||
   vuh_unit_type == V3C_CAD || vuh_unit_type == V3C_PVD ) {
   unsigned int(4) vuh_v3c_parameter_set_id;
  }
  if( vuh_unit_type == V3C_AVD || vuh_unit_type == V3C_GVD ||
    vuh_unit_type == V3C_OVD || vuh_unit_type == V3C_AD ||
    vuh_unit_type == V3C_PVD ) {
    unsigned int(6) vuh_atlas_id;
  }
  if( vuh_unit_type == V3C_AVD ) {
    unsigned int(7) vuh_attribute_index;
    unsigned int(5) vuh_attribute_partition_index;
    unsigned int(4) vuh_map_index;
    unsigned int(1) vuh_auxiliary_video_flag;
  }
  else if( vuh_unit_type == V3C_GVD ) {
    unsigned int(4) vuh_map_index;
    unsigned int(1) vuh_auxiliary_video_flag;
    bit(12) vuh_reserved_zero_12bits;
  }
  else if( vuh_unit_type == V3C_OVD || vuh_unit_type == V3C_AD ||
      vuh_unit_type == V3C_PVD) {
    bit(17) vuh_reserved_zero_17bits;
  }
  else if( vuh_unit_type == V3C_CAD ) {
    bit(23) vuh_reserved_zero_23bits;
  }
  else {
    bit(27) vuh_reserved_zero_27bits;
  }
}

nal_unit_header(){
    bit(1) nal_forbidden_zero_bit;
    bit(6) nal_unit_type;
    bit(6) nal_layer_id;
    bit(3) nal_temporal_id_plus1;
}
nal_unit(NumBytesInNalUnit){
    nal_unit_header();
    NumBytesInRbsp = 0;
    for( i = 2; i < NumBytesInNalUnit; i++ )
      bit(8) rbsp_byte[ NumBytesInRbsp++ ];
}

4.4. Systems and transport interfaces (informative)

In addition to releasing specifications on V3C applications [ISO.IEC.23090-5] and [ISO.IEC.23090-12], MPEG conducted further systems level work on file formats to encapsulate compressed V3C content. The seventh edition of the ISOBMFF specification [ISO.IEC.14496-12] introduces a new media handler 'volv', intended to support volumetric visual media. It also specifies other structures to enable development of derived specifications detailing how various volumetric visual media may be stored in ISOBMFF.¶

One of such derived specifications is [ISO.IEC.23090-10], which defines how V3C content can be stored in a file and streamed over DASH. To a large extent ISO/IEC 23090-10 focuses on describing how ISOBMFF boxes and syntax elements may be used to store volumetric media, but in some cases new boxes and syntax elements are introduced to accommodate the fundamentally different type of new media. While the specification is not directly relevant for defining RTP payload format for V3C atlas data, it is a useful resource that may be considered especially when designing ingestion of encoded V3C content into RTP streaming pipelines.¶

5.1. General

This section describes details related to V3C atlas RTP payload format definitions. Aspects related to RTP header, RTP payload header and general payload structure are considered. RTP payload format(s) for video components is defined in their respective RTP payload format specifications depending on the video codec used.¶

5.2. RTP header

The format of the RTP header is specified in [RFC3550] and replicated below in Figure 2 for convenience. This payload format uses the fields of the header in a manner consistent with that specification.¶

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |V=2|P|X|  CC   |M|     PT      |       sequence number         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                           timestamp                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           synchronization source (SSRC) identifier            |
  +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
  |            contributing source (CSRC) identifiers             |
  |                             ....                              |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The RTP header information to be set according to this RTP payload format is set as follows:¶

Marker bit (M): 1 bit¶

Set for the last packet of the access unit, carried in the current RTP stream. This is in line with the normal use of the M bit in video formats to allow an efficient playout buffer handling.¶

Payload Type (PT): 7 bits¶

The assignment of an RTP payload type for this new packet format is outside the scope of this document and will not be specified here. The assignment of a payload type MUST be performed either through the profile used or in a dynamic way.¶

Sequence Number (SN): 16 bits¶

Set and used in accordance with [RFC3550]¶

Timestamp (32 bits):¶

The RTP timestamp is set to the sampling timestamp of the content. A 90 kHz clock rate MUST be used.¶

If the NAL unit has no timing properties of its own (e.g., parameter set and SEI NAL units), the RTP timestamp MUST be set to the RTP timestamp of the coded atlas of the access unit in which the NAL unit (according to Section 8.4.5.3 of [ISO.IEC.23090-5]) is included.¶

Receivers MUST use the RTP timestamp for the display process, even when the bitstream contains atlas frame timing SEI messages as specified in [ISO.IEC.23090-5].¶

Synchronization source (SSRC): 32 bits¶

Used to identify the source of the RTP packets. By definition a single SSRC is used for all parts of a single bitstream.¶

The remaining RTP header fields are used as specified in [RFC3550].¶

5.3. RTP payload header

The first two bytes of the payload of an RTP packet are referred to as the payload header. The payload header consists of the same fields (F, NUT, NLI, and TID) as the NAL unit header as shown in Section 4.3.2, irrespective of the type of the payload structure. For convenience the structure of RTP payload header is described below in Figure 3.¶

   0                   1
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |F|    NUT    |    NLI    | TID |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

F: nal_forbidden_zero_bit as specified in [ISO.IEC.23090-5] MUST be equal to 0.¶

NUT: nal_unit_type as specified in [ISO.IEC.23090-5] defines the type of the RBSP data structure contained in the NAL unit payload. NUT value could carry other meaning depending on the RTP packet type.¶

NLI: nal_layer_id as specified in [ISO.IEC.23090-5] defines the identifier of the layer to which an ACL NAL unit belongs or the identifier of a layer to which a non-ACL NAL unit applies.¶

TID: nal_temporal_id_plus1 minus 1 as specified in [ISO.IEC.23090-5] defines a temporal identifier for the NAL unit. The value of nal_temporal_id_plus1 MUST NOT be equal to 0.¶

5.4. Payload structures

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |      RTP payload header       |      DONL (conditional)       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
  |      v3c-tile-id (cond)       |                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
  |                                                               |
  |                        NAL unit data                          |
  |                                                               |
  |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               :...OPTIONAL RTP padding        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  RTP payload header (NUT=56)  |      v3c-tile-id (cond)       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                                                               |
  |                  Two or more aggregation units                |
  |                                                               |
  |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               :...OPTIONAL RTP padding        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  DOND (cond)  /  DONL (cond)  |      v3c-tile-id (cond)       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
  |            NALU size          |                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
  |                                                               |
  |                            NAL unit                           |
  |                                                               |
  |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  RTP payload header (NUT=57)  |   FU header   |  DONL (cond)  |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
  |  DONL (cond)  |    v3c-tile-id (cond)         |               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
  |                                                               |
  |                          FU payload                           |
  |                                                               |
  |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               :...OPTIONAL RTP padding        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   0 1 2 3 4 5 6 7
  +-+-+-+-+-+-+-+-+
  |S|E|    FUT    |
  +-+-+-----------+

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |V=2|P|X|  CC   |M|     PT      |       sequence number         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                           timestamp                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           synchronization source (SSRC) identifier            |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            contributing source (CSRC) identifiers             |
  |                             ....                              |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  RTP payload header (NUT=57)  |1|0|    FUT    |               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
  |                                                               |
  |                          FU payload                           |
  |                                                               |
  |                                                               |
  |                                                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |V=2|P|X|  CC   |M|     PT      |       sequence number         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                           timestamp                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           synchronization source (SSRC) identifier            |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |            contributing source (CSRC) identifiers             |
  |                             ....                              |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  RTP payload header (NUT=57)  |0|1|    FUT    |               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
  |                                                               |
  |                          FU payload                           |
  |                                                               |
  |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               :...OPTIONAL RTP padding        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

5.5. Decoding order number

For each atlas NAL unit, the variable AbsDon is derived, representing the decoding order number that is indicative of the NAL unit decoding order. Let NAL unit n be the n-th NAL unit in transmission order within an RTP stream.¶

If sprop-max-don-diff is equal to 0 for all the RTP streams carrying the atlas bitstream, AbsDon[n], the value of AbsDon for NAL unit n, is derived as equal to n.¶

Otherwise (sprop-max-don-diff is greater than 0 for any of the RTP streams), AbsDon[n] is derived as follows, where DON[n] is the value of the variable DON for NAL unit n:¶

  If (n == 0)
    AbsDon[n] = DON[0]
  Else
    If (DON[n] == DON[n-1])
      AbsDon[n] = AbsDon[n-1]
    If (DON[n] > DON[n-1] and DON[n] - DON[n-1] < 32768)
      AbsDon[n] = AbsDon[n-1] + DON[n] - DON[n-1]
    If (DON[n] < DON[n-1] and DON[n-1] - DON[n] >= 32768)
      AbsDon[n] = AbsDon[n-1] + 65536 - DON[n-1] + DON[n]
    If (DON[n] > DON[n-1] and DON[n] - DON[n-1] >= 32768)
      AbsDon[n] = AbsDon[n-1] - (DON[n-1] + 65536 - DON[n])
    If (DON[n] < DON[n-1] and DON[n-1] - DON[n] < 32768)
      AbsDon[n] = AbsDon[n-1] - (DON[n-1] - DON[n])

For any two NAL units m and n, the following applies:¶

• AbsDon[n] greater than AbsDon[m] indicates that NAL unit n follows NAL unit m in NAL unit decoding order.¶

• When AbsDon[n] is equal to AbsDon[m], the NAL unit decoding order of the two NAL units can be in either order.¶

• AbsDon[n] less than AbsDon[m] indicates that NAL unit n precedes NAL unit m in decoding order.¶

7.1. Media type registration

The receiver MUST ignore any parameter unspecified in this memo.¶

Type name: application¶

Subtype name: v3c¶

Required parameters: N/A¶

Optional parameters: sprop-v3c-unit-header, sprop-v3c-unit-type, sprop-v3c-vps-id, sprop-v3c-atlas-id, sprop-v3c-attr-idx, sprop-v3c-attr-part-idx, sprop-v3c-map-idx, sprop-v3c-aux-video-flag, sprop-v3c-parameter-set, sprop-v3c-tile-id, sprop-v3c-tile-id-pres, sprop-v3c-atlas-data, sprop-v3c-common-atlas-data, sprop-v3c-sei, v3c-ptl-level-idc, v3c-ptl-tier-flag, v3c-ptl-codec-idc, v3c-ptl-toolset-idc, v3c-ptl-rec-idc and sprop-max-don-diff.¶

Encoding considerations: This type is only defined for transfer via RTP [RFC3550].¶

Security considerations: Please see Section 11.¶

Interoperability considerations: N/A¶

Published specification: Please refer to [ISO.IEC.23090-5]¶

Applications that use this media type: Any application that relies on V3C-based media services over RTP¶

Additional information: N/A¶

Person & email address to contact for further information:¶

Intended usage: COMMON¶

Restrictions on usage: N/A¶

Author: See Authors' Addresses section of this memo.¶

Change controller: IETF [email protected]¶

Provisional registration? (standards tree only): No¶

7.2. Optional parameters definition

    sprop-v3c-unit-header:

provides a V3C unit header bytes defined in [ISO.IEC.23090-5]. The value contains base64 encoded [RFC4648] representation of the 4 bytes of V3C unit header.¶

    sprop-v3c-unit-type:

sprop-v3c-unit-type provides a V3C unit type value corresponding to vuh_unit_type defined in [ISO.IEC.23090-5], i.e., defines V3C sub-bitstream type.¶

    sprop-v3c-vps-id:

sprop-v3c-vps-id provides a value corresponding to vuh_v3c_parameter_set_id defined in [ISO.IEC.23090-5].¶

    sprop-v3c-atlas-id:

sprop-v3c-atlas-id provides a value corresponding to vuh_atlas_id defined in [ISO.IEC.23090-5].¶

    sprop-v3c-attr-idx:

sprop-v3c-attr-idx provides a value corresponding to vuh_attribute_index defined in [ISO.IEC.23090-5].¶

    sprop-v3c-attr-part-idx:

sprop-v3c-attr-part-idx provides a value corresponding to vuh_attribute_partition_index defined in [ISO.IEC.23090-5].¶

    sprop-v3c-map-idx:

sprop-v3c-map-idx provides a value corresponding to vuh_map_index defined in [ISO.IEC.23090-5].¶

    sprop-v3c-aux-video-flag:

sprop-v3c-aux-video-flag provides a value corresponding to vuh_auxiliary_video_flag defined in [ISO.IEC.23090-5].¶

    sprop-v3c-parameter-set:

sprop-v3c-parameter-set provides V3C parameter set bytes as defined in [ISO.IEC.23090-5]. The value contains base64 encoded [RFC4648] representation of the V3C parameter set bytes.¶

    sprop-v3c-tile-id:

sprop-v3c-tile-id indicates that the RTP stream contains only portion of the tiles in the atlas. sprop-v3c-tile-id is a comma-separated (',') list of integer values, which indicate the sprop-v3c-tile-ids that are present in the RTP stream.¶

    sprop-v3c-tile-id-pres:

sprop-v3c-tile-id-pres indicates that the RTP packets contain v3c-tile-id field.¶

    sprop-v3c-atlas-data:

sprop-v3c-atlas-data MAY be used to convey any atlas data NAL units of the V3C atlas sub bitstream for out-of-band transmission. The value is a comma-separated (',') list of encoded representations of the atlas NAL units as specified in [ISO.IEC.23090-5]. The NAL units SHOULD be encoded as base64 [RFC4648] representations.¶

    sprop-v3c-common-atlas-data:

sprop-v3c-common-atlas-data MAY be used to convey common atlas data NAL units of the V3C common atlas sub bitstream for out-of-band transmission. The value is a comma-separated (',') list of encoded representations of the common atlas NAL units (i.e., NAL_CASPS and NAL_CAF_IDR) as specified in [ISO.IEC.23090-5]. The NAL units SHOULD be encoded as base64 [RFC4648] representations.¶

    sprop-v3c-sei:

sprop-v3c-sei MAY be used to convey SEI NAL units of V3C atlas and common atlas sub bitstreams for out-of-band transmission. The value is a comma-separated (',') list of encoded representations of SEI NAL units (i.e., NAL_PREFIX_NSEI and NAL_SUFFIX_NSEI, NAL_PREFIX_ESEI, NAL_SUFFIX_ESEI) as specified in [ISO.IEC.23090-5]. The SEI NAL units SHOULD be encoded as base64 [RFC4648] representations.¶

    v3c-ptl-level-idc:

v3c-ptl-level-idc provides a value corresponding to ptl_level_idc defined in [ISO.IEC.23090-5].¶

    v3c-ptl-tier-flag:

v3c-ptl-tier-flag provides a value corresponding to ptl_tier_flag defined in [ISO.IEC.23090-5].¶

    v3c-ptl-codec-idc:

v3c-ptl-codec-idc provides a value corresponding to ptl_profile_codec_group_idc defined in [ISO.IEC.23090-5].¶

    v3c-ptl-toolset-idc:

v3c-ptl-toolset-idc provides a value corresponding to ptl_profile_toolset_idc defined in [ISO.IEC.23090-5].¶

    v3c-ptl-rec-idc:

v3c-ptl-rec-idc provides a value corresponding to ptl_profile_reconstruction_idc defined in [ISO.IEC.23090-5].¶

    sprop-max-don-diff:

If the transmission order of NAL units in the RTP stream(s) is the same as the decoding and NAL unit output order, this parameter must be equal to 0.¶

Otherwise, if the decoding order of the NAL units of the RTP stream(s) is the same as the NAL unit transmission order but not the same as NAL unit output order, the value of this parameter MUST be equal to 1.¶

Otherwise, this parameter specifies the maximum absolute difference between the decoding order number (i.e., AbsDon) values of any two NAL units naluA and naluB, where naluA follows naluB in decoding order and precedes naluB in transmission order.¶

The value of sprop-max-don-diff MUST be an integer in the range of 0 to 32767, inclusive.¶

When not present, the value of sprop-max-don-diff is inferred to be equal to 0.¶

9.1. V3C format parameters "v3cfmtp" attribute

This memo defines a new attribute for SDP, intended to carry V3C specific media format parameters. Its functionality is similar to "a=fmtp", with the exception that it SHALL be used without the fmt-token and that it can be used also on a session level. The attribute allows to associate V3C specific media format parameters with any media line in SDP.¶

Contact name: See Authors' Addresses section of this memo.¶

Contact email address: See Authors' Addresses section of this memo.¶

Attribute name: v3cfmtp¶

Attribute value: v3cfmtp-value¶

Attribute syntax:¶

  v3cfmtp-value = byte-string
  ; Notes:
  ; - The V3C format parameters are V3C media type parameters and
  ;   need to reflect their syntax.

Attribute semantics: "v3cfmtp-value" is a byte-string, which MUST contain at least one V3C specific media format parameter as defined in this memo. Multiple semicolon separated V3C media parameters MAY be stored in "v3c-format-specific-params" to be conveyed by SDP and given unchanged to the media tool that will use this format.¶

Usage level: session, media¶

Charset dependent: no¶

Purpose: This attribute allows parameters that are specific to a V3C format to be conveyed in a way that SDP does not have to understand them. It allows to associate V3C specific parameters with the session or with any media line. Parameters signalled as part of session level attribute take effect when conflicting parameters are signalled as media level attribute.¶

O/A procedures: v3cfmtp attribute MAY be present both in offers and answers.¶

Mux Category: NORMAL¶

Reference: "this memo"¶

NOTE: (informative) "this memo" to be replaced with the RFC number, once it becomes available.¶

Example: First line describes session level usage of the attribute, signaling a V3C parameter set. Second line describes media level attribute, signaling V3C unit header and profile tier level flag for the associated media line.¶

  a=v3cfmtp:sprop-v3c-parameter-set=AUH/AAAP/zwAAAAAACgIAtEAgQLAIAAUQ
  BACWAM5QEDgQCAIAAAAABP8CzwAAAAAAAAAQAAAtAE/wLPAAAAAAAg=;

  a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;v3c-ptl-tier-flag=1;

9.2. Mapping of payload type parameters to SDP

  m=application 49170 RTP/AVP 98
  a=rtpmap:98 v3c/90000
  a=fmtp:98 sprop-v3c-tile-id=0,1
  a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;v3c-ptl-tier-flag=1;
    sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQAADkA==

  m=video 49170 RTP/AVP 99
  a=rtpmap:99 H265/90000
  a=fmtp:99 sprop-max-don-diff=0;
  a=v3cfmtp:sprop-v3c-unit-header=EAAAAA==

  m=video 49170 RTP/AVP 99
  a=rtpmap:99 H265/90000
  a=v3cfmtp:sprop-v3c-unit-header=KAAAAA==;
    sprop-v3c-parameter-set=AUH/AAAP/zwAAAAAACgIAtEAgQLAIAAUQBACWAM5Q
    EDgQCAIAAAAABP8CzwAAAAAAAAAQAAAtAE/wLPAAAAAAAg=;
    sprop-v3c-atlas-data=SAGAFAQBaKjuXgABQEKA,SgHmIA==,LgFoDOAFAABaAA
    AAAAA+;
    sprop-v3c-common-atlas-data=YAEHgFA=,YgEAMAAAC/B0qcvv/Dbr/pTvb8oq
    fhC5JQVS9jn7kAQT/As9EFyrjRBcmxEQe+j5DuGbTT9mZmZAQAAAoA==

9.3. Grouping framework

Different V3C components MAY be represented by their own respective RTP streams, whose payload formats are defined in the respective specifications. V3C atlas data RTP payload format is defined in this memo, whereas the video component RTP payload formats are defined for example in [RFC6184] or [RFC7798]. A grouping tool, as defined in [RFC5888], is extended to idicate which media lines constitute a V3C representation.¶

Group attribute with V3C type is provided to allow application to identify "m" lines that belong to the same V3C bitstream. Grouping type V3C MUST be used with the group attribute. The tokens that follow are mapped to 'mid'-values of individual media lines in the SDP.¶

    a=group:V3C <tokens>

The following example shows an SDP including four media lines, three describing V3C video components (PT:96=occupancy, PT:97=geometry, PT:98=attribute) and one V3C atlas component (PT:100). All the media lines are grouped under one V3C group. V3C parameter set is provided via session level V3C media format parameter attribute.¶

  ...
  a=group:V3C 1 2 3 4
  a=v3cfmtp:sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQ
    AADkA==
  m=video 40000 RTP/AVP 96
  a=rtpmap:96 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=EAAAAA==
  a=mid:1
  m=video 40002 RTP/AVP 97
  a=rtpmap:97 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=GAAAAA==
  a=mid:2
  m=video 40004 RTP/AVP 98
  a=rtpmap:98 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=IAAAAA==
  a=mid:3
  m=application 40008 RTP/AVP 100
  a=rtpmap:100 v3c/90000
  a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;
  a=mid:4

The example below describes how content with two atlases can be signalled as separate streams. V3C parameter set is carried in a session level V3C media format parameter attribute and common atlas data are carried as part of the media level V3C media format parameter attribute corresponding to atlas zero. PT equal to 96, 97, 98 and 100 correspond to occupancy, geometry and attribute video component as well as atlas data component for atlas zero. PT equal to 101, 102, 103 and 104 correspond to respective components for atlas one.¶

  ...
  a=group:V3C 1 2 3 4 5 6 7 8
  a=v3cfmtp:sprop-v3c-parameter-set=AAUH/AAAP/zwAAABAADwIAWhBwAAOADjg
    QAADgAA8CAFoQcAADgA44EAAA6AkAgABRIA=;
  m=video 40000 RTP/AVP 96
  a=rtpmap:96 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=EAAAAA==
  a=mid:1
  m=video 40002 RTP/AVP 97
  a=rtpmap:97 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=GAAAAA==
  a=mid:2
  m=video 40004 RTP/AVP 98
  a=rtpmap:98 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=IAAAAA==
  a=mid:3
  m=application 40008 RTP/AVP 100
  a=rtpmap:100 v3c/90000
  a=fmtp:100
    sprop-v3c-common-atlas-data=YAEHgFA=,YgEAMAAAa+96Z5v6VP1D+P7LzRsb
    WDJ/yz+ALzMZNfvCg2389Kjd+d6fZyM6QZBfhrDW3K0vaP2Rr8L+gLAq/ny3wAzs9
    veiXEjjS67MfH+H4xV/RgW4fkl/YkINe/OsWCOBwPAVLACCf4FnogwYZKIME6oiD9
    UCodqjLwCCf4FnogxqBiIMZNwiEBpJIduBUoCCf4FnogwOeSIMCaGiEA9VIdtGwwC
    Cf4FnogvB+aILvWIiEBB6IdqobKfmZmZoCmZmefmZmZoCmZmefmZmZoCmZmefmZmZ
    oCmZmdA=
  a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;
  a=mid:4
  m=video 40010 RTP/AVP 101
  a=rtpmap:101 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=EAIAAA==
  a=mid:5
  m=video 40012 RTP/AVP 102
  a=rtpmap:102 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=GAIAAA==
  a=mid:6
  m=video 40014 RTP/AVP 103
  a=rtpmap:103 H264/90000
  a=v3cfmtp:sprop-v3c-unit-header=IAIAAA==
  a=mid:7
  m=application 40018 RTP/AVP 104
  a=rtpmap:104 v3c/90000
  a=v3cfmtp:sprop-v3c-unit-header=CAIAAA==
  a=mid:8

9.4. Offer and answer considerations

V3C coded content consist of metadata, i.e. atlas data and the video coded bitstreams represented as separate media lines in the SDP (unless packed video is used Section 9.2.2). During the session negotiation offerer lists different V3C components and informs which media lines SHOULD be consumed together. The receiver CAN select media components as suggested by the offer or select a subset of the components and formulate the answer accordingly. This freedom allows the receiver to consume a subset of the V3C coded media in scenarios, where it is fully or partially ignorant of the V3C coding scheme.¶

An example of offer which only sends V3C content. The following example contains video components as three different versions (H.264, H.265, H.266). Further differences between the alternatives would be signaled as part of the media attribute parameters, as is the practice with regular video streams.¶

  ...
  a=group:v3c 1 2 3 4
  a=v3cfmtp:v3c-ptl-level-idc=60;
    sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQAADkA==
  m=video 40000 RTP/AVP 96 97 98
  a=rtpmap:96 H264/90000
  a=rtpmap:97 H265/90000
  a=rtpmap:98 H266/90000
  a=v3cfmtp:sprop-v3c-unit-type=2;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0
  a=sendonly
  a=mid:1
  m=video 40002 RTP/AVP 99 100 101
  a=rtpmap:99 H264/90000
  a=rtpmap:100 H265/90000
  a=rtpmap:101 H266/90000
  a=v3cfmtp:sprop-v3c-unit-type=3;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0
  a=mid:2
  a=sendonly
  m=video 40004 RTP/AVP 102 103 104
  a=rtpmap:102 H264/90000
  a=rtpmap:103 H265/90000
  a=rtpmap:104 H266/90000
  a=v3cfmtp:sprop-v3c-unit-type=4;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0
  a=mid:3
  a=sendonly
  m=application 40006 RTP/AVP 105
  a=rtpmap:105 v3c/90000
  a=v3cfmtp:sprop-v3c-unit-type=1;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0;
  a=mid:4
  a=sendonly

An example of answer which only receives V3C data with the selected versions.¶

  ...
  a=group:v3c 1 2 3 4
  m=video 50000 RTP/AVP 96
  a=rtpmap:96 H264/90000
  a=recvonly
  a=mid:1
  m=video 50002 RTP/AVP 100
  a=rtpmap:100 H265/90000
  a=recvonly
  a=mid:2
  m=video 50004 RTP/AVP 104
  a=rtpmap:104 H266/90000
  a=recvonly
  a=mid:3
  m=application 50006 RTP/AVP 105
  a=rtpmap:105 v3c/90000
  a=recvonly
  a=mid:4

An example offer, which allows bundling different V3C components into one stream, based on [RFC9143].¶

  ...
  a=group:BUNDLE 1 2 3 4
  a=group:v3c 1 2 3 4
  m=video 40000 RTP/AVP 96
  a=rtpmap:96 H264/90000
  a=v3cfmtp:sprop-v3c-unit-type=2;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0
  a=mid:1
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
  m=video 40002 RTP/AVP 97
  a=rtpmap:97 H264/90000
  a=v3cfmtp:sprop-v3c-unit-type=3;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0;
  a=mid:2
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
  m=video 40004 RTP/AVP 98
  a=rtpmap:98 H264/90000
  a=v3cfmtp:sprop-v3c-unit-type=4;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0
  a=mid:3
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
  m=application 40006 RTP/AVP 99
  a=rtpmap:99 v3c/90000
  a=v3cfmtp:sprop-v3c-unit-type=1;sprop-v3c-vps-id=0;
    sprop-v3c-atlas-id=0;
    sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQAADkA==
  a=mid:4
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid

An example answer, which accepts bundling of different V3C components.¶

  a=group:BUNDLE 1 2 3 4
  a=group:v3c 1 2 3 4
  m=video 50000 RTP/AVP 96
  a=rtpmap:96 H264/90000
  a=mid:1
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
  m=video 0 RTP/AVP 97
  a=rtpmap:97 H264/90000
  a=bundle-only
  a=mid:2
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
  m=video 0 RTP/AVP 98
  a=rtpmap:98 H264/90000
  a=bundle-only
  a=mid:3
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
  m=application 0 RTP/AVP 99
  a=rtpmap:99 v3c/90000
  a=bundle-only
  a=mid:4
  a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid

9.5. Declarative SDP considerations

When V3C content over RTP is offered with SDP in a declarative style, the parameters capable of indicating both bitstream properties as well as receiver capabilities are used to indicate only bitstream properties. For example, in this case, the parameters v3c-ptl-level-idc, v3c-ptl-tier-flag, v3c-ptl-codec-idc, v3c-ptl-toolset-idc and v3c-ptl-rec-idc declare the values used by the bitstream, not the capabilities for receiving bitstreams.¶

A receiver of the SDP is required to support all parameters and values of the parameters provided; otherwise, the receiver MUST reject or not participate in the session. It falls on the creator of the session to use values that are expected to be supported by the receiving application.¶

10.1. V3C media type registration

A new media type will be registered with IANA; see Section Section 7.1.¶

10.2. V3C format parameters SDP attribute

This document defines a new session and media level SDP attribute: "v3cfmtp". This attribute will be registered by IANA under attribute-field names (<attribute-name>) registry in Session Description Protocol (SDP). The "v3cfmtp" attribute is used to convey V3C specific media format parameters for any media line. Its format is defined in Section Section 9.1. Further semantics are provided in Table 2.¶

NOTE: (informative) "this memo" to be replaced with the RFC number, once it becomes available.¶

10.3. V3C grouping type extension

Grouping is extended to establish relationships between substreams of a V3C representation. A new group type (V3C) for the group attribute will be registered as defined in Section 9.3. This document registers the semantics in Table 3 with IANA in the "Semantics for the 'group' SDP Attribute" subregistry (under the "Session Description Protocol (SDP) Parameters" registry):¶

NOTE: (informative) "this memo" to be replaced with the RFC number, once it becomes available.¶

12.1. Normative References

[ISO.IEC.23090-12]

ISO/IEC, "Information technology --- Coded representation of immersive media --- Part 12: MPEG Immersive video (MIV)", ISO/IEC 23090-12, 2022, <https://www.iso.org/standard/79113.html>.

[ISO.IEC.23090-5]

ISO/IEC, "Information technology --- Coded representation of immersive media --- Part 5: Visual volumetric video-based coding (V3C) and video-based point cloud compression (V-PCC)", ISO/IEC 23090-5, 2021, <https://www.iso.org/standard/73025.html>.

[RFC2119]

Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.

[RFC3550]

Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, July 2003, <https://www.rfc-editor.org/info/rfc3550>.

[RFC3551]

Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video Conferences with Minimal Control", STD 65, RFC 3551, DOI 10.17487/RFC3551, July 2003, <https://www.rfc-editor.org/info/rfc3551>.

[RFC3711]

Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, DOI 10.17487/RFC3711, March 2004, <https://www.rfc-editor.org/info/rfc3711>.

[RFC4585]

Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, DOI 10.17487/RFC4585, July 2006, <https://www.rfc-editor.org/info/rfc4585>.

[RFC4648]

Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, <https://www.rfc-editor.org/info/rfc4648>.

[RFC5124]

Ott, J. and E. Carrara, "Extended Secure RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/SAVPF)", RFC 5124, DOI 10.17487/RFC5124, February 2008, <https://www.rfc-editor.org/info/rfc5124>.

[RFC5888]

Camarillo, G. and H. Schulzrinne, "The Session Description Protocol (SDP) Grouping Framework", RFC 5888, DOI 10.17487/RFC5888, June 2010, <https://www.rfc-editor.org/info/rfc5888>.

[RFC6184]

Wang, Y.-K., Even, R., Kristensen, T., and R. Jesup, "RTP Payload Format for H.264 Video", RFC 6184, DOI 10.17487/RFC6184, May 2011, <https://www.rfc-editor.org/info/rfc6184>.

[RFC6190]

Wenger, S., Wang, Y.-K., Schierl, T., and A. Eleftheriadis, "RTP Payload Format for Scalable Video Coding", RFC 6190, DOI 10.17487/RFC6190, May 2011, <https://www.rfc-editor.org/info/rfc6190>.

[RFC7798]

Wang, Y.-K., Sanchez, Y., Schierl, T., Wenger, S., and M. M. Hannuksela, "RTP Payload Format for High Efficiency Video Coding (HEVC)", RFC 7798, DOI 10.17487/RFC7798, March 2016, <https://www.rfc-editor.org/info/rfc7798>.

[RFC8083]

Perkins, C. and V. Singh, "Multimedia Congestion Control: Circuit Breakers for Unicast RTP Sessions", RFC 8083, DOI 10.17487/RFC8083, March 2017, <https://www.rfc-editor.org/info/rfc8083>.

[RFC8866]

Begen, A., Kyzivat, P., Perkins, C., and M. Handley, "SDP: Session Description Protocol", RFC 8866, DOI 10.17487/RFC8866, January 2021, <https://www.rfc-editor.org/info/rfc8866>.

[RFC9143]

Holmberg, C., Alvestrand, H., and C. Jennings, "Negotiating Media Multiplexing Using the Session Description Protocol (SDP)", RFC 9143, DOI 10.17487/RFC9143, February 2022, <https://www.rfc-editor.org/info/rfc9143>.

12.2. Informative References

[ISO.IEC.14496-10]

ISO/IEC, "Information technology - Coding of audio-visual objects - Part 10: Advanced video coding", ISO/IEC 14496-10, 2020, <https://www.iso.org/standard/75400.html>.

[ISO.IEC.14496-12]

ISO/IEC, "Information technology --- Coding of audio-visual objects --- Part 12: ISO base media file format", ISO/IEC 14496-12, 2020, <https://www.iso.org/standard/74428.html>.

[ISO.IEC.23008-2]

ISO/IEC, "Information technology --- High efficiency coding and media delivery in heterogeneous environments --- Part 2: High efficiency video coding", ISO/IEC 23008-2, 2020, <https://www.iso.org/standard/75484.html>.

[ISO.IEC.23090-10]

ISO/IEC, "Information technology --- Coded representation of immersive media --- Part 10: Carriage of visual volumetric video-based coding data", ISO/IEC FDIS 23090-10, 2022, <https://www.iso.org/standard/78991.html>.

[ISO.IEC.23090-3]

ISO/IEC, "Information technology --- Coded representation of immersive media --- Part 3: Versatile video coding", ISO/IEC 23090-3, 2021, <https://www.iso.org/standard/73022.html>.

[RFC7201]

Westerlund, M. and C. Perkins, "Options for Securing RTP Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014, <https://www.rfc-editor.org/info/rfc7201>.

[RFC7202]

Perkins, C. and M. Westerlund, "Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Security Solution", RFC 7202, DOI 10.17487/RFC7202, April 2014, <https://www.rfc-editor.org/info/rfc7202>.