OBU length in Sample payload

[cconcolato]

In 1672222, the spec uses a flag in the sample entry to indicate different modes for storing OBUs in sample payloads: Annex B, Section 5, Section 5 optimized. This is because none of the solutions seem ideal:

• Using Annex B only does have an impact on very low bitrate (1 byte per frame).
• Using Section 5 optimized means some use cases (e.g. recording from RTP) will be more complex.

Having multiple options is not ideal either as packager/unpackager tools will have to implement all of them.

[tomfinegan]

Using Annex B only does have an impact on very low bitrate (1 byte per frame).

Annex-B-Prime increases the overhead beyond the above. There will be at least 2 additional length fields per Temporal Unit (TU length and Frame Length):

| TU Len | Frame Len | OBU(s), each with a length |

Muxers could discard the two length fields, but in that case demuxers would need to replace them. Requiring that demuxers replace them requires specialized knowledge of where to insert TU lengths and frame lengths when processing OBUs from a single TU that has omitted the fields.

[Nevcairiel]

From a muxer/demuxer perspective, I would prefer if Annex B is kept out of containers.
Additionally, does the container really need to know if mode 1 or 2 is being used? The presence of the payload length is signaled in the bitstream, there is no requirement to include that differentiation in the configuration record.

So I would suggest the following:

• Disallow Annex B
• Allow both mode 1/2
• Remove OBU_length_mode entirely, since mode 1/2 follow the same syntax.

[cconcolato]

The OBU_length_mode is the result of a compromise.

Some applications (e.g. RTP streaming) will apparently not code the length after the header (not necessarily using Annex B), either before (Annex B) or using some other mechanisms. For those applications, if we had only option 1 or 2, this would mean for every OBU, changing a bit in the OBU Header and inserting the length between the header/payload. This was not acceptable.

I'm unhappy about having to support 2 modes, but I'm not sure we can change it at this moment. However, we could indeed simplify the mode to a boolean: Annex B / not Annex B. We need to make sure that in the non-Annex-B case, setting obu_has_payload_length_field to 0 is only allowed on the last OBU of the sample.

[tdaede]

This makes it really hard for tools like FFmpeg - ideally these tools normalize into some internal format anyway (for example, OBUs with lengths and no annex B)... it hardly matters how this is done when depacketizing RTP. Having two values instead of 3 is far better, but far from good.