extradata format is unclear

[rbouqueau]

should be either raw or annex B
the issue with raw is that we need to separate sps and pps - some APIs don't support that
maybe we need a clearer metadata model with documentation

[rbouqueau]

ok: when demux only, ffmpeg returns AVCC (whatever the input is a udp TS) but it returns Annex B when doing this encoding (with x264?)

Current opinion: when returning the metadata, we need to make the translation into AVCC.

[rbouqueau]

When no transcoding, libavformat returns AVCC. After encoding, avcodec returns Annex B.