Order of VPS, SPS, PPS of H265 in hvcC box is not correct

[l-law]

Issue summary

ffmpeg has error when playing output from shaka-packager:

[hevc @ 0x7fee2f00f000] VPS 0 does not exist
[hevc @ 0x7fee2f00f000] SPS 0 does not exist.

Investigation

From shaka-packager/packager/media/codecs/h265_byte_to_unit_stream_converter.cc:76-93, parameter sets are written to buffer in order of SPS, PPS then VPS.

  // SPS
  const uint8_t kArrayCompleteness = 0x80;
  buffer.AppendInt(static_cast<uint8_t>(kArrayCompleteness | Nalu::H265_SPS));
  buffer.AppendInt(static_cast<uint16_t>(1) /* numNalus */);
  buffer.AppendInt(static_cast<uint16_t>(last_sps_.size()));
  buffer.AppendVector(last_sps_);

  // PPS
  buffer.AppendInt(static_cast<uint8_t>(kArrayCompleteness | Nalu::H265_PPS));
  buffer.AppendInt(static_cast<uint16_t>(1) /* numNalus */);
  buffer.AppendInt(static_cast<uint16_t>(last_pps_.size()));
  buffer.AppendVector(last_pps_);

  // VPS
  buffer.AppendInt(static_cast<uint8_t>(kArrayCompleteness | Nalu::H265_VPS));
  buffer.AppendInt(static_cast<uint16_t>(1) /* numNalus */);
  buffer.AppendInt(static_cast<uint16_t>(last_vps_.size()));
  buffer.AppendVector(last_vps_);

However, SPS has a dependency of VPS in H265, as SPS is referring to active VPS by sps_video_parameter_set_id in 7.3.2.2.1 General sequence parameter set RBSP syntax of ITU-T H.265.

Suggested fix

Error from ffmpeg is fixed by changing above code snippet into following:

  // VPS
  const uint8_t kArrayCompleteness = 0x80;
  buffer.AppendInt(static_cast<uint8_t>(kArrayCompleteness | Nalu::H265_VPS));
  buffer.AppendInt(static_cast<uint16_t>(1) /* numNalus */);
  buffer.AppendInt(static_cast<uint16_t>(last_vps_.size()));
  buffer.AppendVector(last_vps_);

  // SPS
  buffer.AppendInt(static_cast<uint8_t>(kArrayCompleteness | Nalu::H265_SPS));
  buffer.AppendInt(static_cast<uint16_t>(1) /* numNalus */);
  buffer.AppendInt(static_cast<uint16_t>(last_sps_.size()));
  buffer.AppendVector(last_sps_);

  // PPS
  buffer.AppendInt(static_cast<uint8_t>(kArrayCompleteness | Nalu::H265_PPS));
  buffer.AppendInt(static_cast<uint16_t>(1) /* numNalus */);
  buffer.AppendInt(static_cast<uint16_t>(last_pps_.size()));
  buffer.AppendVector(last_pps_);

[kqyang]

Good finding!

It is actually also documented in ISO/IEC 14496-15:2017 8.3.3.1 HEVC decoder configuration record:

It is recommended that the arrays be in the order VPS, SPS, PPS, prefix SEI, suffix SEI.

Thanks for reporting the issue and the suggested fix! Do you mind to submit a pull request for the fix?