Skip to content

Video: inter-frame decode (decoded picture buffer and reference handling) #69

Description

@justin13888

rawshift-hwdec's sequence seam decodes random access points only. An access unit that references other pictures is refused with a clear error (vaapi/video.rs), so keyframe extraction, thumbnails, scrubbing and All-Intra camera modes work, but ordinary long-GOP recordings decode one frame per GOP.

Completing it is a purely internal change behind an unchanged public APIHwVideoDecoder already splits send-packet from receive-frame precisely so reordered output has somewhere to go — so it needs no second breaking release.

What it takes, per codec:

  • HEVC — POC derivation (needs nuh_temporal_id_plus1, which nal_type currently discards, for prevTid0Pic); the full st_ref_pic_set derivation including inter-RPS prediction (today's parse_st_ref_pic_set only counts entries); long-term entries; RefPicSetStCurrBefore/StCurrAfter/LtCurr; reference lists; RASL discard after a CRA that starts the stream or follows a reset. parse_sps currently reads and discards sps_max_num_reorder_pics and sps_max_latency_increase_plus1; both are needed for the bumping process.
  • H.264 — see the separate issue for its picture-parameter path first.
  • Shared — a decoded picture buffer with surface lifetimes, the Annex C bumping process, and populating ReferenceFrames / RefPicList in the VA buffers.

Notes for whoever picks this up:

  • Reorder must live in hwdec, not the caller: VAAPI is a slice-level API where the caller supplies the reference lists, so the buffer and the ordering cannot live above it.
  • The DPB is a pure function over synthetic inputs and should be table-tested (POC types, MMCO, sliding window, hierarchical-B pyramids) without any GPU.
  • The device test should assert bit-exactness against ffmpeg's software decode, not a variance heuristic — H.264/HEVC decode is normative, and only bit-exactness catches a wrong reference list or scaling matrix.
  • Consider splitting the pure-Rust codec logic out from under cfg(hwdec_backend = "vaapi"): the test CI job runs cargo test --workspace with no features, so none of the existing parser unit tests run there today.

Follow-up to #39.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions