Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webm sequence header doesn't match AV1 bitstream, causing problems on playback #5081

Open
jmitz5 opened this issue Apr 17, 2023 · 8 comments

Comments

@jmitz5
Copy link

jmitz5 commented Apr 17, 2023

Problem Description

I'm gonna be honest, this is a bit over my head. I'm passing this along from a VLC dev who is working through a bug I found over there. There is a problem with VLC's decoding of AV1 with the "super-resolution" anamorphic feature. The dev says that at least part of the problem has to do with the output of my handbrake encode:

"I can confirm the sequence header in the WebM part doesn't match the one in the bitstream:

header:
0A 0E 00 00 00 42 A7 BF E3 E0 17 C2 80 80 80 96

bitstream:
0A 0E 00 00 00 42 A7 BF E3 E7 DF DE 80 80 80 96

So technically the file is invalid. It's unrelated to our issue, but it makes fixing harder. You should report the bug to Handbrake so they fix the issue.

One important reason to fix this is that if we want to use hardware decoding, we need to have the proper sequence header before starting decoding, so we can tell if the hardware can decode it or not."

Here is the link to the Videolan thread for more context: VLC cannot properly display this SVT-AV1 webm file with "Super resolution" feature

Basically, there is some issue with VLC's dav1d causing the "super-resolution" scaling to be glitchy, and it is exacerbated by this header issue from the handbrake. The file in question can be found there as well. The logs I provided are correct, but I renamed the file after encoding, so it doesn't appear to match.

Activity Log, Crash Log or any other details

HandBrake Activity Log for Session: 2023-04-14T20:35:34Z
Handbrake Version: 1.6.1 (2023012200)
MARVEL'S CAPTAIN AMERICA- THE WINTER SOLDIER - BLU-RAY-1080-18-5-320-15mlim-grain8-handbrake.webm
Preset: auto-default (Modified)
[15:35:34] Compile-time hardening features are enabled
[15:35:35] hb_init: starting libhb thread
[15:35:35] thread 70000aebe000 started ("libhb")
[15:35:35] macgui: fr.handbrake.HandBrakeXPCService scanning specifically for title: 1
[15:35:35] CPU: Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
[15:35:35]  - Intel microarchitecture Broadwell
[15:35:35]  - logical processor count: 4
[15:35:35] hb_scan: path=/Users/Jacob/teamviewer_transfer/MARVEL'S CAPTAIN AMERICA- THE WINTER SOLDIER - BLU-RAY-1080-18-5-320-15mlim-grain8.webm, title_index=1
[15:35:35] thread 70000af41000 started ("scan")
udfread ERROR: ECMA 167 Volume Recognition failed
disc.c:333: failed opening UDF image /Users/Jacob/teamviewer_transfer/MARVEL'S CAPTAIN AMERICA- THE WINTER SOLDIER - BLU-RAY-1080-18-5-320-15mlim-grain8.webm
disc.c:437: error opening file BDMV/index.bdmv
disc.c:437: error opening file BDMV/BACKUP/index.bdmv
[15:35:35] bd: not a bd - trying as a stream/file instead
libdvdread: DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav: vm: vm: failed to read VIDEO_TS.IFO
[15:35:35] dvd: not a dvd - trying as a stream/file instead
Input #0, matroska,webm, from '/Users/Jacob/teamviewer_transfer/MARVEL'S CAPTAIN AMERICA- THE WINTER SOLDIER - BLU-RAY-1080-18-5-320-15mlim-grain8.webm':
  Metadata:
    creation_time   : 2023-02-23T00:42:02.000000Z
    ENCODER         : Lavf59.27.100
  Duration: 02:15:54.22, start: -0.007000, bitrate: 2451 kb/s
  Chapters:
    Chapter #0:0: start 0.000000, end 213.714000
      Metadata:
        title           : Prologue
    Chapter #0:1: start 213.714000, end 814.147000
      Metadata:
        title           : Hostages on the Lemurian Star
    Chapter #0:2: start 814.147000, end 1078.744000
      Metadata:
        title           : Don't Hold Your Breath
    Chapter #0:3: start 1078.744000, end 1353.519000
      Metadata:
        title           : Steve Visits the Smithsonian
    Chapter #0:4: start 1353.519000, end 2004.836000
      Metadata:
        title           : Project Insight
    Chapter #0:5: start 2004.836000, end 2493.324000
      Metadata:
        title           : In Pursuit
    Chapter #0:6: start 2493.324000, end 3103.934000
      Metadata:
        title           : Steve Meets Alexander Pierce
    Chapter #0:7: start 3103.934000, end 3511.008000
      Metadata:
        title           : Fugitives from S.H.I.E.L.D.
    Chapter #0:8: start 3511.008000, end 4060.557000
      Metadata:
        title           : Uncovering the Past
    Chapter #0:9: start 4060.557000, end 4589.752000
      Metadata:
        title           : Call It a Resume
    Chapter #0:10: start 4589.752000, end 5088.417000
      Metadata:
        title           : Facing Off with the Winter Soldier
    Chapter #0:11: start 5088.417000, end 5546.374000
      Metadata:
        title           : Captain's Orders
    Chapter #0:12: start 5546.374000, end 5831.326000
      Metadata:
        title           : Paying a Price
    Chapter #0:13: start 5831.326000, end 6553.881000
      Metadata:
        title           : Engaging
    Chapter #0:14: start 6553.881000, end 6910.070000
      Metadata:
        title           : Battle over the Potomac
    Chapter #0:15: start 6910.070000, end 7276.436000
      Metadata:
        title           : The End of the Line
    Chapter #0:16: start 7276.436000, end 7591.918000
      Metadata:
        title           : Life After S.H.I.E.L.D.
    Chapter #0:17: start 7591.918000, end 8154.188000
      Metadata:
        title           : End Credits
  Stream #0:0: Video: av1 (Main), yuv420p10le(tv, bt709), 1920x800, Film Grain, SAR 1:1 DAR 12:5, 23.98 fps, 23.98 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 02:15:54.195000000
  Stream #0:1(eng): Audio: opus, 48000 Hz, 5.1, fltp (default)
    Metadata:
      title           : Surround 5.1
      DURATION        : 02:15:54.201000000
  Stream #0:2(eng): Audio: opus, 48000 Hz, stereo, fltp
    Metadata:
      title           : Commentary
      DURATION        : 02:15:54.221000000
  Stream #0:3(eng): Audio: opus, 48000 Hz, stereo, fltp
    Metadata:
      title           : For Visually Impaired
      DURATION        : 02:15:54.221000000
[15:35:35] Found ffmpeg 18 chapters, container=matroska,webm
[15:35:35] Added chapter 1, name='Prologue', dur=19234260, (00:03:34)
[15:35:35] Added chapter 2, name='Hostages on the Lemurian Star', dur=54038970, (00:10:00)
[15:35:35] Added chapter 3, name='Don't Hold Your Breath', dur=23813730, (00:04:25)
[15:35:35] Added chapter 4, name='Steve Visits the Smithsonian', dur=24729750, (00:04:35)
[15:35:35] Added chapter 5, name='Project Insight', dur=58618530, (00:10:51)
[15:35:35] Added chapter 6, name='In Pursuit', dur=43963920, (00:08:08)
[15:35:35] Added chapter 7, name='Steve Meets Alexander Pierce', dur=54954900, (00:10:11)
[15:35:35] Added chapter 8, name='Fugitives from S.H.I.E.L.D.', dur=36636660, (00:06:47)
[15:35:35] Added chapter 9, name='Uncovering the Past', dur=49459410, (00:09:10)
[15:35:35] Added chapter 10, name='Call It a Resume', dur=47627550, (00:08:49)
[15:35:35] Added chapter 11, name='Facing Off with the Winter Soldier', dur=44879850, (00:08:19)
[15:35:35] Added chapter 12, name='Captain's Orders', dur=41216130, (00:07:38)
[15:35:35] Added chapter 13, name='Paying a Price', dur=25645680, (00:04:45)
[15:35:35] Added chapter 14, name='Engaging', dur=65029950, (00:12:03)
[15:35:35] Added chapter 15, name='Battle over the Potomac', dur=32057010, (00:05:56)
[15:35:35] Added chapter 16, name='The End of the Line', dur=32972940, (00:06:06)
[15:35:35] Added chapter 17, name='Life After S.H.I.E.L.D.', dur=28393380, (00:05:15)
[15:35:35] Added chapter 18, name='End Credits', dur=50607270, (00:09:22)
[15:35:35] scan: decoding previews for title 1
[15:35:35] scan: preview 1
[15:35:35] fifo_close: trashing 0 buffer(s)
[15:35:35] scan: audio 0x1: opus, rate=48000Hz, bitrate=1 English (Opus) (5.1 ch)
[15:35:35] fifo_close: trashing 0 buffer(s)
[15:35:35] scan: audio 0x2: opus, rate=48000Hz, bitrate=1 English (Opus) (2.0 ch)
[15:35:35] fifo_close: trashing 0 buffer(s)
[15:35:35] scan: audio 0x3: opus, rate=48000Hz, bitrate=1 English (Opus) (2.0 ch)
[15:35:35] scan: preview 2
[15:35:35] scan: preview 3
[15:35:35] scan: preview 4
[15:35:35] scan: preview 5
[15:35:35] scan: preview 6
[15:35:35] scan: preview 7
[15:35:35] scan: preview 8
[15:35:35] scan: preview 9
[15:35:35] scan: preview 10
[15:35:36] crop: [0] 0/0/0/0
[15:35:36] crop: [1] 0/0/0/0
[15:35:36] crop: [2] 0/0/0/0
[15:35:36] crop: [3] 0/0/0/0
[15:35:36] crop: [4] 0/0/0/0
[15:35:36] crop: [5] 0/0/0/0
[15:35:36] crop: [6] 0/0/0/0
[15:35:36] crop: [7] 0/0/0/0
[15:35:36] crop: [8] 13/0/0/0
[15:35:36] crop: less_than_median_frame_count: 0,
[15:35:36] scan: 10 previews, 1920x800, 23.976 fps, autocrop = 0/0/0/0, aspect 2.40:1, PAR 1:1, color profile: 1-1-1, chroma location: left
[15:35:36] Freed 1 buffers of size 1024
[15:35:36] Freed 1 buffers of size 2048
[15:35:36] Freed 1 buffers of size 8192
[15:35:36] Freed 1 buffers of size 32768
[15:35:36] Freed 1 buffers of size 65536
[15:35:36] Freed 1 buffers of size 131072
[15:35:36] Freed 1 buffers of size 262144
[15:35:36] Freed 1 buffers of size 524288
[15:35:36] Freed 2 buffers of size 4194304
[15:35:36] Allocated 9415680 bytes of buffers on this pass and Freed 9415680 bytes, 0 bytes leaked
[15:35:36] thread 70000af41000 exited ("scan")
[15:35:36] thread 70000af41000 joined ("scan")
[15:35:36] libhb: scan thread found 1 valid title(s)
[15:35:36] macgui: fr.handbrake.HandBrakeXPCService scan done
[15:35:36] thread 70000af41000 started ("work")
[15:35:36] macgui: fr.handbrake.HandBrakeXPCService started encoding MARVEL'S CAPTAIN AMERICA- THE WINTER SOLDIER - BLU-RAY-1080-18-5-320-15mlim-grain8-handbrake.webm
[15:35:36] macgui: fr.handbrake.HandBrakeXPCService with preset auto-default (Modified)
[15:35:36] Starting work at: Fri Apr 14 15:35:36 2023

[15:35:36] 1 job(s) to process
[15:35:36] Starting Task: Encoding Pass
[15:35:36] Skipping vfr filter
[15:35:36] Skipping crop/scale filter
[15:35:36] job configuration:
[15:35:36]  * source
[15:35:36]    + /Users/Jacob/teamviewer_transfer/MARVEL'S CAPTAIN AMERICA- THE WINTER SOLDIER - BLU-RAY-1080-18-5-320-15mlim-grain8.webm
[15:35:36]    + title 1, start 01:02:19.00 stop 01:02:39.00
[15:35:36]    + container: matroska,webm
[15:35:36]    + data rate: 2451 kbps
[15:35:36]  * destination
[15:35:36]    + /Users/Jacob/Downloads/MARVEL'S CAPTAIN AMERICA- THE WINTER SOLDIER - BLU-RAY-1080-18-5-320-15mlim-grain8-handbrake.webm
[15:35:36]    + container: WebM (libavformat)
[15:35:36]      + chapter markers
[15:35:36]  * video track
[15:35:36]    + decoder: libdav1d 10-bit (yuv420p10le)
[15:35:36]    + Output geometry
[15:35:36]      + storage dimensions: 1920 x 800
[15:35:36]      + pixel aspect ratio: 1 : 1
[15:35:36]      + display dimensions: 1920 x 800
[15:35:36]    + encoder: AV1 10-bit (SVT)
[15:35:36]      + preset:  4
[15:35:36]      + options: superres-mode=1:superres-denom=13:superres-kf-denom=13:mbr=9m:mbr-overshoot-pct=0:film-grain=10
[15:35:36]      + profile: auto
[15:35:36]      + level:   auto
[15:35:36]      + quality: 24.00 (RF)
[15:35:36]      + color profile: 1-1-1
[15:35:36]      + chroma location: left
[15:35:36]  * audio track 1
[15:35:36]    + name: Surround 5.1
[15:35:36]    + decoder: English (Opus) (5.1 ch) (track 1, id 0x1)
[15:35:36]      + samplerate: 48000 Hz
[15:35:36]    + Opus Passthru
[15:35:36] sync: expecting 503 video frames
[15:35:36] thread 70000b2cd000 started ("Audio Synchronization")
[15:35:36] encavcodecInit: AV1 encoder
[15:35:36] encavcodec: encoding at CRF 24.00
[15:35:36] encavcodec: encoding with stored aspect 1/1
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:	SVT-AV1 Encoder Lib v1.4.1
Svt[info]: SVT [build]  :	Apple LLVM 14.0.0 (clang-1400.0.29.202)	 64 bit
Svt[info]: LIB Build date: Jan 22 2023 22:41:52
Svt[info]: -------------------------------------------
Svt[warn]: TPL will be disabled when super resolution is enabled!
Svt[info]: Number of logical cores available: 4
Svt[info]: Number of PPCS 24
Svt[info]: [asm level on system : up to avx2]
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile	tier (auto)	level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator 		: 1920 / 800 / 24000 / 1001
Svt[info]: SVT [config]: bit-depth / color format 					: 10 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct 					: 4 / VQ / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type 			: 244 / 16 / key frame
Svt[info]: SVT [config]: BRC mode / CQP Assignment / max bitrate (kbps)			: CQP / 24 / 9000
Svt[info]: SVT [config]: film grain synth / denoising / level 				: 1 / 1 / 10
Svt[info]: -------------------------------------------
libdvdread: Encrypted DVD support unavailable.
libdvdread: Can't open file VIDEO_TS.IFO.
[15:35:36] Writing Metadata to output file...
[15:35:36] thread 700015736000 started ("Muxer")
[15:35:36] thread 7000157b9000 started ("Reader")
[15:35:36] thread 70001583c000 started ("Audio decoder (libavcodec)")
[15:35:36] thread 7000158bf000 started ("Video decoder (libavcodec)")
[15:35:36] thread 700015942000 started ("Video Synchronization")
[15:35:36] thread 7000159c5000 started ("FFMPEG encoder (libavcodec)")
[15:35:36] thread 700015a48000 started ("Muxer")
[15:35:36] ffmpeg_read starting chapter 9 at 336142080
[15:35:37] sync: first pts audio 0x1 is 0
[15:35:37] sync: first pts video is 3538
[15:41:51] sync: reached audio 0x1 pts 1800000, exiting early
[15:41:52] sync: reached video pts 1801584, exiting early
[15:42:43] thread 700015a48000 exited ("Muxer")
[15:42:43] thread 700015a48000 joined ("Muxer")
[15:42:43] work: average encoding speed for job is 1.276266 fps
[15:42:43] thread 7000158bf000 exited ("Video decoder (libavcodec)")
[15:42:43] thread 700015736000 exited ("Muxer")
[15:42:43] thread 700015942000 exited ("Video Synchronization")
[15:42:43] thread 7000159c5000 exited ("FFMPEG encoder (libavcodec)")
[15:42:43] thread 70001583c000 exited ("Audio decoder (libavcodec)")
[15:42:43] thread 70000b2cd000 exited ("Audio Synchronization")
[15:42:43] thread 7000157b9000 exited ("Reader")
[15:42:43] thread 7000157b9000 joined ("Reader")
[15:42:43] thread 70001583c000 joined ("Audio decoder (libavcodec)")
[15:42:43] thread 7000158bf000 joined ("Video decoder (libavcodec)")
[15:42:43] thread 700015942000 joined ("Video Synchronization")
[15:42:43] thread 7000159c5000 joined ("FFMPEG encoder (libavcodec)")
[15:42:43] opus-decoder done: 6112 frames, 0 decoder errors
[15:42:43] libdav1d-decoder done: 2913 frames, 0 decoder errors
[15:42:43] sync: got 479 frames, 503 expected
[15:42:43] sync: framerate min 23.976 fps, max 23.976 fps, avg 23.976 fps
[15:42:43] thread 70000b2cd000 joined ("Audio Synchronization")
[15:42:43] encavcodec: closing libavcodec
[15:42:43] mux: file size, 1704124 bytes
[15:42:43] mux: track 0, 479 frames, 907169 bytes, 361.75 kbps, fifo 512
[15:42:43] mux: track 1, 1000 frames, 787016 bytes, 313.84 kbps, fifo 1024
[15:42:43] mux: overhead, 6.72 bytes per frame
[15:42:43] thread 700015736000 joined ("Muxer")
[15:42:43] fifo_close: trashing 16 buffer(s)
[15:42:43] fifo_close: trashing 0 buffer(s)
[15:42:43] fifo_close: trashing 0 buffer(s)
[15:42:43] fifo_close: trashing 0 buffer(s)
[15:42:43] fifo_close: trashing 0 buffer(s)
[15:42:43] fifo_close: trashing 0 buffer(s)
[15:42:43] fifo_close: trashing 0 buffer(s)
[15:42:43] fifo_close: trashing 0 buffer(s)
[15:42:43] Freed 320 buffers of size 1024
[15:42:43] Freed 32 buffers of size 2048
[15:42:43] Freed 26 buffers of size 4096
[15:42:43] Freed 21 buffers of size 8192
[15:42:43] Freed 9 buffers of size 16384
[15:42:43] Freed 14 buffers of size 32768
[15:42:43] Freed 11 buffers of size 65536
[15:42:43] Freed 2 buffers of size 131072
[15:42:43] Freed 5 buffers of size 262144
[15:42:43] Freed 2 buffers of size 524288
[15:42:43] Freed 2 buffers of size 1048576
[15:42:43] Freed 1 buffers of size 2097152
[15:42:43] Freed 32 buffers of size 8388608
[15:42:43] Allocated 277250048 bytes of buffers on this pass and Freed 277250048 bytes, 0 bytes leaked
[15:42:43] Finished work at: Fri Apr 14 15:42:43 2023

[15:42:43] thread 70000af41000 exited ("work")
[15:42:43] thread 70000af41000 joined ("work")
[15:42:43] libhb: work result = 0

What Operating System are you running?

MacOS 10.13.6

What version of HandBrake are you running?

1.6.1

Where did you download HandBrake from?

handbrake.fr

@jmitz5 jmitz5 changed the title Webm sequence header doesn't match AV1 bitstream header, causing problems on playback Webm sequence header doesn't match AV1 bitstream, causing problems on playback Apr 17, 2023
@galad87
Copy link
Contributor

galad87 commented Apr 17, 2023

Can you try an HandBrake snapshot and see it if helps? https://github.com/HandBrake/HandBrake-snapshots

@robUx4
Copy link

robUx4 commented Apr 17, 2023

BTW it could also be a problem in FFmpeg. I don't know if the code responsible to write the private header in WebM/MP4 is within Handbrake or FFmpeg (libavcodec+libavformat) which is used here to create the file.

It may also be an issue of SVT-AV1 not reporting the proper AV1 Sequence Header on startup.

@galad87
Copy link
Contributor

galad87 commented Apr 17, 2023

It could be, but the SVT-AV1 integration has been reworked in the snapshots, so it's worth a try at least.
Now it just calls svt_av1_enc_stream_header

svt_ret = svt_av1_enc_stream_header(pv->svt_handle, &headerPtr);
to get the header, so if it's still wrong it's probably a SVT-AV1 issue.

@jmitz5
Copy link
Author

jmitz5 commented Apr 17, 2023

Can you try an HandBrake snapshot and see it if helps? https://github.com/HandBrake/HandBrake-snapshots

I tried it again with the snapshot (took some time, av1 is mega slow). Same results in vlc. I don't know how to check if the sequence header mismatch situation has improved. Maybe I'll send the files over through the Videolan thread.

@ghost
Copy link

ghost commented Apr 18, 2023

Please do.

@ghost
Copy link

ghost commented Apr 19, 2023

It just looks like SVT-AV1's svt_av1_enc_stream_header does give us a different sequence header than what he encoder includes in the bitstream; this probably needs to be reported to and discussed with their developers. Adding it to my list.

@ghost ghost self-assigned this Apr 19, 2023
mohit-marathe pushed a commit to mohit-marathe/vlc that referenced this issue Jun 23, 2023
If the AV1 extra data in the container don't match the bitstream (invalid file)
we end up missing some pictures when using extra resolution.

This can be triggered by this bug in Handbrake with SVT-AV1:
HandBrake/HandBrake#5081

Ref. #28002
@sr55
Copy link
Contributor

sr55 commented Sep 18, 2023

Does the latest SVT-AV1 still exhibit this?

@galad87
Copy link
Contributor

galad87 commented Sep 18, 2023

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

4 participants