New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mpv drops frames at VO for specific AV1 WebM #6001

Closed
CounterPillow opened this Issue Jul 21, 2018 · 25 comments

Comments

Projects
None yet
9 participants
@CounterPillow
Contributor

CounterPillow commented Jul 21, 2018

mpv version and platform

mpv 0.28.0-638-g08a6827b3d Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
 built on Fri Jul 20 19:07:42 CEST 2018
ffmpeg library versions:
   libavutil       56.18.102
   libavcodec      58.21.105
   libavformat     58.17.101
   libswscale      5.2.100
   libavfilter     7.26.100
   libswresample   3.2.100
ffmpeg version: N-91498-g9888a19db4

Arch Linux x86_64

Reproduction steps

  1. Play back mpv --no-config https://fratti.ch/tmp/wanderers.webm
  2. Watch the VO frame drops
  3. Try a different VO, notice how it drops there too

Expected behavior

Don't drop frames, like vlc, Chromium and ffplay do.

Actual behavior

Severe VO framedrops

Log file

https://0x0.st/sVHf.log

Sample files

@mc4man

This comment has been minimized.

mc4man commented Jul 22, 2018

So why does the main website where your sample is show as being seized by the nsa?

@Hrxn

This comment has been minimized.

Hrxn commented Jul 22, 2018

With which Chromium versions is this supposed to work?

@CounterPillow

This comment has been minimized.

Contributor

CounterPillow commented Jul 22, 2018

@Hrxn ones which have a devendored ffmpeg that supports AV1, like the one Arch Linux ships.

@aufkrawall

This comment has been minimized.

aufkrawall commented Jul 23, 2018

I only get one frame drop in the very beginning (nothing unusual) and video plays fine otherwise for me on Windows and Linux.

@aufkrawall

This comment has been minimized.

aufkrawall commented Jul 26, 2018

Odd thing: I'm getting lots of delayed frames with lachs0r's build, but not with shinchiro's.

@lachs0r

This comment has been minimized.

Member

lachs0r commented Jul 31, 2018

@aufkrawall: Try the latest one.

@CounterPillow: Well. Consider the following:

  1. Both VLC and Chromium use vendored libaom in Arch
  2. ffplay might not implement frame dropping/synchronization (I don’t know)
  3. Arch’s libaom package fails to enable platform-specific optimization (I blame aom’s dumb build system but openSUSE’s multimedia maintainers pointed this out to me right after I had submitted a package with the same mistake)

I was able to reproduce the judder on my system with my own Windows builds until I changed the build options. For the record, see this spec file:

https://build.opensuse.org/package/view_file/multimedia:libs/libaom/libaom.spec?expand=1

@aufkrawall

This comment has been minimized.

aufkrawall commented Jul 31, 2018

Yep, fine now.

@Hrxn

This comment has been minimized.

Hrxn commented Aug 1, 2018

@lachs0r

Small nit, don't want to open an issue for this:

PS F:\Tools> mpv --V
mpv 0.29.0 Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
 built on UNKNOWN
ffmpeg library versions:
   libavutil       56.18.102
   libavcodec      58.22.100
   libavformat     58.17.101
   libswscale      5.2.100
   libavfilter     7.26.100
   libswresample   3.2.100
ffmpeg version: 4.1+809.g269daf5985

PS F:\Tools> mpv
mpv 0.29.0 Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
 built on UNKNOWN
ffmpeg library versions:
    [...]
    [...]

built on UNKNOWN

Not sure, but probably not intentional? Older builds definitely had their date in that place..

@CounterPillow

This comment has been minimized.

Contributor

CounterPillow commented Aug 1, 2018

@Hrxn that's to make the builds reproducible, AFAIU.

@Hrxn

This comment has been minimized.

Hrxn commented Aug 1, 2018

Good to know, thanks.

Then please disregard.. or maybe scrap the built on UNKNOWN line entirely?

@CounterPillow

This comment has been minimized.

Contributor

CounterPillow commented Aug 1, 2018

@lachs0r I've built aom with optimisations now and that didn't change a thing about mpv's framedrops.

@qyot27

This comment has been minimized.

Contributor

qyot27 commented Aug 1, 2018

Since this didn't go through last night when I tried to post it:

--disable-build-date is what results in 'built on UNKNOWN'. I think it was one of the Debian maintainers that wanted to be able to disable it for determinism reasons or something. Point releases may or may not default to using that option, hence that output. Git builds show the date.

Some years back there was a bug report that wanted it to be more comprehensive and if the --disable-build-date option was used, for the 'built on' line to be omitted completely. I didn't particularly care to open a pull request just for that, but I did whip up a patch to do it.

EDIT: I went ahead and opened a pull request.

@mc4man

This comment has been minimized.

mc4man commented Sep 16, 2018

Seems rather finicky about dropped frames. The linked vid was pretty short so did some local encodes of a bit longer. Seems minor changes to encoding options can create more or less dropped frames in mpv, don't have any other player to compare.
As ex. these 2 vids - the 30 sec one I get no dropped, the 45 sec. I get 11. If I encode diffently the dropped can increase for both.
Both done thru ffmpeg, multithreaded encoding, about 30% of orig. bitrate/file size, 2 pass. 2nd pass options for 30 sec, -

ffmpeg -hide_banner -nostdin -y -i "file:/home/doug/Videos/Special Edition Trailer-avatar-specialeditiontrailer.mov" -map 0:v:0 -map 0:a:0 -threads 8 -c:v libaom-av1 -cpu-used 4 -strict experimental -tile-columns 3 -tile-rows 3 -b:v 3814k -auto-alt-ref 1 -lag-in-frames 25 -g 128 -pix_fmt yuv420p -c:a libopus -b:a 128k -f webm -pass 2 -passlogfile /tmp/boram-198764pEX5Nbjj2ik -metadata title=Avatar "file:/home/doug/Desktop/Special Edition Trailer-avatar-specialeditiontrailer.webm" Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:/home/doug/Videos/Special Edition Trailer-avatar-specialeditiontrailer.mov'

https://0x0.st/sxjk.tar
(- both files in a .tar folder, if uploaded & then downloaded as .webm files then both showed increased frame drops when played, from the downloaded/extracted .tar they behave as the local ones, ..no idea why that

@Kagami

This comment has been minimized.

Contributor

Kagami commented Sep 19, 2018

I have the same issue with AV1 1080p of this video from AV1 Beta Launch Playlist.

$ mpv --version            
mpv 0.29.0-30-g6eb59fea2f-dirty Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
 built on Sun Sep 16 23:58:12 MSK 2018
ffmpeg library versions:
   libavutil       56.19.101
   libavcodec      58.30.100
   libavformat     58.18.101
   libswscale      5.2.100
   libavfilter     7.32.100
   libswresample   3.2.100
ffmpeg version: N-91961-g5109c38162
$ aomdec --help | grep av1
    av1    - AOMedia Project AV1 Decoder 1.0.0-590-g6fa400604
$ youtube-dl -f 399 -o dua.mp4 k2qgadSvNyU
$ time ffmpeg -v warning -i dua.mp4 -frames:v 250 -f null -
269% cpu 4.216 total

As you can see, it takes only 4.2 seconds to decode 10 seconds of video (25 fps) on my machine but when I'm trying to play it with mpv I get a lot of VO frame drops.

rboxeur added a commit to rboxeur/mpv-i686-cross-compiling-MinGW32-Doc that referenced this issue Oct 13, 2018

@aufkrawall

This comment has been minimized.

aufkrawall commented Oct 23, 2018

This issue is still present with latest ffmpeg + mpv builds (both Windows & Linux).
Open
mpv https://www.youtube.com/watch?v=2nXYbGmF3_Q
with
ytdl-format="((bestvideo[vcodec=av01.0.05M.08]/)+bestaudio)/best"
and notice stutter and frames reported as mistimed.

@veikk0

This comment has been minimized.

veikk0 commented Oct 27, 2018

I have a theory that AV1 decoding is limited to a single thread since FFmpeg doesn't support multithreaded encoding for AV1 either. So there's a precedent there.

To test it, download a 1080p AV1 video from Youtube:

youtube-dl -f 399 https://www.youtube.com/watch?v=mIAfxj7nd9k -o av1test.1080p.mp4

Then play it with mpv and record the CPU use of the process with time:

ThinkPad-T420:~/Videos/av1_yt$ /usr/bin/time -f "\ntime\t%E,\nCPU\t%P,\nRAM\t%Mk" mpv av1test.1080p.mp4 
Playing: av1test.1080p.mp4
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: Unknown AV1 Codec Configuration Box version 129
 (+) Video --vid=1 (*) (av1 1920x1080 23.976fps)
VO: [gpu] 1920x1080 yuv420p
V: 00:00:22 / 00:02:38 (14%) Dropped: 149 Cache: 136s+24MB
[input] No key binding found for key 'Ctrl+Alt+SPACE'.
V: 00:02:38 / 00:02:38 (99%) Dropped: 905 Cache:  0s


Exiting... (End of file)

time	3:20.06,
CPU	89%,
RAM	313636k

Playback was stuttery with a large number of dropped frames and took 3 minutes 20 seconds to complete for a 2 minute 38 second video. And as you can see, CPU usage was only 89% of the 400% maximum on a 2-core 4-thread CPU. While observing with htop I saw only 2 mpv processes, one with CPU usage below 100% and another that fluctuated from about 2% to 6.6%. Normally you'd see many more processes, when decoding 1080p VP9 for example I see at least 6 processes.

Now that the latest version of Firefox (v. 63) also supports AV1 decoding we can test that too. The media.av1.enabled flag needs to be set to true in about:config for it to work. From playing back the video in a new tab and observing the CPU usage from htop, it seems Firefox is using about 4 threads/processes for playback and their combined CPU usage is 200%+. Playback is smooth with no apparent frame drops, though there seems to be a bug where the rendering of the video playback can stop in a specific frame but according to htop's CPU usage the video is still being decoded in the background.

If you can't replicate the test result, then you probably aren't using a dual-core laptop CPU from 7 years ago (mine is an i3-2520M) and can just brute-force through the threading issue. In that case you can try one of Elecard's Holi festival AV1 test files. I'd imagine even the beefiest CPU will run into issues when trying to play the 2160p version on one thread. I can't even play the 720p 3.9 mbps file without dropped frames on mpv, while Firefox is again able to play it just fine.

@mc4man

This comment has been minimized.

mc4man commented Oct 27, 2018

At least here mpv & ffplay are ok with av1 files encoded with libaom0, particularly if done thru ffmpeg, higher rez's can have some dropped frames (I'm ok to 1080p. )
The elecard files 1080p or less play back ok.

However the youtube ones don't, neither in mpv or ffplay. They do in vlc but vlc does not use use ffmpeg libs for decoding av1. What's the difference between local, elecard & youtube I've no idea..

As far as mt encoding in ffmpeg, that is quite possible, ie. with -tile-columns & -tile-rows , ffmpeg needs to be patched for -tile-rows.
Additionally here for any locally encoded av1 or the elecard samples mpv & ffplay decoding is multithreaded. Again not so for the youtube samples..

Maybe by next year there will be a better decoder & mpv/ffmpeg will perform better. (dav1d

@richardpl

This comment has been minimized.

Contributor

richardpl commented Oct 27, 2018

Why this is still open, mpv does not have own decoders any more...

@Kagami

This comment has been minimized.

Contributor

Kagami commented Oct 27, 2018

Because ffmpeg decodes video faster than realtime but there are frame drops in mpv?

@CounterPillow

This comment has been minimized.

Contributor

CounterPillow commented Oct 28, 2018

@richardpl maybe you should actually try to reproduce this before making a stupid comment; other applications using the same decoder work fine, and mpv drops frames at vo.

@aufkrawall

This comment has been minimized.

aufkrawall commented Nov 6, 2018

Issue disappears when using libdav1d instead of libaom.

@richardpl

This comment has been minimized.

Contributor

richardpl commented Nov 6, 2018

Off course.... I told you so.

@CounterPillow

This comment has been minimized.

Contributor

CounterPillow commented Nov 19, 2018

Closing this, since it's probably a combination of the following:

  • libaom is slow
  • framedrops appear as VO instead of decoder because mpv in recent versions doesn't drop frames at the decoder anymore (my main source of confusion attributing this to mpv)
  • Chromium/ffplay probably don't properly implement frame dropping or something

With dav1d I still get stutters but they're less severe, so I guess it is just the decoder after all.

@aufkrawall

This comment has been minimized.

aufkrawall commented Nov 19, 2018

Do you get any anomalies reported? If I'm not mistaken, libdav1d is entirely free of stutter for me. Would have to test this more thoroughly though.

@CounterPillow

This comment has been minimized.

Contributor

CounterPillow commented Nov 19, 2018

I still get delayed frames, but that's likely because this is a 2 core 4 thread laptop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment