-ao pulse causes visual jerkiness for 20 seconds after unpause

[Infinoid]

Important Information

• mpv version = 0.33.0 and newer (tested up to 0.35.1)
• Linux Distribution and Version = debian bookworm
• Source of the mpv binary = debian bookworm, and self-built from git (see below)
• If known which version of mpv introduced the problem = 0.33.0
• Window Manager and version = xfce
• GPU driver and version = nvidia 525.125.06-1

||/ Name           Version              Architecture Description
+++-==============-====================-============-=========================================================
ii  mpv            0.35.1-4             amd64        video player based on MPlayer/mplayer2
ii  nvidia-driver  525.125.06-1~deb12u1 amd64        NVIDIA metapackage
ii  pulseaudio     16.1+dfsg1-2+b1      amd64        PulseAudio sound server
ii  xfce4          4.18                 all          Meta-package for the Xfce Lightweight Desktop Environment

Reproduction steps

simplest config possible:

• mpv --no-config -ao pulse /path/to/any/anime.mkv
• seek to the begining of a panning scene
• hit spacebar to pause
• wait 10 seconds (at least 7 seconds in my testing)
• hit spacebar to resume
• watch the visuals fail to pan smoothly, instead be very jerky for the next ~20 seconds

more scientific reproducer (with numbers!)

• mpv --no-config -ao pulse --video-sync=display-resample /path/to/any/anime.mkv
• hit "I" to enable on-screen statistics
• hit spacebar to pause
• wait 10 seconds
• hit spacebar to resume
• watch the "Mistimed:" counter increase for ~20 seconds (note: this line does not appear in the stats overlay without --video-sync=display-resample)

When I type in the Mistimed numbers I see, every time the stats overlay refreshes, I get a sequence like this:

1 6 15 28 41 51 67 75 87 103 109 114 116 119 122 122 124 125 125 125

After that it settles down. It seems the worst delays are about 5 seconds after the unpause... for whatever that's worth.

Expected behavior

Playback is smooth after unpause.

Actual behavior

Playback is visually jerky for about 20 seconds after unpause. "Mistimed:" counter increases by about 10 every time the stats screen updates. After about 20 seconds, it settles down and playback seems fine.

Log file

mpv-jerky-video.txt

More info

It seems like the pause needs to last at least 7 seconds in order to trigger the issue. A 10 second pause is a completely reliable reproducer. It happens 100% of the time I try it, on 100% of the video files I've tried.

I started noticing this issue at around version 0.34.0. The previous version of Debian (bullseye) has an older package of mpv_0.32.0-3_amd64.deb which I can install on this bookworm system, leaving all of the dependencies alone, and it works fine. (Installs fine, it plays back fine, pauses and resumes fine.)

I was able to git bisect this issue, working around a few build issues along the way, and found initially that it fails in 0.33.0 and works in 0.32.0. I further narrowed it down to d27ad96. That commit expresses some concern about the stability of -ao pulse; I guess this is the evidence.

Seeing that the issue was audio related, I did some more testing and found:

• -ao pulse: visual stuttering after pause
• -ao pulse --pulse-latency-hacks=yes: seems fine
• -ao alsa: seems fine
• -ao sdl: seems fine

The commit message recommends alsa, and alsa does work, but mpv seems to use pulse by default.

This bug report might be a more specific version of #10954 or it might be unrelated; hard to be sure.

[sfan5]

0.35.x and older are unsupported, please re-test with 0.36.0.

[Infinoid]

0.35.x and older are unsupported, please re-test with 0.36.0.

Done. The problem also exists in 0.36.0.

[sfan5]

I did not expect to be able to reproduce on an AMD system but I can.
It doesn't seem to matter where you pause, as long as you wait a while and unpause display resample will get lots of mistimed frames.

[Infinoid]

Without display-resample, I still see lots of jumpiness in the video. It seems like about the same amount, to me, though it's not as easy to measure.

[sfan5]

Doing the following

diff --git a/audio/out/ao_pulse.c b/audio/out/ao_pulse.c
index ae19c875c0..d7644e0abc 100644
--- a/audio/out/ao_pulse.c
+++ b/audio/out/ao_pulse.c
@@ -633,6 +633,10 @@ static void audio_get_state(struct ao *ao, struct mp_pcm_state *state)
 
     state->playing = priv->playing;
 
+    MP_ERR(ao, "free=%d queued=%d delay=%.4f play=%d\n", state->free_samples,
+        state->queued_samples, state->delay,
+        !!state->playing);
+
     pa_threaded_mainloop_unlock(priv->mainloop);
 
     // Otherwise, PA will keep hammering us for underruns (which it does instead

it is apparent that PA returns too high latency values twice after unpausing:

AV: 00:00:04 / 00:24:10 (0%) A-V: -0.003 DS: 3.620/5 Dropped: 1
[ao/pulse] free=0 queued=4410 delay=0.1349 play=1
[ao/pulse] free=0 queued=4410 delay=0.1206 play=1
[ao/pulse] free=1765 queued=2645 delay=0.1205 play=1
AV: 00:00:04 / 00:24:10 (0%) A-V: -0.003 DS: 3.590/5 Dropped: 1
[ao/pulse] free=0 queued=4410 delay=0.1598 play=1
[ao/pulse] free=0 queued=4410 delay=0.1347 play=1
[ao/pulse] free=1766 queued=2644 delay=0.1204 play=1
[ao/pulse] free=0 queued=4410 delay=0.1599 play=1
[ao/pulse] free=0 queued=4410 delay=0.1515 play=1
(Paused) AV: 00:00:04 / 00:24:10 (0%) A-V: -0.003 DS: 3.590/5 Dropped: 1
[ao/pulse] free=0 queued=4410 delay=4.6162 play=1
[ao/pulse] free=0 queued=4410 delay=4.6162 play=1

Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).

AV: 00:00:04 / 00:24:10 (0%) A-V: -4.474 DS: 3.550/5 Dropped: 1
[ao/pulse] free=0 queued=4410 delay=0.1453 play=1
AV: 00:00:04 / 00:24:10 (0%) A-V:  0.473 DS: 3.820/5 Dropped: 1
[ao/pulse] free=0 queued=4410 delay=0.1260 play=1
[ao/pulse] free=1774 queued=2636 delay=0.1202 play=1
[ao/pulse] free=0 queued=4410 delay=0.1601 play=1
AV: 00:00:04 / 00:24:10 (0%) A-V:  0.473 DS: 3.820/5 Dropped: 1
[ao/pulse] free=0 queued=4410 delay=0.1350 play=1
[ao/pulse] free=1766 queued=2644 delay=0.1204 play=1
[ao/pulse] free=0 queued=4410 delay=0.1599 play=1
[ao/pulse] free=0 queued=4410 delay=0.1347 play=1
AV: 00:00:04 / 00:24:10 (0%) A-V:  0.473 DS: 3.820/5 Dropped: 1
[...]

This causes A-V desync that lasts for a bit, matching what you described.

As such I think this is a duplicate of #10333. This issue also goes away reliably when you use --audio-stream-silence.

The only notable fact is that you say mpv 0.32.0 can do it correctly on your system. This hints at this being not PA bug but a problem in how mpv uses PA.

[Infinoid]

The only notable fact is that you say mpv 0.32.0 can do it correctly on your system. This hints at this being not PA bug but a problem in how mpv uses PA.

Yes. On my machine, before and after d27ad96 is like night and day. If bugs only show themselves during the day.

[raphael-costa]

I'd like to add the feedback that this is a very high priority issue, the stuttering after unpausing makes mpv the worst video player.

If reverting d27ad96 is not realistic due to the large amount of work done, then I suggest temporarily switching the default ao to alsa, that's what I did on my config to make mpv the best video player again.

Temporarily printing a warning in console when using pulseaudio ao is another good idea, I suggest doing both.

[sfan5]

No need for all that when you can just use --pulse-latency-hacks=yes.

[Dudemanguy]

Is it worth making that the default?

[sfan5]

I was planning to propose that, yes. IIRC it was the default before during another time when PA was buggy.

[Infinoid]

No need for all that when you can just use --pulse-latency-hacks=yes.

This is how I've worked around the issue for now. It resolves the issue on both of the systems I use mpv on. I don't see any additional problems or extra CPU usage from it, so I'm unaware of any downsides.

[raphael-costa]

dc2d053

It was disabled in 2015

[Infinoid]

Thanks for the fix!