Optimization for reverse trickspeed

[fwolter]

I just want to summarize the facts and findings around the reverse trickspeed mode, if, in the future, we want to optimize it, i.e., find a solutions that works with all hardware platforms and all FFmpeg versions without any quirks *dreaming*.

Potential code optimizations/simplifications:

• Reuse DRM buffers like in forward trickspeed and normal playback. The recreate/destroy logic could be removed.
• Remove Raspberry PI specific quirk QUIRK_CODEC_FLUSH_WORKAROUND.

Facts:

• Currently the video decoder is flushed in reverse trickspeed after a fixed number of frames. This number depends on the codec and if it's an interlaced or progressive stream.
• Forward trickspeed works without flushing the video decoder. Question remains if that's a coincidence or if the decoder can be fed with only keyframes without flushing it on a regular basis. AI told me that some video codecs can reference frames across key frame boundaries, i.e., P/B-frames before the last I-frame. That statement needs to be verified, but it seems reasonable to flush the decoder after each key frame to reset the decoders internal state (which is not done so far, without any problems).

The following tests were done in reverse trickspeed, reusing the DRM buffers:

Codec	RPI5, w/o flushing	RPI5, with flushing, applying quirk	RPI5, with flushing, not applying quirk	RPI4, with flushing, not applying quirk
MPEG2 576i SW decoded. Also with mixed interlaced frames.	OK	OK	OK	Not tested
HEVC HW decoded	OK	Cycles through the last 7-8 key frames	OK	Not tested
h.264 720p SW decoded	Receiving no frames in avcodec_receive_frame()	OK	OK	Doesn't work (HW decoded)
h.264 1080i SW decoded	Receiving no frames in avcodec_receive_frame()	OK	OK	Not tested

Faking the PTS, so that the video decoder only sees monotonically increasing PTS, doesn't make any difference with RPI5. Not tested with RPI4.

When destroying and recreating the DRM buffers, and applying the quirk after each decoder flush, all codecs work on RPI4 and RPI5.

Tested with this FFmpeg version:

# ffmpeg -version
ffmpeg version 5.1.7-0+deb12u1+rpt1 Copyright (c) 2000-2025 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14+deb12u1)
configuration: --prefix=/usr --extra-version=0+deb12u1+rpt1 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --disable-mmal --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sand --enable-sdl2 --disable-sndio --enable-libjxl --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
libavutil      57. 28.100 / 57. 28.100
libavcodec     59. 37.100 / 59. 37.100
libavformat    59. 27.100 / 59. 27.100
libavdevice    59.  7.100 / 59.  7.100
libavfilter     8. 44.100 /  8. 44.100
libswscale      6.  7.100 /  6.  7.100
libswresample   4.  7.100 /  4.  7.100
libpostproc    56.  6.100 / 56.  6.100

H.264 in reverse trick speed mode on RPI4, reusing the DRM buffers, not using QUIRK_CODEC_FLUSH_WORKAROUND, using rpi-ffmpeg, does not work. Question remains if that is an rpi-ffmpeg issue or if it is related to the combination of RPI4 and its H.264 HW decoder.

[fwolter]

I assume that all DMA-BUF handles become invalid when the decoder is reopened. I don't know if it is a problem when the handle is invalidated while the frame is still being displayed. This is probably the cause why this pops up during reverse trickspeed:

[softhddevice] drmbuffer: Destroy: error closing prime fd 32 (9): Bad file descriptor

The handle is closed in cDrmBuffer::Destroy(), which is called by the display thread, but the reopening of the decoder is done asynchronously in the decoder thread. When the decoder thread reopened the decoder before the display thread closed the handle, closing the handle (again) fails.

[fwolter]

RPI4 with rpi-ffmpeg seems to need destroying each buffer also in forward trickspeed. Otherwise, there will be more than max RENDERBUFFERS (36) prime FD <-> DMA-BUF handle combinations, and the cache will become exhausted.