-
Notifications
You must be signed in to change notification settings - Fork 74
Gstreamer Video4Linux2 plugin patches
Instructions for Raspberry Pi users about a patch for the GStreamer Video4Linux2 plugin that may be needed for hardware video decoding by the Broadcom GPU on the Pi. (For GStreamer < 1.22)
-
Since UxPlay 1.63, software video decoding (using
-avdecoption) also works quite well on Raspberry Pi when the-vsyncoption (now the default since UxPlay 1.64) is also used, so while using hardware decoding is still preferred, it is no longer absolutely essential. -
Raspberry Pi OS (Bullseye) now supplies a patched plugin (but the patch here is more complete); Ubuntu 21.10, 22.04 LTS, 22.10 need patching. Latest Ubuntu 23.04 and Manjaro provide GStreamer-1.22, and do not need a patch.
-
IMPORTANT, kernel module bcm2835_codec.ko MUST be present for hardware video decoding. (Raspberry Pi OS, Ubuntu and Manjaro all have it, but if you are using a different distribution, you should check.) Look for it with "
find /lib/modules -name bcm2835_codec.ko". There is no point in patching if the bcm2835_codec module is missing.
The kernel module bcm2835_codec is maintained by Raspberry Pi at https://github.com/raspberrypi/linux/tree/rpi-5.15.y/drivers/staging/vc04_services and is not in the mainline Linux kernel. It merely accesses the Broadcom firmware, which does the actual video decoding. Only Raspberry Pi OS and distributions available with Raspberry Pi Imager (Ubuntu, Manjaro) appear to have it. If you want to use this module, and your distribution does not supply it, you need to build and use a kernel with the Raspberry Pi kernel source: instructions for this are beyond the scope of this guide.
2022-07-29 patches (backports for GStreamer 1.18.4 and later, for use with Raspberry Pi's Broadcom GPU) updated:
v4l2_1_xx_x.diff3.txt now supplies a new GStreamer patch where Apple's color settings are recognized without using the "bt709" workaround (from GStreamer development branch). (Apple Airplay Mirror protocol uses an uncommon "full-range" variant of bt709 color standard for HDTV Broadcasts that was not previously recognized by the Video4Linux2 plugin.)
-
do NOT use UxPlay option -bt709 with these new patches.
-
GStreamer-1.22 used in Ubuntu-23.04 and current Manjaro does not need any patches. (Some GStreamer error messages (due to a minor bug in the Broadcom firmware accessed by the bcm2835 module) which were suppressed by the patch will be shown by GStreamer-1.22 and can be ignored)
v4l2_1_18_4_diff3.txt for Raspberry Pi OS Bullseye
wget https://github.com/FDH2/UxPlay/files/9196641/v4l2_1_18_4_diff3.txt
v4l2_1_18_5_diff3.txt for Ubuntu 21.10
wget https://github.com/FDH2/UxPlay/files/9196643/v4l2_1_18_5_diff3.txt
wget https://github.com/FDH2/UxPlay/files/9196647/v4l2_1_18_6_diff3.txt
wget https://github.com/FDH2/UxPlay/files/9196648/v4l2_1_20_0_diff3.txt
v4l2_1_20_1_diff3.txt for Ubuntu 22.04
wget https://github.com/FDH2/UxPlay/files/9196657/v4l2_1_20_1_diff3.txt
wget https://github.com/FDH2/UxPlay/files/9196660/v4l2_1_20_2_diff3.txt
v4l2_1_20_3_diff3.txt for Ubuntu 22.04.1 and 22.10.
wget https://github.com/FDH2/UxPlay/files/9196663/v4l2_1_20_3_diff3.txt
wget https://github.com/FDH2/UxPlay/files/9212951/v4l2_1_20_4_diff3.txt
v4l2_1_20_5_diff3.txt will also work with 1.20.6.
wget https://github.com/FDH2/UxPlay/files/10298147/v4l2_1_20_5_diff3.txt
Patching instructions for GStreamer 1.18.4 and later (backport from GStreamer 1.22, will not be needed for GStreamer-1.22 or later): tested on R Pi model 4B, also confirmed to work on model 3B+.
-
An alternative to patching is to completely replace your GStreamer with the latest development version built from source: see here However, it is much easier to just patch your distribution's GStreamer package.
-
Recipes for patching The GStreamer Video4Linux2 plugin from gstreamer1.0-plugins-good are given here for (1) GStreamer-1.18.4 as used on Debian Bullseye-based systems such as Rapberry Pi OS (2) GStreamer-1.18.5 systems such as Ubuntu 21.20, and (3) GStreamer-1.20.1 systems such as Ubuntu-22.04 (4) GStreamer-1.20.3 systems such as Ubuntu-22.04-1, as well as (5) the Arch-based GStreamer-1.20.4 system Manjaro-Arm-RPi4-22.10. (These are for use until your distribution provides these updates.)
-
Note: the patched version of libgstvideo4linux2.so that you will install here will get overwritten by re-installation or installation of updates of the gstreamer1.0-plugins-good package.
The Video4Linux2 plugin works by using a kernel module to access the firmware for the Broadcom GPU which is written in the Pi's EEPROM chip. To check that the firmware is up to date, run
sudo rpi-eeprom-update
It this shows updates are available, run "sudo rpi-eeprom-update -a", and then reboot to install them.
After the reboot, run "sudo rpi-eeprom-update" again to verify that the firmware is now up to date.
Use gst-inspect-1.0 video4linux2 (In Debian-based systems, this requires that gstreamer1.0-tools is installed):
pi@raspberrypi:~ $ gst-inspect-1.0 video4linux2
Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
Version 1.18.4
License LGPL
Source module gst-plugins-good
Source release date 2021-03-15
Binary package GStreamer Good Plug-ins source release
Origin URL Unknown package origin
v4l2src: Video (video4linux2) Source
v4l2sink: Video (video4linux2) Sink
v4l2radio: Radio (video4linux2) Tuner
v4l2deviceprovider: Video (video4linux2) Device Provider
v4l2jpegdec: V4L2 JPEG Decoder
v4l2h264dec: V4L2 H264 Decoder
v4l2h264enc: V4L2 H.264 Encoder
v4l2convert: V4L2 Video Converter
v4l2video18convert: V4L2 Video Converter
v4l2jpegenc: V4L2 JPEG Encoder
10 features:
+-- 9 elements
+-- 1 device providers
UxPlay will need the v4l2h264dec and v4l2convert plugins, and v4l2h264dec needs to have fixes from gstreamer-1.21 development series backported to it. If none of the decoders, encoders and converters are shown, your system may be missing a required kernel module such as bcm2835_codec.ko.
Check for the Broadcom kernel drivers with
lsmod | grep bcm
This should show that the bcm2835_codec module is present.
- Without the bcm2835_codec kernel module, and the video4linux2 decoders, hardware decoding of h264 video by UxPlay will not be available on the RPi. Correct this before proceeding. The most likely cause for a missing v4l2h264dec plugin is that firmware is not updated, gstreamer1.0-plugins-good is not installed, or that kernel modules that access the firmware are missing.
Before you begin, install the GStreamer packages libgstreamer1.0-dev, libgstreamer-plugins-base1.0-dev, which are needed in the build process. (If you have already built UxPlay, these will be already be installed.) Also install gstreamer1.0-plugins-good.
-
The current Raspberry Pi OS (Bullseye) now has a patched package
gstreamer1.0-plugins-good-1.18.4-2+deb11u1+rpt1: you no longer need to patch gstreamer, but will need to use the-bt709option with UxPlay-1.56 or later, as the patch used was (almost) equivalent tov4l2_1_18_4_diff2.txt. (There are still some issues with the patch in the current Raspberry Pi OS package, such as (if the -v4l2 or -rpi* options are used) a crash when the image on the client is rotated, so you may still wish to apply the patch given here.)
Download the patch: v4l2_1_18_4_diff3.txt; if you are doing this from the command line, use
wget https://github.com/FDH2/UxPlay/files/9196641/v4l2_1_18_4_diff3.txt
The first step is to download the gstreamer1.0-plugins-good source package: edit /etc/apt/sources.list to uncomment the deb-src entries (or duplicate your deb entries and change "deb" to "deb-src")
and then run sudo apt update. Then do
sudo apt install devscripts
sudo apt build-dep gstreamer1.0-plugins-good
Next (in a directory where you will rebuild the package, and without using "sudo"), do
apt source gstreamer1.0-plugins-good
cd gst-plugins-good1.0-1.18.4
debuild
This rebuilds the unpatched package. When the building phase is finished, debuild seems to stall with "ninja: no work to do" while the post-build gstreamer tests are running: it's OK to just terminate debuild with ctrl-c at this point. (Some GStreamer tests that debuild runs will probably fail because some GStreamer components they need are not installed, and even if they all pass, debsign will fail as debuild finishes.)
-
If you want to be sure that the build succeeded do "
cd obj*; ninja ; cd ..". The response should be: "ninja: no work to do".
Don't worry about debuild error messages and just continue to the next step:
patch -p1 < (path to the patch)/v4l2_1_18_4_diff3.txt
cd obj*
ninja
The build system "ninja" should report something like "[24/24] Linking target sys/v4l2/libgstvideo4linux2.so" to show that the patched rebuild was successful. Then install the new patched version of libgstvideo4linux2.so by by stripping it (removing debug symbols) and overwriting the unpatched version that came with your distribution.
cd sys/v4l2
strip libgstvideo4linux2.so
sudo cp libgstvideo4linux2.so (location)/gstreamer-1.0/
Here
(location) = /usr/lib/aarch64-linux-gnu (64 bit version)
(location) = /usr/lib/arm-linux-gnueabihf (32 bit version)
and obj* is either obj-aarch64-linux-gnu or obj-arm-linux-gnueabihf.
(It is possible you may need to reboot before the patch works.) Unless it is already running, you need to start avahi-daemon
(sudo systemctl start avahi-daemon) before testing, and if the Pi is running a firewall, open up the network ports you will use
with UxPlay's -p option.
- If you are using the "Lite" version of Raspberry Pi OS without X11, use uxplay -vs kmssink . In "Desktop" versions just use uxplay (or uxplay -vs waylandsink if Wayland graphics are being used) (This assumes UxPlay-1.55 or later). The extra option -v4l2 may sometimes be helpful, as may be alternative
-vsvideosink choices on Desktop systems.
(In latest UxPlay, these are also available combined with -v4l2 as-rpi,-rpigl,-rpifband-rpiwl.)
In the test on a clean install of "Lite", sound through hdmi was got to work by installing pulseaudio (reboot after installing, and use raspi-config to select the sound output device).
- NOTE: sometimes just "uxplay" with no options gives better performance than using the -v4l2 option. This seems to vary between different operating system distributions and their versions. Explore using/not using -v4l2 and choosing different videosinks.
Procedure is same as for RPi OS, except replace 1.18.4 by 1.20.1, 1_18_4 by 1_20_1, etc.
patch is: v4l2_1_20_1_diff3.txt.
wget https://github.com/FDH2/UxPlay/files/9196657/v4l2_1_20_1_diff3.txt
Use "settings" to set the sound output device. UxPlay options -v4l2 -vs glimagesink (or -rpigl) seem to work well.
- Installation on Wayland-based Ubuntu 21.10 is similar, using the patch for 1.18.5. Here use -vs waylandsink.
Same as above, but replace 1.20.1 , 1_20_1 by 1.20.3 and 1_20_3.
patch is: v4l2_1_20_3_diff3.txt
wget https://github.com/FDH2/UxPlay/files/9196663/v4l2_1_20_3_diff3.txt
In this case, with Ubuntu-22.04.1, some post-recompilation testing steps in debuild also fail, before "debsign" is reached. This seems to be because Ubuntu 22.04.1 updated some of gstreamer to 1.20.3, but left the installed base packages at version 1.20.1. The failed tests do not involve video4linux2, so they don't matter. Ignore them and continue to the patching step. (3 tests failed on Ubuntu-22.10: again ignore them.)
Non-Debian systems (e.g., Manjaro-Arm 64bit Linux for RPi 4) with GStreamer-1.20.x: patch the official GStreamer source.
- (No longer needed, as current Manjaro supplied GStreamer-1.22)
First make sure build tools such as git, gcc, cmake, and patch are installed. Also test for pkg-config with "which pkg-config": install it if it is not found (packages for pkg-config can also have names like "pkgconfig" or "pkgconf").
Before you begin, install the GStreamer packages needed for building UxPlay, plus the GStreamer "plugins-good" package.
The GStreamer source will be taken from the official GStreamer site. Patches can be found at the top of the this page.
On Manjaro (currently using GStreamer-1.20.5), (install package) = pacman -S.
Use the patch v4l2_1_20_5_diff3.txt
wget https://github.com/FDH2/UxPlay/files/10298147/v4l2_1_20_5_diff3.txt
- (install package) meson flex bison
- git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git
- cd gstreamer
- git checkout 1.20.5
- cd subprojects/gst-plugins-good
- patch -p1 < (path to the patch)/v4l2_1_20_5_diff3.txt
- meson setup build
- cd build
- ninja
- cd sys/v4l2
- strip libgstvideo4linux.so
- sudo cp libgstvideo4linux.so (location)/gstreamer-1.0/
On Manjaro, (location) = /usr/lib/.
- make sure that step 6
meson buildsucceeded before continuing to step 7 (it will fail if bison or flex are not installed)
To patch GStreamer 1.18.x , (example is 1.18.4: patches only available for 1.18.4 - 1.18.6), using gitlab.freedesktop.org source:
- (install package) meson flex bison
- git clone https://gitlab.freedesktop.org/gstreamer/gst-build.git
- cd gst-build
- git checkout 1.18.4
- meson setup build
- cd subprojects/gst-plugins-good
- patch -p1 < (path to patch)/v4l2_1_18_4_diff3.txt
- meson setup build
- cd build
- ninja
- cd sys/v4l2
- strip libgstv4l2videodec.so
- sudo cp libgstv4l2videodec.so (location)/gstreamer-1.0/
- make sure that step 4
meson buildsucceeded before continuing to step 5 (it will fail if bison or flex are not installed)