gstreamer 1.20.0 breaks video on Linux

[kflak]

Hi,

I just updated my system, and it seems gstreamer breaks any video playback. Getting this when running the videoPlayer example:

[notice ] ofGstUtils: Got context from element 'vaapipostproc0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";

[notice ] ofGstUtils: Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0", gst.vaapi.Display.GObject=(GstObject)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0";


(videoPlayerExample:12633): GStreamer-Base-CRITICAL **: 16:43:43.970: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

(videoPlayerExample:12633): GStreamer-Base-CRITICAL **: 16:43:43.970: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

(videoPlayerExample:12633): GStreamer-Base-CRITICAL **: 16:43:43.971: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

(videoPlayerExample:12633): GStreamer-Base-CRITICAL **: 16:43:43.971: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

(videoPlayerExample:12633): GStreamer-Base-CRITICAL **: 16:43:43.971: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

(videoPlayerExample:12633): GStreamer-Base-CRITICAL **: 16:43:43.971: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

(videoPlayerExample:12633): GStreamer-Base-CRITICAL **: 16:43:43.972: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink
[ error ] ofGstUtils: startPipeline(): unable to pause pipeline after 5s[ error ]
ofGstUtils: gstHandleMessage(): embedded video playback halted for plugin, module qtdemux0  reported: Internal data stream error.[warning] ofGstVideoUtils: update(): ofGstVideoUtils not loaded

Needless to say, this is a HUGE bug. Please, please, please wait with upgrading gstreamer until this is fixed.

Running this on Arch Linux, oF version 0.11.2. Tested on xorg (bspwm) and wayland (sway).

EDIT: tested now with the latest nightly build, of_v20220210_linux64gcc6_nightly, and getting the same result.

[kflak]

Additional information: My computer runs an AMD iGPU and an NVIDIA eGPU. My second computer, which only has an intel iGPU, shows the same behavior: works perfectly fine before upgrade, no joy after.

[kflak]

I downgraded now to 1.18, and all is well. Of course, this is not good still, as it effectively locks my computer to a specific point in time, excluding any updates until this works. I am very inexperienced with c++ in general, but if somebody could point me in the right direction I might take a stab at working on a fix?

[kflak]

Starting to look into this a bit more now... How can I set the path to the gstreamer binary that oF looks for? I am thinking of keeping a separate build of gstreamer 0.18 on my system, so that I can use this for my oF projects. Processing has released a new version of the video library that bundles the old version, and this works on my system.

[StaffanMelin]

I can confirm this problem.

It works on Debian 11 Stable with GStreamer 1.18
but not on Debian 11 Testing with GStreamer 1.20.

I get the following error in case 2:

[notice ] ofGstVideoGrabber: initGrabber(): selected device: SHUNCCM2MP: SHUNCCM2MP
[notice ] ofGstVideoGrabber: initGrabber(): selected format: 640x480 video/x-raw YUY2 framerate: 30/1
[notice ] ofGstUtils: setPipelineWithSink(): gstreamer pipeline: v4l2src name=video_source device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1  ! videoconvert   ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw, format=RGB, width=640, height=480"
Segmentation fault

I will try to get more information on the error.

[kflak]

Thanks for confirming! I was starting to wonder if I am the only one with a 1.20 problem :-) For what it's worth, I wrote a guide on how to revert to 1.18 on Arch Linux. Not exactly recommended practices, though... https://roosnaflak.com/tech-and-research/restoring-gstreamer-to-1-18-on-arch-linux/

[StaffanMelin]

Thank you for the link @kflak! I will check it out.

I tried to debug the issue, using the videoGrabberExample program.

ofApp.cpp:
vidGrabber.initGrabber(camWidth, camHeight);

which calls

bool ofGstVideoUtils::setPipeline(string pipeline, ofPixelFormat pixelFormat, bool isStream, int w, int h)

(in ofGstutils.c line1076)

This function seems to work, but when it returns (line 1164)

my program seg faults.

[kflak]

Unfortunately I am not able to do a debug anymore, since I am back on 1.18, however I never got a segfault. In my case the plugin was simply disabled, and I got a gray screen instead of video. The program kept running, which makes me a bit suspicious if we are dealing with the same bug...?

[StaffanMelin]

Good question, @kflak!

In the OF forums there is a similar report (to mine, with the seg fault):
https://forum.openframeworks.cc/t/segmentation-fault-on-examples-computer-vision-opencvexample/39508

[StaffanMelin]

Last try: I am now on GStreamer 1.20.1, and using the OF nightly build from 4th of April. Still a seg fault, unfortunately. Going forwards with debugging and fixing is beyond my abilities. But I will gladly help making more tests, providing info etc! :)

[dimitre]

Not sure if it is related but I've seen an issue on GStreamer itself posted here
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1083
And it seems some videos, not all trigger the segfault. Maybe this is related to the difficulty to reproduce in different systems.

[StaffanMelin]

Thanks, @dimitre, this could be related. I see the person with the bug was running 1.18.5 of GStreamer, I have 1.18.3/4 on my Debian 11 Stable (which works).

@kflak, exactly which version did you revert to?

[kflak]

I reverted to the newest possible version of 1.18, which is 1.18.5.

[kflak]

Not sure if it is related but I've seen an issue on GStreamer itself posted here https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1083 And it seems some videos, not all trigger the segfault. Maybe this is related to the difficulty to reproduce in different systems.

A bit silly of me not to raise an issue on gstreamer immediately, but I wasn't sure if this was just an idiosyncratic quirk of my system or not. I will soon install arch on a new computer, and will take the opportunity to test 1.20 then on a fresh install.

PS: the work on your website looks great, @dimitre!

[StaffanMelin]

I don't really know how to move further with this. I get a seg fault even when trying to do video capture...

[cherio]

I confirm I have the same issue in GStreamer Core Library version 1.20.1 when playing the well-known 4K demo video from SONY "Sony Swordsmith HDR UHD 4K Demo" in parole. The error I get is

(parole:4094): GStreamer-Base-CRITICAL **: 13:27:30.761: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad vaapipostproc0:sink

It is reliably reproducible with that video on a freshly updated ArchLinux with packages from the standard Arch repositories. This looks like something of a pretty high priority to me

[dimitre]

great @cherio I think you should add this info to the gstreamer gitlab so they have more info to reproduce the issue (hopefully fix)
with a reproducible case somebody can use "git-bisect" to identify exactly when the issue was introduced.

[StaffanMelin]

I have posted a link to this thread on the Gitlab thread for the issue (the similar one) for GStreamer. I hope this can be of some help. I hope I did the right thing! :)

[kflak]

Great! Hope this will find a resolution soon...

[arturoc]

This error and the one in the gstreamer gitlab don't seem related. This is not a segfault and is likely related to the latest gstreamer using hardware accelerated components to decode video for which we don't have very good support yet since we recover the pixels of each frame to be able to access them from the cpu. There is a similar fix for a previous version to avoid using the hardware accelerated path but it seems that doesn't work anymore. The support for that is also probably much better now and it might be easier to integrate rather than completely avoid it, which would also make playback of big videos way less cpu consuming

[kflak]

Sounds really good if this could be integrated. A lot of my use-cases are simple video playback + processing, and getting hardware acceleration going would be very good... I'd be happy to guineapig any solutions on my arch linux laptop.

[cyphunk]

To add some salt, I get same issue using a Magewell Capture SDI USB3 (and also when trying to capture from laptop webcam). Arch linux also, gstreamer 1.20.1. Segfault at same point

However, if I attempt to run gstreamer directly, using the pipeline that ofVideoGrabber example prints on init, it works:

gst-launch-1.0 v4l2src name=video_source device=/dev/video2 ! video/x-raw,format=YUY2,width=320,height=240,framerate=30/1 ! videoconvert ! appsink name=ofappsink enable-last-sample=0 caps="video/x-raw, format=RGB, width=320, height=240"

Above test will not show a image but will show that the pipeline is running. To see an image I used this:

gst-launch-1.0 v4l2src name=video_source device=/dev/video2 ! videoconvert ! videoscale ! queue ! videoconvert ! ximagesink name=img_origin

For what it is worth running under valgrind seems to imply an invalid dereference:

    ==553247== 1 errors in context 1 of 1:
    ==553247== Jump to the invalid address stated on the next line
    ==553247==    at 0xA: ???
    ==553247==    by 0x48896F4: ??? (in /usr/lib/libgstapp-1.0.so.0.2001.0)

[StaffanMelin]

Here is another report on the issue from the oF forum:
[Va-api ofVideoPlayer error intel and nvidia Linux](https://forum.openframeworks.cc/t/va-api-ofvideoplayer-error-intel-and-nvidia-linux/39780)

[kflak]

This is becoming more than a bit of a problem, not just in terms of openFrameworks, but also in terms of other pieces of software that depend on it and no longer work after update... Is there any way to specify which binaries openFrameworks will look for when linking to gstreamer? I would prefer to keep a separate gstreamer 1.18 + plugins around for oF until the issue has been resolved, but I have no idea how to go around pointing oF in the right direction... Any help is MUCH appreciated!

[dimitre]

@kflak I've seen some serious regressions introduced in Fontforge project, (which is large and complex)
and I remember they used the bisect git command to identify exactly where the regression was introduced first to re-evaluate the portion of code which was causing the issue in the first place.

I think GStreamer project could use this strategy to identify exactly where things stopped working

[kflak]

My hunch is that it's probably a bunch of breaking changes on the gstreamer side, and that the oF side of things needs to be updated to catch up, unfortunately. Would be happy to lean into it, but I don't know nearly enough about these things to contribute much, except for testing the results...

[kflak]

Now things are a bit different, though, from when I first reported the bug. Using all the latest updates on everything (including oF nightly 20220528), I simply get a segfault and a coredump that looks like this:

           PID: 28966 (videoPlayerExam)
           UID: 1000 (kf)
           GID: 1000 (kf)
        Signal: 11 (SEGV)
     Timestamp: Sat 2022-05-28 21:23:21 EEST (28s ago)
  Command Line: bin/videoPlayerExample
    Executable: /home/kf/of_v20220528_linux64gcc6_release/examples/video/videoPlayerExample/bin/videoPlayerExample
 Control Group: /user.slice/user-1000.slice/session-1.scope
          Unit: session-1.scope
         Slice: user-1000.slice
       Session: 1
     Owner UID: 1000 (kf)
       Boot ID: 28ed1c28eedd490987638def96460232
    Machine ID: c06afc9c267144ec9253e59c7c767a00
      Hostname: t480s
       Storage: /var/lib/systemd/coredump/core.videoPlayerExam.1000.28ed1c28eedd490987638def96460232.28966.1653762201000000.zst (present)
     Disk Size: 4.4M
       Message: Process 28966 (videoPlayerExam) of user 1000 dumped core.
                
                Module /home/kf/of_v20220528_linux64gcc6_release/examples/video/videoPlayerExample/bin/videoPlayerExample with build-id 52fab3acc9b30a87dc2738e513182e22f5678a9e
                Module linux-vdso.so.1 with build-id 6342ef35896f0698c8f766d26f0dba7d3c720bc0
                Module libgstdeinterlace.so with build-id 878471e7bbf61e0ad4ac519336a7dccab3604d24
                Module libgstvideoscale.so with build-id 4aa8ddf11fe026c7280290c8f3668c45017c48cf
                Module libgstvideoconvert.so with build-id 1d6d709382be3a41bc0bef7eeb42b715947ac093
                Module libgstvideofilter.so with build-id 92dab7f237c65f6dcbd0b3825ff0fc9c02a7c1d9
                Module libdatrie.so.1 with build-id 6fe3b6ece2c8e7d11869fa051375128d8f808f58
                Module libthai.so.0 with build-id a7ac5010b4275c49308021200d23690533952702
                Module libpangoft2-1.0.so.0 with build-id c2c09f789578900f61b7fca4a4311d8b94d9a750
                Module libdl.so.2 with build-id 94198b268228074fa9f405bbedbbae94112593ed
                Module libmvec.so.1 with build-id 2c6aec81c79e91d66940b68be0819fdf780a976c
                Module libpango-1.0.so.0 with build-id 7e27c1e46a1d958f6b16e1ba199f8bdb3f100566
                Module libpangocairo-1.0.so.0 with build-id b65f507d9e33adfbd19369acef5e5a0a2422c6d1
                Module libgdk_pixbuf-2.0.so.0 with build-id 5b8422ab971b1a8a8e1c43b88738d4ee217f609e
                Module libcairo-gobject.so.2 with build-id 9927090dddfb180365695a8f37d1cac89463f109
                Module libhogweed.so.6 with build-id 2d70cff7b1841b4d9ca4e8e7726cd4b944c07fdc
                Module libnettle.so.8 with build-id 9a878e513c02007598fcf1e2e286c2203f13536e
                Module libtasn1.so.6 with build-id ee3429ca5e94718aea4fe5249fc859e0cd88e4e9
                Module libbrotlienc.so.1 with build-id 74adbc62e4fbb5da9d37b5aa458471f4130862ff
                Module libp11-kit.so.0 with build-id cc372ea3c28c4d3dfc633b4d2e933c8584d2af16
                Module libsoxr.so.0 with build-id 9e49936ed807c1e52705b88d5d71706fe11d9ed1
                Module libvdpau.so.1 with build-id fccd021ff13c37ebf7144d184a3250624e086f52
                Module libva-x11.so.2 with build-id 0c5ca7869e7110b0e92180d5e7ddb330d6f14341
                Module libva-drm.so.2 with build-id 7a02a127b38763e2ec9a2503f0ae4c3d57c20327
                Module libxvidcore.so.4 with build-id 43bc390a7842a7ca3a0302e93442ea561d7f013a
                Module libx265.so.199 with build-id 4fd2d7155725df753c6b32274c442349fc05e632
                Module libx264.so.164 with build-id 1ebb8708f2101bd1c95b701142163133e8dac465
                Module libtheoradec.so.1 with build-id 19d321fd5455c7b7611ca9602a804b496124e06f
                Module libtheoraenc.so.1 with build-id cde750f0621140f313e4ef4e5f212e662f24a38b
                Module libSvtAv1Enc.so.0 with build-id e9143f88be53d2aebfa4f9249295869f09e22be4
                Module libspeex.so.1 with build-id 39721dbe21b62978562c2943279f6dffbffe8946
                Module librav1e.so.0 with build-id 5f8fc2c271f0d96480ab1c71447e038a66aa1c57
                Module libopencore-amrnb.so.0 with build-id a3d78948c85fa235e66549119b2f130e69af7b70
                Module libmp3lame.so.0 with build-id 48c901805551a3dd58d02502c4cd67f6aa848107
                Module libgsm.so.1 with build-id 1cc6240f7eaedaaac302bc7c0f64ebbeae596f22
                Module libaom.so.3 with build-id 3c057fe8c5fa7033366274352148acb4dbb42728
                Module librsvg-2.so.2 with build-id 8c033bf594f51f48c7b2ee078fa4586ddb70513a
                Module libopencore-amrwb.so.0 with build-id 28da8ed3dfca59280e835c46bd40da295dae99e2
                Module libdav1d.so.6 with build-id a93560a23a7f1def734a3cf54f09eadd7c3c46e9
                Module libvpx.so.7 with build-id 347ca2580a23a6da7347be245c72585415bfa8e2
                Module libssh.so.4 with build-id 8490056688e60790230f9d990c439b181236e9b6
                Module libsrt.so.1.4 with build-id ad13eeb20e5ea3f04a0b089de1152ff89a9fa65c
                Module libgnutls.so.30 with build-id 856619ec7814fc7f2704e235247d54ed23648b98
                Module libgmp.so.10 with build-id 26cec2ebe94cc5c4cb99e6988717347222b324fd
                Module libbluray.so.2 with build-id 3ed75b9f8aa64ee4707bbfcb986bcf289432d673
                Module libmodplug.so.1 with build-id b00dcb12e7d2d5882b4fa414295eba449875dae4
                Module libmfx.so.1 with build-id c5ae5843bb9f5a71745214de0b8c2edc8ccb7570
                Module libzimg.so.2 with build-id c46ecaf11c38c05ef7a61dde386654c87be24e41
                Module libvidstab.so.1.1 with build-id 8f75d19083cbc44f1cfda8fca285a23424dd2b2a
                Module libva.so.2 with build-id ae5f91d7b4849e6f918da2e24825f9492c526118
                Module libass.so.9 with build-id b58e2f7ce00154755f6d39e7e83718f82ff2bfa1
                Module libvmaf.so.1 with build-id 835cb1e52c0e6e216daeac24e5416dfb46ad4aa6
                Module libfribidi.so.0 with build-id fe9f35ac2a0074108c8306c517793f7279bd9b37
                Module libswresample.so.4 with build-id afca7bcb3d53246c257a67b852fc922b240fe14a
                Module libpostproc.so.56 with build-id 06de1e4801c6086c862f311fb68340d528432365
                Module libswscale.so.6 with build-id 644e14c585951b7f68aad608aec698b6bceb27da
                Module libavutil.so.57 with build-id 02316d81b1e1685729cedd19caebefd52d8fb986
                Module libavcodec.so.59 with build-id 9b7c8f13e401225a3e10e315841d360348415f55
                Module libavformat.so.59 with build-id cadfebbf354b318442c077a7eeb253de55d58a6b
                Module libavfilter.so.8 with build-id 56e9a3a219bff388b003e6a1027970b4129030f9
                Module libgstlibav.so with build-id 4feb5aa8f97f3d9219c5c5cf730cd5556d898a28
                Module libgstcodecparsers-1.0.so.0 with build-id c068c39260a54ff24622841be06b37203c46ce3d
                Module libgstvideoparsersbad.so with build-id 0fa2606972bab07deb41fac49022bfef50d78d19
                Module libgstrtp-1.0.so.0 with build-id 089b8cd17b05b21ddb46ddd136eb09e666cebdfb
                Module libgstriff-1.0.so.0 with build-id 316c186fe35012f87e65060b5b6c70b5c0bf8fa1
                Module libgstisomp4.so with build-id 65532ed2ae67c68b84857644fde47eec8aaa73a8
                Module libblkid.so.1 with build-id 140694a62d8d4d07c6c320a501f948dd1b389d73
                Module libmount.so.1 with build-id 4436aeea0cd8c01b5a77969e0531184f8b3513ce
                Module libgio-2.0.so.0 with build-id 8c411770513e0bc758a1bbe661af14c972e2a3c4
                Module libgsttypefindfunctions.so with build-id 26c68db72f33688232bd91f70065d9b854a73193
                Module libgstcoreelements.so with build-id 460ef2453d38c8546389c0b5f3b8cc35e5f2e0cf
                Module libgsttag-1.0.so.0 with build-id e7ab2f427c15e4fe2f27cafc16822028a4d8209c
                Module libgstpbutils-1.0.so.0 with build-id 679e6750fc39c0720a37de66b4c293c69608a533
                Module libgstaudio-1.0.so.0 with build-id 43f3817e44d149650f277ea5f160f091751ca76e
                Module libgstplayback.so with build-id f3b6d23f29ed90bedb91817b161cf7bc657b922f
                Module libicudata.so.71 with build-id 4fef196388e678deb881978139e125e20ee2d94d
                Module libicuuc.so.71 with build-id 633fdc0c5385d916571f6140e7a978ad0630ef55
                Module libxml2.so.2 with build-id 8cdf00fa954d9a27f2f184c4d354cb14677446ac
                Module libncursesw.so.6 with build-id b9917757481e6fa6097e2a1f31f5bb5eaf138c4e
                Module libedit.so.0 with build-id 4b0babfcad161c2ad0af6e59e2493258db23a331
                Module libvulkan.so.1 with build-id 198212a538d4e3b290530d75a0614e803b03d39e
                Module libdrm_nouveau.so.2 with build-id af8afb2331ac08c52132102ecbdc6076a8eb4c6f
                Module libdrm_amdgpu.so.1 with build-id 8eb3977b830f3012d9ead7b4def5f4d2f38ec688
                Module libdrm_radeon.so.1 with build-id 43e0aecc70c4ce905ba35c948e1c9a5d999b14c0
                Module libsensors.so.5 with build-id dc8b2c1c0d8525411aca188ea3cc3fb86d381d30
                Module libLLVM-13.so with build-id 90a760361bef9d2de117d7e696dad26fc4cb24f4
                Module iris_dri.so with build-id e00d40d9af60ae8c5890e421bd6414c8479b6bce
                Module libxcb-xfixes.so.0 with build-id e231a68d00ee4cfa12a2c31e755a9e2c1e7be450
                Module libxshmfence.so.1 with build-id d9b3ce4edb0b6a0818bcc57dda020d2da5e3c895
                Module libxcb-sync.so.1 with build-id 13025f6de23a271636ad321c77eda98801e62e8e
                Module libxcb-present.so.0 with build-id 8ed389d9cd6ad7110fbb00c93b32e0efaa71a6b1
                Module libxcb-dri3.so.0 with build-id 088c750254cc139e7d62b8a3fc7795a138447ccf
                Module libxcb-dri2.so.0 with build-id 7bde0abc84135ae6258a8d1d130864071c2cc327
                Module libxcb-glx.so.0 with build-id 71d5b6691f53a235696da49ffa2178d643ecc228
                Module libdrm.so.2 with build-id 00816f0a71c7e4e388c10db6245dbb9c031732a9
                Module libglapi.so.0 with build-id 7320999a92c1f2c0f14ab2485507d54d7317bcc0
                Module libGLX_mesa.so.0 with build-id d3d9aa7a3a9daef558ce72ea111a64a183efe8cf
                Module libX11-xcb.so.1 with build-id f6beacd41cd16a23f53e5e56bb599e98f039d0d6
                Module libXinerama.so.1 with build-id 8198240259261b612189e89c9fcfc902b025b382
                Module libXfixes.so.3 with build-id 0a05c7e8714522bfbdd7c0027c3e2a94965664b0
                Module libXcursor.so.1 with build-id f0525ae3715ac95f96a8cdb44795b03f831334b0
                Module libXrandr.so.2 with build-id 154e55f082ee9e685d0794c98c5b76ffe9c8868e
                Module libXi.so.6 with build-id 16603be937a02a7e61b0b0395d064be7efd86f49
                Module libXxf86vm.so.1 with build-id b8b37b0c66be0a760a8491e3c86a00c4efdc825d
                Module libgomp.so.1 with build-id 5dad42990289bea94eac11b596b80f28415a8c22
                Module liblcms2.so.2 with build-id c01e5649a38dbf37768d02dc751b52de0dcbd141
                Module libjasper.so.6 with build-id 57c3ea3d5415d7cc76bc11e1dca499a3c48daf71
                Module libIlmThread-3_1.so.30 with build-id fb61aaa8abdf748eb68984c0ef25b0ae85a6565f
                Module libjpegxr.so.0 with build-id f7adc32a52bc5e991329263d8460fa27c977d3b6
                Module libXdmcp.so.6 with build-id d864159ab0008415667db8d5f251696d75c90df2
                Module libXau.so.6 with build-id 60db1eac70f819bea9d4c366603c1583067510b4
                Module libresolv.so.2 with build-id 89a368a6ad1b392d126a2a5beb9c2f61ade00279
                Module libkeyutils.so.1 with build-id ac405ddd17be10ce538da3211415ee50c8f8df79
                Module libkrb5support.so.0 with build-id 36db7c21bc57a9b934ad2d3463782db9fef4af07
                Module libcom_err.so.2 with build-id 358b783c9b3d12ba8248519ea2e7f3da4c4e0297
                Module libk5crypto.so.3 with build-id f1c113fea46023cdf62c7e51cb643c45831abdd5
                Module libkrb5.so.3 with build-id 9350783c946d77aec8ee8fd9c6bc12f4f7b72be9
                Module libunistring.so.2 with build-id 617dbf3d3d6f85d6556a7a036e23845e95490158
                Module libbrotlicommon.so.1 with build-id acfd597a977c8087bb6184383daae2e828a9ce42
                Module libgraphite2.so.3 with build-id ce58945ebb55b86d3a4e717b6eae29efc4720d8e
                Module libelf.so.1 with build-id c09e357a1a3b061bf219bcfb243ca31b175e77fa
                Module liblzma.so.5 with build-id 28b40c7af8098a66af6ee093b6986b91cad7694d
                Module libpthread.so.0 with build-id 95ae4f30a6f12ccbff645d30f8e1a3ee23ec7d36
                Module libwebp.so.7 with build-id 0add7a572705244ed810b7e722cce7da0dcfb62c
                Module libwebpmux.so.3 with build-id 4232d5263a45719ba12f58ed7d8cae3a0fc887de
                Module libtiff.so.5 with build-id 9e8868622f8b7144fd82dabfa8ac2fcaf6d45a34
                Module libraw.so.20 with build-id 6140fabd31b4e8f1e8a6a2e6cff9d53a3358e8e6
                Module libopenjp2.so.7 with build-id 5fa7037f0c932c285735dbcd42365abe6c3abb05
                Module libImath-3_1.so.29 with build-id 4c19b87c4e4467447b18d010cd539647a59d5b69
                Module libIex-3_1.so.30 with build-id 5e8a8f33fa7d4b83185155f19d4604501a86cf62
                Module libOpenEXR-3_1.so.30 with build-id 875e7df47bd5ce2de8806bb9fa046061fa43a8ef
                Module libjxrglue.so.0 with build-id 330109323ba220c8ed17d87747d03928bf908e04
                Module libjpeg.so.8 with build-id 8e6d3f3e8f438912b561c43b6e7f66e6e5e097d0
                Module libpcre.so.1 with build-id 845483dd0acba86de9f0313102bebbaf3ce52767
                Module libffi.so.8 with build-id f0a9586cf0f42d2b9971bd1065ca3a6b19f4a2c2
                Module libpixman-1.so.0 with build-id d2170a3ac106c2a68597bf7910ab04b1cdd69c14
                Module libxcb-shm.so.0 with build-id 828fec4d856e2710e732ea8d92c3f250c807b1c2
                Module libxcb-render.so.0 with build-id b1ca498d665807ab0ccdafbe8070853efd058173
                Module libXrender.so.1 with build-id 42e386d2acf3cde61081959d9671ca74acfb3edc
                Module libXext.so.6 with build-id 17beadf1cb40d41ab36629db3b4eed74110678a7
                Module libxcb.so.1 with build-id 13d677412a71468381b11092915d231f664d18d3
                Module libGLX.so.0 with build-id 02bd4ac6acc4da9288c543ea1a0cdaa70c5e151a
                Module libGLdispatch.so.0 with build-id de981c020bd9dbdc159b11f4a2a541310cc3b8d5
                Module libzstd.so.1 with build-id 3bccb8fe08e48d5ea135b1d0f99de0d771dd752f
                Module libgssapi_krb5.so.2 with build-id 1813217c8baf82a1078772eed0d868310cd59f8b
                Module libcrypto.so.1.1 with build-id d54a7ee1e288aeae436d073277ff986e03994b15
                Module libssl.so.1.1 with build-id 25a55106cb13dec5d495c34ca1caf2c2f3114f11
                Module libpsl.so.5 with build-id 0229a201aaf5652186c9fdc192ebe52baf19d7f1
                Module libssh2.so.1 with build-id a4adfe44cc7ebd295b3b783361acc3dcfcea1d50
                Module libidn2.so.0 with build-id 1ce2b50ad9f9821c2c629b521cf5a3c99593d332
                Module libnghttp2.so.14 with build-id f2738fead8e6593084b4fb8756f460aa8cf5535a
                Module ld-linux-x86-64.so.2 with build-id fc93487393eea02b5bc6e76e48976fc325294c24
                Module libogg.so.0 with build-id ca0cae30a809f10db1f16e54a23b4e5ff7b2c54c
                Module libvorbis.so.0 with build-id f4e086347f6c1a66eb73cfe13974a29bd39aabca
                Module libopus.so.0 with build-id 35fafc4024675e9cfe57c7d50c3965eac6bac358
                Module libFLAC.so.8 with build-id d3c01b1bab8e878ff3928ba64a5817506bad1aa6
                Module libvorbisenc.so.2 with build-id 2b1ce7fcdcc925bc663ebe2c1c008c0dd96e5bd0
                Module libbrotlidec.so.1 with build-id 66c54e9301f7e102ecc1d88547e5f0e8a056fe22
                Module libharfbuzz.so.0 with build-id c58fe082cbde02fc176e3c3663a6d81386eb5027
                Module libpng16.so.16 with build-id 2dc0bce07f199bf983c07a05fb95a6f4af83a9b3
                Module libbz2.so.1.0 with build-id 919597c477c9b2cb9cdbb7745ed6494ac0e6da60
                Module libz.so.1 with build-id fefe3219a96d682ec98fcfb78866b8594298b5a2
                Module libexpat.so.1 with build-id 113bb5a3e9ad856801bfcfc029102c9bdc13d67e
                Module libdw.so.1 with build-id 0b01e42545607f3bf9f564f38e24664b6ca8051b
                Module libunwind.so.8 with build-id e7f3b22127b00d8b14b2560df707cbbeb0134861
                Module libgmodule-2.0.so.0 with build-id c6df28882f153ee03d0fb33bb8aac2b219ebaacc
                Module liborc-0.4.so.0 with build-id a95fb07ab70adc17bb91684e07695723aac3a4b8
                Module libc.so.6 with build-id 388993b6ef62f964bc7bf473c069fbfe957b9e44
                Module libgcc_s.so.1 with build-id 0e3de903950e35ae59a5de8c00b1817a4a71ca01
                Module libm.so.6 with build-id 210ec9905e41825671210f8f7d0b24d6c371196a
                Module libstdc++.so.6 with build-id b3eaebcb5369487562e52dd8d016c9cd77f80c54
                Module liburiparser.so.1 with build-id d361ae41e1efe517bbdfb8f158a0eb1cc9f8a8b4
                Module libfreeimage.so.3 with build-id 2358653c6a3cba9e15c8074f043e2bd9a90a310a
                Module libmpg123.so.0 with build-id 10c27395f4c5b56e99eec209708f782b92243100
                Module libglib-2.0.so.0 with build-id e268e80e668ce871312cf50e9969f88b086e33f2
                Module libgobject-2.0.so.0 with build-id 4337ec84b2f33ec39a95863d47bfca0a28afd488
                Module libcairo.so.2 with build-id a222d042e56108d2786ece7bf291b56ba2069591
                Module libX11.so.6 with build-id d8e0be8e0323aa421366f19065ecd1c76405c130
                Module libGL.so.1 with build-id b915a2e48be6c9fd3cc6f9f4c7aab1f7aeed8a4c
                Module libGLEW.so.2.2 with build-id 1b7e477ea0eccd1622a983c8589930e043700545
                Module libglfw.so.3 with build-id f6e0794d1a70b45f1d03eac55b47ad4b7c4cc755
                Module libcurl.so.4 with build-id 8ab1352b26fc2bb50933b8caa480e032b0b7d6b3
                Module libopenal.so.1 with build-id 84afd51dd3f7ff484bce2d675078edb40bc1a7de
                Module libsndfile.so.1 with build-id 407b2376cac1726631e3f7360a9337901be04ec0
                Module libfreetype.so.6 with build-id f89dd5502e75aca28fb5c3ccd0dbd26fe822bfef
                Module libfontconfig.so.1 with build-id 36be6951b8c1e42a7dd05684a37400fc8ef9147c
                Module libgstreamer-1.0.so.0 with build-id 76909eb7d52bf4f1dbdc5c024b6e40d7acdc8866
                Module libgstbase-1.0.so.0 with build-id 8bb544daffa02b07770b07cd332dec383b4f6a9c
                Module libgstvideo-1.0.so.0 with build-id 1ec76acd797515b691547164b53205305f8417b4
                Module libgstapp-1.0.so.0 with build-id c4c5bec188619a8cc7e3f33db181c9dbfacac784
                Stack trace of thread 28987:
                #0  0x000000000000000a n/a (n/a + 0x0)
                #1  0x0000000000000021 n/a (n/a + 0x0)
                ELF object binary architecture: AMD x86-64

[kflak]

I managed to get the videoPlayerExample program to compile and run with gstreamer 0.10, which is helpfully packaged in the Arch Linux AUR repository. I installed gstreamer0.10 and gstreamer0.10-{bad,base,base-plugins,good}, and changed $OF_ROOT/libs/openFrameworksCompiled/project/makefileCommon/config.linux.common.mk like this:

#check gstreamer version
# ifndef GST_VERSION
# 	ifeq ($(CROSS_COMPILING),1)
# 		ifeq ($(shell export PKG_CONFIG_LIBDIR=$(PKG_CONFIG_LIBDIR);pkg-config gstreamer-1.0 --exists; echo $$?),0)
# 			GST_VERSION = 1.0
# 			ifdef MAKEFILE_DEBUG
#                 $(info GSTVERSION=$(GST_VERSION))
# 			endif
# 		else
# 			GST_VERSION = 0.10
# 			ifdef MAKEFILE_DEBUG
#                 $(info GSTVERSION=$(GST_VERSION))
#                 $(info $(shell export PKG_CONFIG_LIBDIR=$(PKG_CONFIG_LIBDIR);pkg-config gstreamer-1.0 --exists; echo $$?))
# 			endif
# 		endif
# 	else
# 		ifeq ($(shell pkg-config gstreamer-1.0 --exists; echo $$?),0)
# 			GST_VERSION = 1.0
# 		else
# 			GST_VERSION = 0.10
# 		endif
# 	endif
# endif
GST_VERSION = 0.10

lines 42-63.

The problem remains, however, that the video still doesn't play back, and I get the error

[ error ] ofPixels: format doesn't support channels
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration

The videoGrabberExample compiles as well, but segfaults immediately with:

[warning] ofGstVideoGrabber: listDevices(): only supported for gstreamer 1.0
[notice ] ofGstVideoGrabber: Probing devices with udev...
[notice ] ofGstVideoGrabber: Found device 04f2:b604, getting capabilities...
[notice ] ofGstVideoGrabber: detected v4l2 device: Integrated Camera: Integrated C
[notice ] ofGstVideoGrabber: driver: uvcvideo, version: 331560
[notice ] ofGstVideoGrabber: Capabilities: 0x84A00001
[notice ] ofGstVideoGrabber: Found device 04f2:b604, getting capabilities...
[notice ] ofGstVideoGrabber: detected v4l2 device: Integrated Camera: Integrated C
[notice ] ofGstVideoGrabber: driver: uvcvideo, version: 331560
[notice ] ofGstVideoGrabber: Capabilities: 0x84A00001
[notice ] device /dev/video1 seems to not have the capture capability, (radio tuner/metadata device?)
[notice ] removing it from device list
[ error ] ofGstUtils: error getting device data: no element "v4l2src"
zsh: segmentation fault (core dumped)  ../videoGrabberExample/bin/videoGrabberExample

Which is clear enough... no support for fancy video grabbing in 0.10.

So, that's the state of play right now. Will keep looking if it's possible to point the makefile to 0.18 instead.

[StaffanMelin]

Thank you for your efforts @kflak! I would like to contribute, but this is beyond me at this point. It is annoying because for the next step of my project (https://forum.openframeworks.cc/t/dstudio-make-music-with-openframeworks-v1-0-released) I would really like to integrate video.

[edovino]

sudo ./scripts/linux/fedora/install_codecs.sh ( this is relative to the OF root )
I wonder if missing the codecs is part of the issue?

Everything seems to be complete - I compared the install_codecs.sh scripts from a couple of different linux distros (just to rule out if Fedora had magically repackaged some stuff), but nothing jumped out. (If anything, the Fedora scripts install an extra package: ffmpeg & gstreamer-ffmpeg)

If that doesn't resolve it I guess I'm installing Fedora on my Win/Ubuntu/Mint system laughing

Wot, it's only triple-boot right now? 😁
I'll try Ubuntu as well & see what happens on my laptop.

I've attached two logs: one using playbin (without video), and one using playbin3 (with video)
playbin.log.gz
playbin3.log.gz
(I'm afraid I know next to nothing about GStreamer)

[ofTheo]

Thanks @edovino - that is helpful.
This line in the log points to the issue: could not transform video/x-raw(memory:VASurface), format=(string)NV12...

Basically the vaapipostproc isn't able to go from the GPU VASurface in NV12 format to our requested: x-raw RGB format. ( I think RGBx might work instead, but that would require more core changes as it would result in 4 channels ).

On my laptop it uses the same NV12 but can decode to RGB - so it doesn't seem to be an issue with the format, but I think the decoder is not connecting or inserting the convertors needed. I guess playbin3 which uses decodebin3 is doing that conversion in your case.

I think for now I updated the PR to use playbin3 as that seems to work for all platforms.

I'll try and get F36 installed next week and see if I can narrow it down further, but will merge the PR in the meantime so at least the nightly builds work for Linux users.

Thanks for all the help!!

[ofTheo]

@edovino copying from the forum:

I just installed Fedora 36 and got similar warnings.

But one I got was:
module uridecodebin0 reported: Your GStreamer installation is missing a plug-in

So I looked into it and it seems that not all the gstreamer plugins are getting installed in Fedora.

To fix that I installed the equivalent of Ubuntu restricted via: Configuration - RPM Fusion

And then once it was installed ( via software manager ).
I ran:

sudo dnf install @multimedia

and it installed the codecs the video needed.

Doing this makes the videoPlayerExample work without playbin3.

Def some extra steps, but curious if that works for you too?

[edovino]

Hmm - what packages does sudo dnf install @multimedia install for you? I already had RPMFusion enabled (it's part of the install_codecs.sh script for Fedora).
I ask because the install command doesn't add anything on my system.

[ofTheo]

@edovino huh - weird.
Someone on the forum said the same thing.

For me I did:

• a clean install of F36
• installed codecs and dependencies
• then dnf update as I was having a freeimage linker error which that fixed.
• then got the issue of the GStreamer installation is missing a plug-in so did the steps in the last comment.

I can't remember exactly what it installed but it did add an H264 package that wasn't installed.

Anyway, good to know that was unrelated to the issues you were having.

I left playbin3 in the PR ( which seems to work for everyone ) and its now merged, so I think the nightly builds are now hopefully fixed for Linux / 1.20.3.

Thanks again for testing this all!

[f00b455]

The videoGrabberExample seems to work on my ubuntu studio 22.04.1 LTS.
But the videoPlayerExample still seems to have issues.

When i press "f" to stop and use the arrow keys, the example freezes with following output:

[ error ] ofGstUtils: gstHandleMessage(): embedded video playback halted for plugin, module vaapidecode0 reported: No valid frames decoded before end of stream
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration

on:

commit bd40423 (HEAD -> master, origin/master, origin/HEAD)
Author: Theodore Watson theo@openframeworks.cc
Date: Mon Aug 29 10:44:49 2022 -0700

WIP: Fixes for gstreamer 1.20 broken video playback. closes #6871 (#7050)

#changelog #linux

On start following messages:

➜ videoPlayerExample git:(master) ✗ make RunRelease
[notice ] ofGstUtils: Got context from element 'vaapipostproc0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"(GstGLDisplayWayland)\ gldisplaywayland0";

[notice ] ofGstUtils: Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"(GstVaapiDisplayEGL)\ vaapidisplayegl0", gst.vaapi.Display.GObject=(GstObject)"(GstVaapiDisplayEGL)\ vaapidisplayegl0";

[ofTheo]

Thanks @f00b455 for checking!

Just to note the current 0.11.2 with earlier GStreamer versions don't really work for frame by frame advancing either.
On Linux Mint and GStreamer 1.18 I can go one frame forward, but then it doesn't advance further.

I don't get the error message you do, but it still doesn't work and I think we could open some issues to address this.

[edovino]

@edovino huh - weird. Someone on the forum said the same thing.
* a clean install of F36

What desktop did you install? I'm now wondering if this could be a Wayland vs Xorg issue. I'm going to experiment a bit more.
Anyway, thanks for this fix!

[kflak]

I am getting exactly the same behavior on sway (wayland) and i3 (xorg). The player works (YEEAAAAH!), but I get this message when moving the mouse over the app window: (videoPlayerExample:122253): GStreamer-CRITICAL **: 17:28:27.688: gst_sample_set_caps: assertion 'gst_sample_is_writable (sample)' failed After pressing `f` and arrow key, I am able to advance one frame. Pressing `f` again restarts the playback, with the accompanying error code.

-- Roosna & Flak - Contemporary Dance & Music Web: roosnaflak.com Code: {github,gitlab}.com/kflak Mastodon: @***@***.***

On 31 Aug 2022 23:54, Edwin van Ouwerkerk Moria wrote: @edovino huh - weird. Someone on the forum said the same thing. * a clean install of F36 What desktop did you install? I'm now wondering if this could be a Wayland vs Xorg issue. I'm going to experiment a bit more. Anyway, thanks for this fix! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.*Message ID: <openframeworks/

***@***.***>

…

[kflak]

Oh, and forgot to mention this is on Arch...

[ofTheo]

@kflak - I see those same errors messages on Ubuntu 22 and macOS.
At least its consistent -🙂

I think both of these will be addressable with some tweaks to the seeking code.
I opened an issue for it #7063

[dnabanita7]

Thanks @ofTheo for the updates on this! I am getting few issues after changing playbin to playbin3. But it gives segmentation fault.

[notice ] ofGstUtils: Got context from element 'vaapipostproc0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";

[notice ] ofGstUtils: Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0", gst.vaapi.Display.GObject=(GstObject)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0";

Segmentation fault (core dumped)

If I don't change it to playbin3 in ofGstVideoPlayer.cpp, I am getting this exact same error. I am on Jammy Jellyfish version.

[dnabanita7]

Also, same case with webcam

[ofTheo]

@dnabanita7 - I just sent a reply on the forum, but the playbin3 change alone isn't everything needed.
If you try the nightly builds at the bottom of the download page it should work now - the relevant PR is #7050 ( with most of the changes in ofGSTUtils.cpp ) if you wanted to update both the files from there.

Hope that works for you!

[dnabanita7]

Thank you! It works perfectly fine for me

…

On Mon, Sep 5, 2022 at 9:32 PM Theodore Watson ***@***.***> wrote: @dnabanita7 <https://github.com/dnabanita7> - I just sent a reply on the forum, but the playbin3 change alone isn't everything needed. If you try the nightly builds at the bottom of the download page <https://openframeworks.cc/download/> it should work now - the relevant PR is #7050 <#7050> ( with most of the changes in ofGSTUtils.cpp ) if you wanted to update both the files from there. Hope that works for you! — Reply to this email directly, view it on GitHub <#6871 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHQZX566Z5YEKZ35AJHDHPDV4YKTDANCNFSM5OA6R2ZA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- *nabanita*

[dnabanita7]

Although, webcam works fine. The video is not played. I get an error

[ error ] ofGstUtils: startPipeline(): unable to pause pipeline

and it is not loading up the GstVideoUtils.cpp file.

[ofTheo]

Hi! Can you share more details? Do you have issues with the examples/video/videoPlayerExample or examples/computer_vision/openCVExample ? ( in the night download ) And did you run install_codecs.sh in the scripts/Linux/Ubuntu folder?

[kflak]

Unfortunately the gstreamer bogey is back in town, at least on my system. The videoGrabber example works fine, but with the videoPlayer I am getting

[ error ] ofGstUtils: startPipeline(): unable to pause pipeline after 5s
[ error ] ofGstUtils: gstHandleMessage(): embedded video playback halted for plugin, module parsebin0  reported: Your GStreamer installation is missing a plug-in.
[warning] ofGstVideoUtils: update(): ofGstVideoUtils not loaded
[ error ] ofPixels: format doesn't support channels
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration
[warning] ofGstUtils: getDurationNanos(): couldn't query time duration
[warning] ofGstUtils: setSpeed(): unable to change speed

So the big question is which plugin I am missing here... I have run the install_dependencies script in the archlinux script folder, and I verified that all gst-plugins exist on my system.

The openCV example gives me the same error.

Additional information: I created a very minimal video playback app, and the sound from the video is played back just fine.

void ofApp::setup(){
    video.load( "/home/kf/Videos/SingularityTrailers/SingularityTurningTrailer.mp4");
    video.play();
}
void ofApp::update(){
    video.update();
}

//--------------------------------------------------------------
void ofApp::draw(){

    video.draw(0, 0, 1920, 1080);
}

This gives me the following error:

[ error ] ofGstVideoPlayer: allocate(): cannot get pipeline caps

[StaffanMelin]

It still works on my Debian 11 Testing, just updated, using the latest nightly of oF.

libgstreamer is now 1.20-3.

I also have
libgstreamer-opencv1.0-0
libgstreamer-plugins-bad1.0-0
libgstreamer-plugins-bad1.0-dev
libgstreamer-plugins-base1.0-0
libgstreamer-plugins-base1.0-dev
all version 1.20.3

[kflak]

It still works on my Debian 11 Testing, just updated, using the latest nightly of oF.

libgstreamer is now 1.20-3.

I also have libgstreamer-opencv1.0-0 libgstreamer-plugins-bad1.0-0 libgstreamer-plugins-bad1.0-dev libgstreamer-plugins-base1.0-0 libgstreamer-plugins-base1.0-dev all version 1.20.3

Aha! Figured out the problem. I needed to install gstreamer-vaapi for the videoPlayer example to work. So it seems this should be included in the archlinux install script. I also noticed that another non-existent package is included in the script, gst-libav-x11. I suppose this has been superceded by gst-libav, but I am not sure about this...

[ofTheo]

@kflak - thanks for this!
Would you want to do a Pull Request for the archlinux install_dependencies scripts?

[kflak]

Sure! Will try to get to it today.

…

On Thu, Oct 20, 2022 at 20:51, Theodore Watson ***@***.***> wrote: ***@***.***(https://github.com/kflak) - thanks for this! Would you want to do a Pull Request for the archlinux install_dependencies scripts? — Reply to this email directly, [view it on GitHub](#6871 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ADHRJIHVWEGWN7CZWBFI5BTWEGBBDANCNFSM5OA6R2ZA). You are receiving this because you were mentioned.Message ID: ***@***.***>