Unable to play video using glimagesink on rpi3 with zeus image

[xouillet]

Hi,

I've been trying to generate a weston image for raspberrypi3 with vc4graphics stack. I'm able to play a video via gst-play-1.0 using waylandsink (and even kmssink with weston not started) but glimagesink always throw me this error:

root@raspberrypi3:~# gst-play-1.0 --videosink 'glimagesink' poissons.mp4 
Press 'k' to see a list of keyboard shortcuts.
Now playing /home/root/poissons.mp4
0:00:00.0 / 0:00:24.6       
** (gst-play-1.0:815): CRITICAL **: 19:43:11.395: gst_gl_memory_setup_buffer: assertion '!wrapped_data || n_mem * views == n_wrapped_pointers' failed
Segmentation fault

The same errors appears in cog when trying to access a page containing video.

Yocto build configuration:

Build Configuration:
BB_VERSION           = "1.44.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "arm-poky-linux-gnueabi"
MACHINE              = "raspberrypi3"
DISTRO               = "poky"
DISTRO_VERSION       = "3.0.1"
TUNE_FEATURES        = "arm vfp cortexa7 neon vfpv4 thumb callconvention-hard"
TARGET_FPU           = "hard"
meta                 
meta-poky            
meta-yocto-bsp       = "zeus:cf92a2d567260b91a259652bad0ecd790750f710"
meta-oe              
meta-multimedia      
meta-python          
meta-networking      
meta-filesystems     = "zeus:e855ecc6d35677e79780adc57b2552213c995731"
meta-raspberrypi     = "zeus:85b75588fdedf72242870628f7c22f034d308740"
meta-webkit          = "master:f81f28436f6111a284d3471cb5969dd2fd8ac4e1"

I've been trying to build a simple core-image-weston and still hit the bug. Has anyone experienced this ?

[philn]

Which version of GStreamer?

[philn]

https://github.com/Igalia/meta-webkit/wiki/Providing-useful-GStreamer-Zero-copy-issue-reports

[xouillet]

Oops, sorry

root@raspberrypi3:~# gst-inspect-1.0 --version
gst-inspect-1.0 version 1.16.1
GStreamer 1.16.1
Unknown package origin

The gst plugins are the default ones for wpewebkit with gst_gl support, so the -base, -good, -ugly and -bad. I've not included the -omx as I'm trying to run over vc4graphics open source stack

[xouillet]

Content of /tmp/gst.log:

0:00:00.458207448  1518  0x1471dc0 FIXME                    bin gstbin.c:4341:gst_bin_query: implement duration caching in GstBin again
0:00:01.045229948  1518 0x73203060 WARN                 qtdemux qtdemux_types.c:239:qtdemux_type_get: unknown QuickTime node type pasp
0:00:01.045491979  1518 0x73203060 WARN                 qtdemux qtdemux.c:3237:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:01.227597760  1518 0x711496f0 WARN                    v4l2 gstv4l2object.c:4194:gst_v4l2_object_probe_caps:<v4l2h264dec0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
0:00:01.832820885  1518 0x711496f0 WARN            v4l2videodec gstv4l2videodec.c:810:gst_v4l2_video_dec_decide_allocation:<v4l2h264dec0> Duration invalid, not setting latency
0:00:02.144399947  1518  0x14a16c0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1275:gst_v4l2_buffer_pool_dqbuf:<v4l2h264dec0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:02.339462760  1518  0x1471490 ERROR                gldebug gstgldebug.c:306:_gst_gl_debug_callback:<glcontextegl0> high: GL error from API id:1, GL_INVALID_VALUE in glTexImage2D(internalFormat=GL_R8)
0:00:02.346662030  1518  0x1471490 ERROR                gldebug gstgldebug.c:306:_gst_gl_debug_callback:<glcontextegl0> high: GL error from API id:1, GL_INVALID_VALUE in glTexImage2D(internalFormat=GL_R8)
0:00:02.346901510  1518  0x1471490 ERROR                gldebug gstgldebug.c:306:_gst_gl_debug_callback:<glcontextegl0> high: GL error from API id:1, GL_INVALID_VALUE in glTexImage2D(internalFormat=GL_R8)```

[philn]

And your kernel / Mesa versions?
I think this driver requires recent versions (latest stable) of those components.

[philn]

In any case as this is not (yet) related with wpewebkit, the issue could be reported in the OpenEmbedded tracker.

[xouillet]

I'm using zeus branch of oe. The kernel is 4.19.75 and mesa 19.1.6.

Ok for reporting it on OpenEmbedded

[eocanha]

I had a similar gst_gl_memory_setup_buffer: assertion '!wrapped_data || n_mem * views == n_wrapped_pointers' failed assertion issue while trying to make the V4L2 H264 video decoder on the Raspberry Pi 3.

I first debugged it using custom gst-launch-1.0 pipelines until I found the proper setting I had to force to get things working. In my case, I had to force the videosink caps to be compatible with what glimagesink supports.

This was the pipeline that worked for me to play an mp4 (h264 + aac) video (ignore the "cat" and "fdsrc" weirdness, it's just to force the pipeline in push mode, line MSE on WPE does):

   cat video.mp4 | gst-launch-1.0 -v fdsrc fd=0 \
    ! parsebin name=p ! multiqueue name=q \
      p.src_0 ! q.sink_0 \
      p.src_1 ! q.sink_1 \
      q.src_0 ! v4l2h264dec capture-io-mode=4 \
    ! 'video/x-raw, format=(string)RGB16' \
    ! glimagesink \
      q.src_1 ! queue2 ! decodebin ! audioconvert ! autoaudiosink

The key for me here was to use gst-inspect-1.0 to check what video formats were supported by the video decoder (v4l2h264dec in my case, it might be different on your hardware platform) and by glimagesink. Then I just forced compatible caps (RGB16 for me) by using the implicit capsfilter.

Once I got that working in gst-launch, I wrote an experimental patch to do the same on WPE side. This is for the (downstream) WPE version I'm using, so there's no guarantee that it would apply cleanly in your version, but it can give you hints on how to fix it on your side:

WebPlatformForEmbedded/WPEWebKit@c525cb6

I hope it helps!

[philn]

I think the OE kernel differs from the one used by the RPi Foundation folks, iirc they (use to?) heavily patch their kernel.

[xouillet]

@eocanha Thank you very much ! I haven't tested your wpewebkit patch yet, but I can now play the video with gst-launch !

[xouillet]

The patch works great on 2.26 with some adaptations, thank you again. Closing the issue.

[clopez]

Great to hear that it worked.
I added on the WiKi for the RPi a link to this issue for future reference.

@eocanha I think it would be a good idea to try to upstream that patch :)

[philn]

RGB16 seems to be an odd format to negotiate... I'd like to have this further investigated with RGBA at some point.

[eocanha]

First you would have to persuade v4l2h264dec to accept RGBA as a valid format in its src pad. From gst-inspect-1.0 v4l2h264dec it looks like it's not among the available formats:

  SRC template: 'src'
    Capabilities:
      video/x-raw
                 format: { (string)I420, (string)YV12, (string)NV12, (string)NV21, (string)RGB16 }

I think I got something working by using v4l2videoconvert some months ago, but it decreased the performance and ruined zero-copy, so direct plugging using RGB16 looked like a much better option back then.

[philn]

Ok, I think I420 could be a good candidate. Let's follow-up elsewhere, this issue is closed now :)