intel-vaapi-driver fails to load in a Gnome Shell Wayland session

[vanvugt]

The Intel i965 VA-API driver works well in Xorg sessions (Unity7 and Gnome Shell). But it refuses to load when in a Wayland session:

$ totem bbb_sunflower_1080p_60fps_normal.mp4
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)

$ env LIBVA_DRIVER_NAME=i965 totem bbb_sunflower_1080p_60fps_normal.mp4
totem: intel_driver.c:112: intel_driver_init: Assertion `VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_DRI1) || VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_DRI2) || VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_CUSTOM)' failed.
Aborted (core dumped)

$ gst-play-1.0 bbb_sunflower_1080p_60fps_normal.mp4
Press 'k' to see a list of keyboard shortcuts.
Now playing /home/dan/Videos/bbb_sunflower_1080p_60fps_normal.mp4
Redistribute latency...
Redistribute latency...
ERROR Internal error: could not render surface for file:///home/dan/Videos/bbb_sunflower_1080p_60fps_normal.mp4
ERROR debug information: ../../../gst/vaapi/gstvaapisink.c(1482): gst_vaapisink_show_frame_unlocked (): /GstPlayBin:playbin/GstPlaySink:playsink/GstBin:vbin/GstVaapiSink:vaapisink0

See also: https://bugs.launchpad.net/bugs/1698287

[xhaihao]

totem drops write permission before accessing device file, please refer to https://github.com/01org/libva/issues/53. As for gst-play, it seems this is an issue in gst-vaapi.

[montvid]

try mpv player it does not use gst-vaapi. On my ubuntu 17.04 gst-vaapi lags mpv does not.

[xhaihao]

Thanks @montvid , I closed this bug as 'not our bug'. @vanvugt Please file a bug on http://bugzilla.gnome.org to track your gst-vaapi issue.

[vanvugt]

mpv does not use VAAPI by default. It defaults to high-CPU software decoding (50% CPU). If you force it to use VAAPI (mpv --hwdec vaapi) then performance is excellent (2% CPU). However the same issue remains -- intel-vaapi-driver is only willing to work in a Xorg session but fails in a Wayland session.

This proves that VAAPI works well with ffmpeg (gst-vaapi is not in use). And that the only common factor in this bug is intel-vaapi-driver. The problem is not Gstreamer...

Output from a Gnome-on-Wayland session:

$ mpv --hwdec vaapi bbb_sunflower_1080p_60fps_normal.mp4
Playing: vaapi
[file] Cannot open file 'vaapi': No such file or directory
Failed to open vaapi.

Playing: bbb_sunflower_1080p_60fps_normal.mp4
 (+) Video --vid=1 (*) (h264 1920x1080 60.000fps)
 (+) Audio --aid=1 --alang=und (*) (mp3 2ch 48000Hz)
     Audio --aid=2 --alang=und (*) (ac3 6ch 48000Hz)
File tags:
 Artist: Blender Foundation 2008, Janus Bager Kristensen 2013
 Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
 Composer: Sacha Goedegebure
 Genre: Animation
 Title: Big Buck Bunny, Sunflower version
[vaapi] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[vaapi] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
AO: [pulse] 48000Hz stereo 2ch s16
Using hardware decoding (vaapi-copy).
VO: [opengl] 1920x1080 nv12
AV: 00:00:00 / 00:10:34 (0%) A-V:  0.000


$ env LIBVA_DRIVER_NAME=i965 mpv --hwdec vaapi bbb_sunflower_1080p_60fps_normal.mp4                
Playing: vaapi
[file] Cannot open file 'vaapi': No such file or directory
Failed to open vaapi.

Playing: bbb_sunflower_1080p_60fps_normal.mp4
 (+) Video --vid=1 (*) (h264 1920x1080 60.000fps)
 (+) Audio --aid=1 --alang=und (*) (mp3 2ch 48000Hz)
     Audio --aid=2 --alang=und (*) (ac3 6ch 48000Hz)
File tags:
 Artist: Blender Foundation 2008, Janus Bager Kristensen 2013
 Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
 Composer: Sacha Goedegebure
 Genre: Animation
 Title: Big Buck Bunny, Sunflower version
mpv: intel_driver.c:112: intel_driver_init: Assertion `VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_DRI1) || VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_DRI2) || VA_CHECK_DRM_AUTH_TYPE(ctx, VA_DRM_AUTH_CUSTOM)' failed.
Aborted (core dumped)

[vanvugt]

Please reopen this bug. It has been shown to not be an issue that is specific to GStreamer, but occurs with FFmpeg hwaccel too.

Put another way, both GStreamer and FFmpeg playback crashes in the same place inside intel-vaapi-driver when you're using Wayland.

[vanvugt]

Ah, mpv actually seems to reimplement VA-API support independently of ffmpeg's hwaccel architecture. However the important point remains that it's also not using GStreamer. So whatever this bug is, it exists in mpv too...

[xhaihao]

The assertion failure was fixed on master branch weeks ago. Does your wayland compositor support wl_drm interface? You can run weston-info to check whether wl_drm is used if you is using weston.

$> weston-info | grep drm

Note: the assertion failure means the player fails to create a workable connection to the native display server for VA-API, you should check your environment.

[xhaihao]

mpv works for me under Wayland.

$ mpv --hwdec vaapi 720p.mp4
Compiled without libass.
There will be no OSD and no text subtitles.
Playing: vaapi
[file] Cannot open file 'vaapi': No such file or directory
Failed to open vaapi.

Playing: 720p.mp4
(+) Video --vid=1 (*) (h264 1280x720 29.970fps)
Using hardware decoding (vaapi).
VO: [opengl] 1280x720 vaapi[nv12]

[xhaihao]

gst-play-1.0 works too, the cpu usage is about 2~3.

$ gst-play-1.0 720p.mp4
Press 'k' to see a list of keyboard shortcuts.
Now playing /home/xhh/720p.mp4
Redistribute latency...
0:00:09.7 / 0:05:29.7

[vanvugt]

I'm using Gnome Shell on Wayland....

$ weston-info
interface: 'wl_drm', version: 2, name: 1
interface: 'wl_compositor', version: 3, name: 2
interface: 'wl_shm', version: 1, name: 3
	formats: XRGB8888 ARGB8888
interface: 'wl_output', version: 2, name: 4
	x: 0, y: 0, scale: 1,
	physical_width: 520 mm, physical_height: 320 mm,
	make: 'DEL', model: 'DELL U2413',
	subpixel_orientation: unknown, output_transform: normal,
	mode:
		width: 1920 px, height: 1200 px, refresh: 59.950 Hz,
		flags: current preferred
interface: 'wl_data_device_manager', version: 3, name: 5
interface: 'gtk_primary_selection_device_manager', version: 1, name: 6
interface: 'zxdg_shell_v6', version: 1, name: 7
interface: 'wl_shell', version: 1, name: 8
interface: 'gtk_shell1', version: 1, name: 9
interface: 'wl_subcompositor', version: 1, name: 10
interface: 'zwp_pointer_gestures_v1', version: 1, name: 11
interface: 'zwp_tablet_manager_v2', version: 1, name: 12
interface: 'wl_seat', version: 5, name: 13
	name: seat0
	capabilities: pointer keyboard
	keyboard repeat rate: 33
	keyboard repeat delay: 500
interface: 'zwp_relative_pointer_manager_v1', version: 1, name: 14
interface: 'zwp_pointer_constraints_v1', version: 1, name: 15
interface: 'zxdg_exporter_v1', version: 1, name: 16
interface: 'zxdg_importer_v1', version: 1, name: 17

but still:

$ mpv --hwdec=vaapi bbb_sunflower_1080p_60fps_normal.mp4
Playing: bbb_sunflower_1080p_60fps_normal.mp4
 (+) Video --vid=1 (*) (h264 1920x1080 60.000fps)
 (+) Audio --aid=1 --alang=und (*) (mp3 2ch 48000Hz)
     Audio --aid=2 --alang=und (*) (ac3 6ch 48000Hz)
File tags:
 Artist: Blender Foundation 2008, Janus Bager Kristensen 2013
 Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
 Composer: Sacha Goedegebure
 Genre: Animation
 Title: Big Buck Bunny, Sunflower version
[vaapi] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
[vaapi] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
VO does not support requested hardware decoder, or loading it failed.
AO: [pulse] 48000Hz stereo 2ch s16
VO: [opengl] 1920x1080 yuv420p
AV: 00:00:00 / 00:10:34 (0%) A-V:  0.000

It only works in a Xorg session still.

[vanvugt]

Perhaps weston-info isn't actually getting that data from Gnome Shell, but the weston binaries (which we're not using).

[vanvugt]

Yes, I can confirm Weston works fine. This bug seems to be Gnome Shell on Wayland only.

[vanvugt]

OK, we have multiple bugs here.

mpv can be made to work in a Gnome Shell Wayland session if you unset $DISPLAY (which is always set for legacy apps to use Xwayland). However that does not fix gst-play-1.0 or totem. Seems https://github.com/01org/libva/issues/53 is still a separate bug for totem. And there might be a third bug for gst-play-1.0.

[montvid]

For wayland support in mpv use --opengl-backend=wayland it works with latest software for sure. Also check https://wiki.ubuntu.com/IntelQuickSyncVideo Good luck.

[vanvugt]

Thanks. That seems to further confirm the problem is incorrect detection of Wayland vs X. This works in Gnome Shell on Wayland: mpv --hwdec --opengl-backend=wayland

I don't really need to check out that wiki page though, because I wrote it :)