Implement wlr_export_dmabuf_unstable_v1 protocol

[emersion]

This PR adds support for the wlr_export_dmabuf_unstable_v1 protocol.

• Add dma-buf support to backends
  • On DRM via GBM
  • On other backends via the GL extension (see #906) not possible, that would require to render to a texture
• Support capturing clients
• Add byte buffer support to DRM
• Implement the protocol
  • Basic functionality
  • Frame synchronization
  • Timestamps
  • Frame locking
• Add example client (cc @atomnuker)

Fixes #709

[emersion]

All right, this PR is ready for review.

(Please don't comment the protocol here)

[emersion]

All right, this PR is ready for review.

(Please don't comment the protocol here)

[agx]

Looks good to me codewise.

[agx]

Looks good to me codewise.

[atomnuker]

Ping? This should be ready for merging.

[atomnuker]

Ping? This should be ready for merging.

[SirCmpwn]

How am I supposed to use this?

~/s/w/build > ./examples/dmabuf-capture 23 vaapi /dev/dri/renderD128 libx264 nv12 12 dmabuf_recording_01.mkv
Capturable output: Dell Inc. Model: DELL P2417H: ID: 21
Capturable output: Goldstar Company Ltd Model: EW224: ID: 22
Capturable output: Goldstar Company Ltd Model: LG Ultra HD: ID: 23
Capturable output: Ancor Communications Inc Model: ASUS VS228: ID: 24
[AVHWDeviceContext @ 0x55e7217dd480] Failed to initialise VAAPI connection: -1 (unknown libva error).
[dmabuf-capture @ 0x7fff63d5cc90] Failed to create a hardware device: Input/output error

[SirCmpwn]

How am I supposed to use this?

~/s/w/build > ./examples/dmabuf-capture 23 vaapi /dev/dri/renderD128 libx264 nv12 12 dmabuf_recording_01.mkv
Capturable output: Dell Inc. Model: DELL P2417H: ID: 21
Capturable output: Goldstar Company Ltd Model: EW224: ID: 22
Capturable output: Goldstar Company Ltd Model: LG Ultra HD: ID: 23
Capturable output: Ancor Communications Inc Model: ASUS VS228: ID: 24
[AVHWDeviceContext @ 0x55e7217dd480] Failed to initialise VAAPI connection: -1 (unknown libva error).
[dmabuf-capture @ 0x7fff63d5cc90] Failed to create a hardware device: Input/output error

[SirCmpwn]

Troubleshooted here and got further https://wiki.archlinux.org/index.php/Hardware_video_acceleration

~/s/w/build > ./examples/dmabuf-capture 24 vaapi /dev/dri/renderD128 libx264 nv12 12 dmabuf_recording_01.mkv
Capturable output: Dell Inc. Model: DELL P2417H: ID: 21
Capturable output: Goldstar Company Ltd Model: EW224: ID: 22
Capturable output: Goldstar Company Ltd Model: LG Ultra HD: ID: 23
Capturable output: Ancor Communications Inc Model: ASUS VS228: ID: 24
[libx264 @ 0x55cb4fbfe080] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x55cb4fbfe080] profile High, level 6.1
[libx264 @ 0x55cb4fbfe080] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=abr mbtree=1 bitrate=12000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'dmabuf_recording_01.mkv':
    Stream #0:0: Video: h264, nv12, 1920x1080, q=2-31, 12000 kb/s, 60 fps, 1k tbn
[AVHWFramesContext @ 0x55cb4fc01c00] Failed to read image from surface 0x2: 1 (operation failed).
[libx264 @ 0x55cb4fbfe080] AVFrame.format is not set
[libx264 @ 0x55cb4fbfe080] AVFrame.width or height is not set
[libx264 @ 0x55cb4fbfe080] Input picture width (1920) is greater than stride (0)
[dmabuf-capture @ 0x7ffe8d836ec0] Error encoding: Generic error in an external library!
[dmabuf-capture @ 0x7ffe8d836ec0] Wrote trailer!
[libx264 @ 0x55cb4fbfe080] final ratefactor: 42.32

vainfo

libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.0)
vainfo: Driver version: Mesa Gallium driver 18.1.1 for Radeon RX 580 Series (POLARIS10, DRM 3.25.0, 4.17.2-1-ARCH, LLVM 6.0.0)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

[SirCmpwn]

Troubleshooted here and got further https://wiki.archlinux.org/index.php/Hardware_video_acceleration

~/s/w/build > ./examples/dmabuf-capture 24 vaapi /dev/dri/renderD128 libx264 nv12 12 dmabuf_recording_01.mkv
Capturable output: Dell Inc. Model: DELL P2417H: ID: 21
Capturable output: Goldstar Company Ltd Model: EW224: ID: 22
Capturable output: Goldstar Company Ltd Model: LG Ultra HD: ID: 23
Capturable output: Ancor Communications Inc Model: ASUS VS228: ID: 24
[libx264 @ 0x55cb4fbfe080] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x55cb4fbfe080] profile High, level 6.1
[libx264 @ 0x55cb4fbfe080] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=abr mbtree=1 bitrate=12000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'dmabuf_recording_01.mkv':
    Stream #0:0: Video: h264, nv12, 1920x1080, q=2-31, 12000 kb/s, 60 fps, 1k tbn
[AVHWFramesContext @ 0x55cb4fc01c00] Failed to read image from surface 0x2: 1 (operation failed).
[libx264 @ 0x55cb4fbfe080] AVFrame.format is not set
[libx264 @ 0x55cb4fbfe080] AVFrame.width or height is not set
[libx264 @ 0x55cb4fbfe080] Input picture width (1920) is greater than stride (0)
[dmabuf-capture @ 0x7ffe8d836ec0] Error encoding: Generic error in an external library!
[dmabuf-capture @ 0x7ffe8d836ec0] Wrote trailer!
[libx264 @ 0x55cb4fbfe080] final ratefactor: 42.32

vainfo

libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.0)
vainfo: Driver version: Mesa Gallium driver 18.1.1 for Radeon RX 580 Series (POLARIS10, DRM 3.25.0, 4.17.2-1-ARCH, LLVM 6.0.0)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

[atomnuker]

Either your card doesn't support it, your installation's borked, there's some external libva patches you need, or something I don't know about. The problem's on your end somewhere.

[atomnuker]

Either your card doesn't support it, your installation's borked, there's some external libva patches you need, or something I don't know about. The problem's on your end somewhere.

[SirCmpwn]

Well, I can't test this until I can narrow it further. Do you know of some good troubleshooting resources? Other vaapi applications which are less experimental I can try?

[SirCmpwn]

Well, I can't test this until I can narrow it further. Do you know of some good troubleshooting resources? Other vaapi applications which are less experimental I can try?

[atomnuker]

Try ./build/examples/dmabuf-capture 24 opencl 0 huffyuv bgra 12 dmabuf_recording_01.mkv if you've got opencl.
Otherwise, try ffmpeg's kmsgrab:
sudo ffmpeg -init_hw_device "vaapi=vp:/dev/dri/renderD128" -f kmsgrab -device /dev/dri/card0 -crtc_id 0 -i /dev/null -filter_hw_device vp -vf hwmap -c:v h264_vaapi -y output.mkv
Set the crtc_id to whatever display's crtc_id you need (@acrisci's drm_info will print them out for you), as well as the /dev/dri paths to whatever card they're on.
This has been tested to work on pretty much everything from arm boards to multi gpu systems and has been known to be very robust (survives standbys), so if it doesn't work then you probably have some issues.

[atomnuker]

Try ./build/examples/dmabuf-capture 24 opencl 0 huffyuv bgra 12 dmabuf_recording_01.mkv if you've got opencl.
Otherwise, try ffmpeg's kmsgrab:
sudo ffmpeg -init_hw_device "vaapi=vp:/dev/dri/renderD128" -f kmsgrab -device /dev/dri/card0 -crtc_id 0 -i /dev/null -filter_hw_device vp -vf hwmap -c:v h264_vaapi -y output.mkv
Set the crtc_id to whatever display's crtc_id you need (@acrisci's drm_info will print them out for you), as well as the /dev/dri paths to whatever card they're on.
This has been tested to work on pretty much everything from arm boards to multi gpu systems and has been known to be very robust (survives standbys), so if it doesn't work then you probably have some issues.

[agx]

Given the recent comments having some usage / trouble shoot instructions here would IMHO be good

[SirCmpwn]

@acrisci's drm_info will print them out for you

hm, where's this?

[SirCmpwn]

@acrisci's drm_info will print them out for you

hm, where's this?

[atomnuker]

Sorry, I meant @ascent12's drm_info, https://github.com/ascent12/drm_info

[atomnuker]

Sorry, I meant @ascent12's drm_info, https://github.com/ascent12/drm_info

[SirCmpwn]

This looks good to me. Nice work!

This ended up solving my problem:

atomnuker sircmpwn: hmm, I think I might know what's wrong with your vaapi
atomnuker I think amd might not be able to do on-download conversion between bgr0 and nv12
atomnuker try: ./build/examples/dmabuf-capture vaapi /dev/dri/renderD128 libx264rgb bgr0 12 dmabuf_recording_01.mkv

[SirCmpwn]

This looks good to me. Nice work!

This ended up solving my problem:

atomnuker sircmpwn: hmm, I think I might know what's wrong with your vaapi
atomnuker I think amd might not be able to do on-download conversion between bgr0 and nv12
atomnuker try: ./build/examples/dmabuf-capture vaapi /dev/dri/renderD128 libx264rgb bgr0 12 dmabuf_recording_01.mkv

[martinetd]

Shouldn't this be posted to the client instead? (I'm not sure what posting an error to the manager_resource implies)

[martinetd]

Shouldn't this be posted to the client instead? (I'm not sure what posting an error to the manager_resource implies)

[emersion]

iirc there's no big difference between wl_client_post_no_memory and wl_resource_post_no_memory

[martinetd]

Sorry, the difference I'm pointing at is that you post to manager_resource here while below we notify the client which sent us a request. The later sounds better to me.

[emersion]

How so? Reading libwayland's source code, the only difference is that wl_client_post_no_memory is:

wl_resource_post_error(client->display_resource, WL_DISPLAY_ERROR_NO_MEMORY, "no memory");

And wl_resource_post_no_memory is:

wl_resource_post_error(resource->client->display_resource, WL_DISPLAY_ERROR_NO_MEMORY, "no memory");

So I'd argue that wl_resource_post_no_memory is slightly better because it indicates the interface that has caused the out-of-memory error.

[martinetd]

Ok, I wasn't sure what posting error to a non-client resource did but that looks like the final result is the same indeed; thanks for checking.