New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement wlr_export_dmabuf_unstable_v1 protocol #992

Merged
merged 24 commits into from Jun 22, 2018

Conversation

Projects
None yet
6 participants
@emersion
Copy link
Member

emersion commented May 21, 2018

This PR adds support for the wlr_export_dmabuf_unstable_v1 protocol.

  • Add dma-buf support to backends
    • On DRM via GBM
    • On other backends via the GL extension (see #906) not possible, that would require to render to a texture
  • Support capturing clients
  • Add byte buffer support to DRM
  • Implement the protocol
    • Basic functionality
    • Frame synchronization
    • Timestamps
    • Frame locking
  • Add example client (cc @atomnuker)

Fixes #709

@emersion emersion changed the title [WIP] Implement wlr-screencontent protocol [WIP] Implement wlr_export_dmabuf_unstable_v1 protocol May 22, 2018

@emersion emersion force-pushed the emersion:screencontent branch 5 times, most recently from 9a0aded to 45b7035 May 22, 2018

@emersion emersion changed the title [WIP] Implement wlr_export_dmabuf_unstable_v1 protocol Implement wlr_export_dmabuf_unstable_v1 protocol May 28, 2018

emersion and others added some commits May 21, 2018

@emersion emersion force-pushed the emersion:screencontent branch from 013c872 to c844eaa May 29, 2018

@emersion

This comment has been minimized.

Copy link
Member

emersion commented May 29, 2018

All right, this PR is ready for review.

(Please don't comment the protocol here)

@agx

This comment has been minimized.

Copy link
Contributor

agx commented May 30, 2018

Looks good to me codewise.

atomnuker and others added some commits Jun 17, 2018

@atomnuker

This comment has been minimized.

Copy link
Contributor

atomnuker commented Jun 21, 2018

Ping? This should be ready for merging.

@ddevault

This comment has been minimized.

Copy link
Member

ddevault commented Jun 21, 2018

How am I supposed to use this?

~/s/w/build > ./examples/dmabuf-capture 23 vaapi /dev/dri/renderD128 libx264 nv12 12 dmabuf_recording_01.mkv
Capturable output: Dell Inc. Model: DELL P2417H: ID: 21
Capturable output: Goldstar Company Ltd Model: EW224: ID: 22
Capturable output: Goldstar Company Ltd Model: LG Ultra HD: ID: 23
Capturable output: Ancor Communications Inc Model: ASUS VS228: ID: 24
[AVHWDeviceContext @ 0x55e7217dd480] Failed to initialise VAAPI connection: -1 (unknown libva error).
[dmabuf-capture @ 0x7fff63d5cc90] Failed to create a hardware device: Input/output error
@ddevault

This comment has been minimized.

Copy link
Member

ddevault commented Jun 21, 2018

Troubleshooted here and got further https://wiki.archlinux.org/index.php/Hardware_video_acceleration

~/s/w/build > ./examples/dmabuf-capture 24 vaapi /dev/dri/renderD128 libx264 nv12 12 dmabuf_recording_01.mkv
Capturable output: Dell Inc. Model: DELL P2417H: ID: 21
Capturable output: Goldstar Company Ltd Model: EW224: ID: 22
Capturable output: Goldstar Company Ltd Model: LG Ultra HD: ID: 23
Capturable output: Ancor Communications Inc Model: ASUS VS228: ID: 24
[libx264 @ 0x55cb4fbfe080] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x55cb4fbfe080] profile High, level 6.1
[libx264 @ 0x55cb4fbfe080] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=abr mbtree=1 bitrate=12000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'dmabuf_recording_01.mkv':
    Stream #0:0: Video: h264, nv12, 1920x1080, q=2-31, 12000 kb/s, 60 fps, 1k tbn
[AVHWFramesContext @ 0x55cb4fc01c00] Failed to read image from surface 0x2: 1 (operation failed).
[libx264 @ 0x55cb4fbfe080] AVFrame.format is not set
[libx264 @ 0x55cb4fbfe080] AVFrame.width or height is not set
[libx264 @ 0x55cb4fbfe080] Input picture width (1920) is greater than stride (0)
[dmabuf-capture @ 0x7ffe8d836ec0] Error encoding: Generic error in an external library!
[dmabuf-capture @ 0x7ffe8d836ec0] Wrote trailer!
[libx264 @ 0x55cb4fbfe080] final ratefactor: 42.32

vainfo

libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.0)
vainfo: Driver version: Mesa Gallium driver 18.1.1 for Radeon RX 580 Series (POLARIS10, DRM 3.25.0, 4.17.2-1-ARCH, LLVM 6.0.0)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc
@atomnuker

This comment has been minimized.

Copy link
Contributor

atomnuker commented Jun 21, 2018

Either your card doesn't support it, your installation's borked, there's some external libva patches you need, or something I don't know about. The problem's on your end somewhere.

@ddevault

This comment has been minimized.

Copy link
Member

ddevault commented Jun 21, 2018

Well, I can't test this until I can narrow it further. Do you know of some good troubleshooting resources? Other vaapi applications which are less experimental I can try?

@atomnuker

This comment has been minimized.

Copy link
Contributor

atomnuker commented Jun 21, 2018

Try ./build/examples/dmabuf-capture 24 opencl 0 huffyuv bgra 12 dmabuf_recording_01.mkv if you've got opencl.
Otherwise, try ffmpeg's kmsgrab:
sudo ffmpeg -init_hw_device "vaapi=vp:/dev/dri/renderD128" -f kmsgrab -device /dev/dri/card0 -crtc_id 0 -i /dev/null -filter_hw_device vp -vf hwmap -c:v h264_vaapi -y output.mkv
Set the crtc_id to whatever display's crtc_id you need (@acrisci's drm_info will print them out for you), as well as the /dev/dri paths to whatever card they're on.
This has been tested to work on pretty much everything from arm boards to multi gpu systems and has been known to be very robust (survives standbys), so if it doesn't work then you probably have some issues.

@@ -0,0 +1,743 @@
#define _XOPEN_SOURCE 700

This comment has been minimized.

@agx

agx Jun 21, 2018

Contributor

Given the recent comments having some usage / trouble shoot instructions here would IMHO be good

@ddevault

This comment has been minimized.

Copy link
Member

ddevault commented Jun 21, 2018

@acrisci's drm_info will print them out for you

hm, where's this?

@atomnuker

This comment has been minimized.

Copy link
Contributor

atomnuker commented Jun 21, 2018

Sorry, I meant @ascent12's drm_info, https://github.com/ascent12/drm_info

@ddevault

This comment has been minimized.

Copy link
Member

ddevault commented Jun 22, 2018

This looks good to me. Nice work!

This ended up solving my problem:

atomnuker sircmpwn: hmm, I think I might know what's wrong with your vaapi
atomnuker I think amd might not be able to do on-download conversion between bgr0 and nv12
atomnuker try: ./build/examples/dmabuf-capture vaapi /dev/dri/renderD128 libx264rgb bgr0 12 dmabuf_recording_01.mkv

@ddevault ddevault merged commit e459fe0 into swaywm:master Jun 22, 2018

1 check passed

builds.sr.ht builds.sr.ht job completed successfully
Details

@emersion emersion deleted the emersion:screencontent branch Jun 22, 2018

@martinetd

This comment has been minimized.

Copy link
Contributor

martinetd commented on types/wlr_export_dmabuf_v1.c in 36bd479 Jun 30, 2018

Shouldn't this be posted to the client instead? (I'm not sure what posting an error to the manager_resource implies)

This comment has been minimized.

Copy link
Member

emersion replied Jun 30, 2018

iirc there's no big difference between wl_client_post_no_memory and wl_resource_post_no_memory

This comment has been minimized.

Copy link
Contributor

martinetd replied Jun 30, 2018

Sorry, the difference I'm pointing at is that you post to manager_resource here while below we notify the client which sent us a request. The later sounds better to me.

This comment has been minimized.

Copy link
Member

emersion replied Jun 30, 2018

How so? Reading libwayland's source code, the only difference is that wl_client_post_no_memory is:

wl_resource_post_error(client->display_resource, WL_DISPLAY_ERROR_NO_MEMORY, "no memory");

And wl_resource_post_no_memory is:

wl_resource_post_error(resource->client->display_resource, WL_DISPLAY_ERROR_NO_MEMORY, "no memory");

So I'd argue that wl_resource_post_no_memory is slightly better because it indicates the interface that has caused the out-of-memory error.

This comment has been minimized.

Copy link
Contributor

martinetd replied Jun 30, 2018

Ok, I wasn't sure what posting error to a non-client resource did but that looks like the final result is the same indeed; thanks for checking.

@Sunderland93

This comment has been minimized.

Copy link

Sunderland93 commented Jul 13, 2018

Does it works with Pipewire?

@emersion

This comment has been minimized.

Copy link
Member

emersion commented Jul 13, 2018

No, but we have plans similar to KDE's to make Pipewire use this. See https://pars.ee/articles/wayland_screen_cap.html

@emersion

This comment has been minimized.

Copy link
Member

emersion commented Jul 13, 2018

(Ping me on IRC if you're interested inb making Pipewire work in sway)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment