[Wayland] Add mouse grab/lock functionality. #15103

vanfanel · 2023-03-18T15:58:47Z

Description

This implements mouse grab/lock in Wayland, needed for some cores like Dosbox-core to work correctly with mouse-driven games.

This was done with massive help from @ColinKinloch

Related Issues

This long-standing issue is now fixed:
realnc/dosbox-core#48
Mouse was misbehaving in Wayland because of the missing feature this PR adds: mouse grab/lock.

i30817 · 2023-03-18T17:30:44Z

This PR makes my touch pad think it's always dragging in the last direction in scummvm even if i'm not touching it anymore. Driver is sdl2 or udev or x or linuxraw .

It happens already in the main menu (and the games too). I think you have a bit of more work to do.

Edit: although i was using a debug core, to try to debug this same bug with the author, so i'll redownload it and try again.

edit: still happens.

vanfanel · 2023-03-18T18:07:16Z

Ok, don't merge it yet. There's something still going on...

ColinKinloch · 2023-03-18T18:46:02Z

Relative motion is working. But the cursor lock isn't. I'll have a play around and see if I can get it working.

ColinKinloch

This will allow the user to unlock the cursor.

input/drivers/wayland_input.c

input/common/wayland_common.h

ColinKinloch · 2023-03-18T19:29:49Z

Okay, I've figured it out, the main issue is that the void *data field of input_wl_grab_mouse is the retroarch wayland input object not the graphics option, so all the wayland objects passed to the locking function are NULL.

This should make things work better:

static void input_wl_grab_mouse(void *data, bool state)
{
   input_ctx_wayland_data_t *wl = (input_ctx_wayland_data_t*)data;

ColinKinloch · 2023-03-18T19:37:43Z

Also a locked pointer has a listener which can notify retroarch when the pointer has succesfully locked / unlocked.
I think the escape key forces unlock which the unlocked event will notify.

Co-authored-by: Colin Kinloch <colin@kinlo.ch>

vanfanel · 2023-03-18T19:56:36Z

@ColinKinloch This change:


static void input_wl_grab_mouse(void *data, bool state)
{
   input_ctx_wayland_data_t *wl = (input_ctx_wayland_data_t*)data;

leads to some compilation errors:


input/drivers/wayland_input.c: In function 'input_wl_grab_mouse':
input/drivers/wayland_input.c:410:10: error: 'input_ctx_wayland_data_t' {aka 'struct input_ctx_wayland_data'} has no member named 'pointer_constraints'
  410 |    if (wl->pointer_constraints) {
      |          ^~
input/drivers/wayland_input.c:413:12: error: 'input_ctx_wayland_data_t' {aka 'struct input_ctx_wayland_data'} has no member named 'locked_pointer'
  413 |          wl->locked_pointer = zwp_pointer_constraints_v1_lock_pointer(wl->pointer_constraints,
      |            ^~
input/drivers/wayland_input.c:413:73: error: 'input_ctx_wayland_data_t' {aka 'struct input_ctx_wayland_data'} has no member named 'pointer_constraints'
  413 |          wl->locked_pointer = zwp_pointer_constraints_v1_lock_pointer(wl->pointer_constraints,
      |                                                                         ^~
input/drivers/wayland_input.c:414:15: error: 'input_ctx_wayland_data_t' {aka 'struct input_ctx_wayland_data'} has no member named 'surface'
  414 |             wl->surface, wl->wl_pointer, NULL, ZWP_POINTER_CONSTRAINTS_V1_LIFETIME_PERSISTENT);
      |               ^~
input/drivers/wayland_input.c:414:28: error: 'input_ctx_wayland_data_t' {aka 'struct input_ctx_wayland_data'} has no member named 'wl_pointer'
  414 |             wl->surface, wl->wl_pointer, NULL, ZWP_POINTER_CONSTRAINTS_V1_LIFETIME_PERSISTENT);
      |                            ^~
input/drivers/wayland_input.c:418:42: error: 'input_ctx_wayland_data_t' {aka 'struct input_ctx_wayland_data'} has no member named 'locked_pointer'
  418 |          zwp_locked_pointer_v1_destroy(wl->locked_pointer);
      |                                          ^~
input/drivers/wayland_input.c:419:12: error: 'input_ctx_wayland_data_t' {aka 'struct input_ctx_wayland_data'} has no member named 'locked_pointer'
  419 |          wl->locked_pointer = NULL;
      |            ^~

ColinKinloch · 2023-03-18T20:01:50Z

Yup, the input member of the gfx object is a pointer to the input object and vice verca.

ColinKinloch · 2023-03-18T20:07:49Z

I should spend some time making the wayland driver more readable. Not naming two separate objects wl adding member names to the listeners etc.

vanfanel · 2023-03-18T20:42:15Z

Yup, the input member of the gfx object is a pointer to the input object and vice verca.

@ColinKinloch I did the needed changes in this commit:

vanfanel@6feaa60

However, I am still seeing the same pointer drifting / auto-moving.

ColinKinloch · 2023-03-18T21:31:43Z

Okay, so it's successfully locking the pointer and delta_x is being set correctly.
On my end when the pointer is locked it doesn't move.
So in input_wl_poll you want to set wl->mouse.x += wl->mouse.delta_x but only if the pointer is locked.

vanfanel · 2023-03-18T21:41:00Z

Okay, so it's successfully locking the pointer and delta_x is being set correctly. On my end when the pointer is locked it doesn't move. So in input_wl_poll you want to set wl->mouse.x += wl->mouse.delta_x but only if the pointer is locked.

@ColinKinloch And how do I know if the pointer is locked at that point?

ColinKinloch · 2023-03-18T21:47:11Z

You should add a listener with locked and unlocked function using zwp_locked_pointer_v1_add_listener to the locked pointer. On unlocked you should set wl->locked_pointer = NULL.
Then you can test whether the pointer is locked by the existence of the locked_pointer property.

…hen locked.

vanfanel · 2023-03-19T09:54:39Z

@ColinKinloch Ok, did what you said in commit 74913ea

But the cursor is still drifting / auto-moving.

ColinKinloch · 2023-03-19T11:31:27Z

Oh, I see, move the lines from input_wl_poll to handle_relative_motion

vanfanel · 2023-03-19T12:15:39Z

@ColinKinloch Done so in commit f29a351

But the pointer is still auto-moving / drifting.

ColinKinloch · 2023-03-19T12:34:16Z

Hmm, could you take a recording?

vanfanel · 2023-03-19T12:46:48Z

Hmm, could you take a recording?

@ColinKinloch Of course. Here it is:

ishar-retroarch.mp4

As you can see, it auto-moves after I manually move it: as if the pointer had inertia.
It doesn't start auto-moving until I move it.
Maybe something related to acceleration or speed?

i30817 · 2023-03-19T13:22:53Z

In the touchpad here, if i press exactly in the center i can 'stop' the cursor after it starts moving even i wasn't pressing it anymore if i first counteract the movement.

It's really is some kind of inertia.

ColinKinloch · 2023-03-19T15:47:19Z

Ah, I was wrong about the unlocked. Since the lifetime is set to persistent the locked_pointer object is still valid, so we don't want to set locked_pointer to NULL.

ColinKinloch

This change will fix a crash I'm having

input/drivers/wayland_input.c

input/common/wayland_common.c

ColinKinloch · 2023-03-19T16:19:51Z

Since relative_motion only fires when the pointer is moving you need to reset delta_x to zero.
So I suppose do it here:

RetroArch/input/drivers/wayland_input.c

Lines 269 to 272 in 2a36245

    
           case RETRO_DEVICE_ID_MOUSE_X: 
        
              return screen ? wl->mouse.x : wl->mouse.delta_x; 
        
           case RETRO_DEVICE_ID_MOUSE_Y: 
        
              return screen ? wl->mouse.y : wl->mouse.delta_y;

something like

int x = screen ? wl->mouse.x : wl->mouse.delta_x;
wl->mouse.delta_x = 0;
return x;

vanfanel · 2023-03-19T20:46:10Z

@ColinKinloch Ok! Reset these deltas in commit e7ac2af, and everything seems to work well no, no more strange "inertia"!

@i30817 Can you try in your touchpad configuration, please, and tell us if there's something else you notice? :)

i30817 · 2023-03-19T21:26:53Z

Everything working. Even myst 3, and versailles 1865 rotoscaped landscapes are working (these games have 360º viewports that rotate around with the mouse).

ColinKinloch · 2023-03-19T21:43:04Z

From limited testing I see one more thing.
Currently in the RetroArch menu, when the pointer is grabbed, I can lose my cursor off the side of the screen.
The x11 driver has this code to clamp the cursor position:

RetroArch/input/drivers/x11_input.c

Lines 543 to 554 in 3f485de

    
           /* Clamp X */ 
        
           if (x11->mouse_x < 0)  
        
              x11->mouse_x       = 0; 
        
           if (x11->mouse_x >= win_attr.width) 
        
           x11->mouse_x          = (win_attr.width - 1); 
        
           /* Clamp Y */ 
        
           if (x11->mouse_y < 0)  
        
              x11->mouse_y       = 0; 
        
           if (x11->mouse_y >= win_attr.height) 
        
              x11->mouse_y       = (win_attr.height - 1);

ColinKinloch · 2023-03-19T22:13:22Z

Are there any FPSs with mouse look on RetroArch?

vanfanel · 2023-03-19T22:28:58Z

Are there any FPSs with mouse look on RetroArch?

@ColinKinloch I thought TyrQuake would support mouselook, but it doesn't look like it, I have just tried and mouse doesn't seem to work with it.

What would be the best place to do the coordinates clamping as in X11?

ColinKinloch · 2023-03-19T22:30:31Z

I've figured out the FPS thing. You need to set "Port 1 Controls" to mouse and keyboard, works great :)

ColinKinloch · 2023-03-19T22:32:20Z

In the x11 the clamping is done in the poll function when the input isn't grabbed, so I suppose it should be done in input_wl_poll.

vanfanel · 2023-03-19T22:38:01Z

I've figured out the FPS thing. You need to set "Port 1 Controls" to mouse and keyboard, works great :)

Did you do it with TyrQuake? Or did you use another core?

ColinKinloch · 2023-03-19T22:41:54Z

TyrQuake, PrBoom, vitaQuakeII.
Anything with content available via the content downloader :)

ColinKinloch · 2023-03-19T22:47:59Z

You should take a look at video_driver_translate_coord_viewport_wrap and video_driver_translate_coord_viewport used in the winraw and x11 drivers. It seems to clamp the values.

vanfanel · 2023-03-19T23:25:02Z

@ColinKinloch I added the clamping as x11 does in commit fb47a49

How does it work for you?

input/drivers/wayland_input.c

Co-authored-by: Colin Kinloch <colin@kinlo.ch>

ColinKinloch

This fixes HiDPI displays getting clamped to a smaller area.

input/drivers/wayland_input.c

Co-authored-by: Colin Kinloch <colin@kinlo.ch>

r3claimer · 2023-03-24T13:57:37Z

@ColinKinloch
Hello, this commit appears to have broken wayland display out for me. RA still runs fine and I can hear audio but the screen is black and I get the follow errors in the log. Launching the UI by itself also results in a black screen. I am one of the developers for Jelos, a custom linux distro. I get the following error in the logs:

[INFO] [Video]: Found display server: "null".
error marshalling arguments for lock_pointer (signature noo?ou): null value passed for arg 2
Error marshalling request: Invalid argument

Rolling back to prior commit fixes the issue: 8d1e575

ColinKinloch · 2023-03-24T14:08:21Z

Which compositor are you using? KDE, Gnome?

ColinKinloch · 2023-03-24T14:10:23Z

The log is saying that wl->wl_pointer is NULL in:

RetroArch/input/drivers/wayland_input.c

Lines 431 to 432 in eeda1f6

    
           gfx->locked_pointer = zwp_pointer_constraints_v1_lock_pointer(gfx->pointer_constraints, 
        
              gfx->surface, gfx->wl_pointer, NULL, ZWP_POINTER_CONSTRAINTS_V1_LIFETIME_PERSISTENT);

ColinKinloch · 2023-03-24T15:35:31Z

@brooksytech what compositor does Jelos use?

r3claimer · 2023-03-25T13:30:22Z

@brooksytech what compositor does Jelos use?

Weston in Kiosk shell mode.

r3claimer · 2023-03-25T13:47:31Z

And looks like its been fixed already with the new commits you made. Thank You!

[Wayland] Add mouse grab functionality.

ee00814

vanfanel mentioned this pull request Mar 18, 2023

Mouse is not grabbed on Wayland #15057

Closed

vanfanel changed the title ~~[Wayland] Add mouse grab/lock functionality.~~ [WIP] [Wayland] Add mouse grab/lock functionality. Mar 18, 2023

ColinKinloch suggested changes Mar 18, 2023

View reviewed changes

input/drivers/wayland_input.c Outdated Show resolved Hide resolved

input/common/wayland_common.h Show resolved Hide resolved

vanfanel and others added 2 commits March 18, 2023 20:40

Update input/drivers/wayland_input.c

e58bd5f

Co-authored-by: Colin Kinloch <colin@kinlo.ch>

Update input/common/wayland_common.h

258db60

Co-authored-by: Colin Kinloch <colin@kinlo.ch>

[Wayland] Correct data pointer type in input_wl_grab_mouse

6feaa60

[Wayland] Add locked pointer listener and increment with delta only w…

74913ea

…hen locked.

[Wayland] Move delta increments to handle_relative_motion.

f29a351

ColinKinloch suggested changes Mar 19, 2023

View reviewed changes

input/drivers/wayland_input.c Outdated Show resolved Hide resolved

input/common/wayland_common.c Outdated Show resolved Hide resolved

input/common/wayland_common.c Outdated Show resolved Hide resolved

i30817 mentioned this pull request Mar 19, 2023

Mouse cursor problems when starting the core without any game libretro/scummvm#15

Closed

[Wayland] Add clamping to window dimensions.

fb47a49

ColinKinloch suggested changes Mar 19, 2023

View reviewed changes

input/drivers/wayland_input.c Outdated Show resolved Hide resolved

Update input/drivers/wayland_input.c

d70ad4c

Co-authored-by: Colin Kinloch <colin@kinlo.ch>

ColinKinloch suggested changes Mar 19, 2023

View reviewed changes

input/drivers/wayland_input.c Outdated Show resolved Hide resolved

Update input/drivers/wayland_input.c

e8aedfd

Co-authored-by: Colin Kinloch <colin@kinlo.ch>

ColinKinloch approved these changes Mar 20, 2023

View reviewed changes

vanfanel changed the title ~~[WIP] [Wayland] Add mouse grab/lock functionality.~~ [Wayland] Add mouse grab/lock functionality. Mar 20, 2023

vanfanel mentioned this pull request Mar 20, 2023

Missing mouse grabbing feature? labwc/labwc#809

Closed

LibretroAdmin merged commit aaa53da into libretro:master Mar 20, 2023

ColinKinloch mentioned this pull request Mar 24, 2023

(Wayland) Check for pointer before locking it #15130

Merged

i30817 mentioned this pull request May 3, 2023

[Linux] Physical mouse and invisible walls in cannon fodder in amiga core PUAE #15249

Open

i30817 mentioned this pull request Nov 30, 2023

Mouse Grab not enabled for Android #14016

Open

[Wayland] Add mouse grab/lock functionality. #15103

[Wayland] Add mouse grab/lock functionality. #15103

Conversation

vanfanel commented Mar 18, 2023 • edited Loading

Description

Related Issues

i30817 commented Mar 18, 2023 • edited Loading

vanfanel commented Mar 18, 2023

ColinKinloch commented Mar 18, 2023

ColinKinloch left a comment

Choose a reason for hiding this comment

ColinKinloch commented Mar 18, 2023

ColinKinloch commented Mar 18, 2023

vanfanel commented Mar 18, 2023 • edited Loading

ColinKinloch commented Mar 18, 2023

ColinKinloch commented Mar 18, 2023 • edited Loading

vanfanel commented Mar 18, 2023 • edited Loading

ColinKinloch commented Mar 18, 2023

vanfanel commented Mar 18, 2023

ColinKinloch commented Mar 18, 2023 • edited Loading

vanfanel commented Mar 19, 2023

ColinKinloch commented Mar 19, 2023

vanfanel commented Mar 19, 2023 • edited Loading

ColinKinloch commented Mar 19, 2023

vanfanel commented Mar 19, 2023 • edited Loading

i30817 commented Mar 19, 2023 • edited Loading

ColinKinloch commented Mar 19, 2023

ColinKinloch left a comment

Choose a reason for hiding this comment

ColinKinloch commented Mar 19, 2023

vanfanel commented Mar 19, 2023

i30817 commented Mar 19, 2023

ColinKinloch commented Mar 19, 2023

ColinKinloch commented Mar 19, 2023

vanfanel commented Mar 19, 2023

ColinKinloch commented Mar 19, 2023

ColinKinloch commented Mar 19, 2023

vanfanel commented Mar 19, 2023

ColinKinloch commented Mar 19, 2023 • edited Loading

ColinKinloch commented Mar 19, 2023

vanfanel commented Mar 19, 2023

ColinKinloch left a comment

Choose a reason for hiding this comment

r3claimer commented Mar 24, 2023

ColinKinloch commented Mar 24, 2023

ColinKinloch commented Mar 24, 2023

ColinKinloch commented Mar 24, 2023

r3claimer commented Mar 25, 2023

r3claimer commented Mar 25, 2023

vanfanel commented Mar 18, 2023 •

edited

Loading

i30817 commented Mar 18, 2023 •

edited

Loading

vanfanel commented Mar 18, 2023 •

edited

Loading

ColinKinloch commented Mar 18, 2023 •

edited

Loading

vanfanel commented Mar 18, 2023 •

edited

Loading

ColinKinloch commented Mar 18, 2023 •

edited

Loading

vanfanel commented Mar 19, 2023 •

edited

Loading

vanfanel commented Mar 19, 2023 •

edited

Loading

i30817 commented Mar 19, 2023 •

edited

Loading

ColinKinloch commented Mar 19, 2023 •

edited

Loading