Multimonitor Offset for Mouse Over #1339

totaam · 2016-10-12T23:42:02Z

Issue migrated from trac ticket # 1339

component: client | priority: major | resolution: fixed | keywords: mouse cursor scroll

2016-10-12 23:42:02: jdawgzim created the issue

Title: Multimonitor Offset for Mouse Over

Situation: Using the 2nd monitor on windows client rotated to portrait and then shifted up higher then the primary in the multiple displays settings.

Result: Scroll wheel and other mouse over commands behave like the mouse pointer is off by the same distance the 2nd monitor is shifted above the primary monitor. So in all application irregardless of which monitor they are currently in will not scroll when mouse cursor is higher then that distance from the top of the application windows (doesn't matter if it's maximized or not). Below that offset it will scroll what is offset that distance above the mouse cursor. I hope this makes sense. Please check screenshot for monitor config.

totaam · 2016-10-12T23:42:58Z

2016-10-12 23:42:58: jdawgzim uploaded file `xpra_scroll_bug.zip` (464.4 KiB)

totaam · 2016-10-21T13:50:50Z

2016-10-21 13:50:50: antoine uploaded file `xp-two-screens.png` (714.7 KiB)

two side by side screens with windows XP

totaam · 2016-10-21T13:54:53Z

2016-10-21 13:54:53: antoine uploaded file `rotate-config.png` (41.5 KiB)

the screen config as shown in the bug report screenshot

totaam · 2016-10-21T13:58:06Z

2016-10-21 13:58:06: antoine changed owner from antoine to jdawgzim

totaam · 2016-10-21T13:58:06Z

2016-10-21 13:58:06: antoine commented

Display data from the bug report tool for this Microsoft Windows 7 client:
{'screens': {
 0: {
    'name': '1\\WinSta0\\Default',
    'height_mm': 508, 'width_mm': 793,
    'settings': {}, 'resolution': -1, 'primary_monitor': 0,
    'height': 1920, 'width': 3000,
    'visual': {
        'rgb': {
            'visual_type': 'TRUE_COLOR', 'byte_order': 'LSB', 'depth': 24, 'bits_per_rgb': 42, 'colormap_size': 256
        },
        'system_visual': {
            'visual_type': 'TRUE_COLOR', 'byte_order': 'LSB', 'depth': 24, 'bits_per_rgb': 42, 'colormap_size': 256
        }
     },
     'root': (0, 0, 1920, 1200, 24),
     'monitors': 2,
     'monitor': {
         0: {
             'height_mm': 423, 'width_mm': 677, 'height': 1200, 'width': 1920, 'y': 120, 'x': 0, 'plug_name': '\\\\.\\DISPLAY1' \
         },
         1: {
             'height_mm': 677, 'width_mm': 381, 'height': 1920, 'width': 1080, 'y': 0, 'x': 1920, 'plug_name': '\\\\.\\DISPLAY2' \
            }
         }
     }
},
'device': {
    0: {
        '': 'Core Pointer', 'keys': [], 'num_axes': 2, 'axes': ((1, 0.0, 0.0), (2, 0.0, 0.0)), 'num_keys': 0, 'source': 'MOUSE', 'mode': 'SCREEN', 'has_cursor': True
    }
},
'maximal_cursor_size': (32, 32), 'pointer': (1623, 892),
'name': '1\\WinSta0\\Default',
'root': (0, 0, 1920, 1200, 24),
'root-size': (3000, 1920),
'devices': 1,
'default_cursor_size': 32L,
'pointer_is_grabbed': False,
'supports': {
    'clipboard_persistence': False, 'composite': False, 'cursor_color': True, 'shapes': True, 'selection_notification': True, 'cursor_alpha': True}
}
This seems to match the configuration in the screenshot you included:
[[Image(rotate-config.png)]]

So the first monitor has a "y" offset of 120.
I believe that win32 and/or GTK may be using negative coordinates for the second monitor, and X11 will discard those.
When I tested it though, it worked without problems:
[[Image(xp-two-screens.png)]]

So please try to run the client with the "-d mouse" flag and post the output.
We want to see what coordinates are reported in the mouse events for when the window is near the top of the second monitor.
(just in case, please also include the server log output so we can see the initial screen setup, and maybe also "xpra info")

totaam · 2016-10-28T17:39:26Z

2016-10-28 17:39:26: jdawgzim uploaded file `xpra_scroll_bug_2.zip` (401.5 KiB)

totaam · 2016-10-28T17:43:29Z

2016-10-28 17:43:29: jdawgzim commented

Submitted xpra_scroll_bug_2.zip while "-d mouse" was enabled.

I can now fix my issue by moving my portrait monitor to the left and then making it the primary monitor.

totaam · 2016-10-28T17:46:07Z

2016-10-28 17:46:07: antoine commented

@jdawgzim: you've attached the bug report data, we need to see the "xpra_cmd.exe" command output instead. (with the misbehaving configuration obviously)

totaam · 2016-10-28T18:30:20Z

2016-10-28 18:30:20: jdawgzim commented

Replying to [comment:3 antoine]:
How do I do that?

totaam · 2016-10-28T18:40:16Z

2016-10-28 18:40:16: antoine commented

Run xpra from the command line, ie: open did command prompt then
cd ..
cd ..
cd Program Files
cd Xpra
Xpra_cmd.exe attach to:host:port -d mouse
(Paths may vary slightly)

totaam · 2016-10-28T19:00:07Z

2016-10-28 19:00:07: jdawgzim commented

I use cmd.exe and run "Xpra_cmd.exe attach to:host:port -d mouse" and I get a lot of text. It doesn't let me copy the text. Is there a .log output somewhere or some way to capture this? Thank you for your patience with me

totaam · 2016-10-28T19:05:34Z

2016-10-28 19:05:34: jdawgzim uploaded file `xpra_scroll_bug.jpg` (1489.8 KiB)

totaam · 2016-10-28T19:30:00Z

2016-10-28 19:30:00: afarr commented

If you left click at the top left of the cmd.exe window, you should find an 'edit' option, which will allow you to select/enable copying. Then you'll just need to left-click to highlight the text you want to copy, then left click on it again to sync it with the clipboard.

totaam · 2016-10-28T19:40:16Z

2016-10-28 19:40:16: jdawgzim uploaded file `xpra_scroll_bug.txt` (20.7 KiB)

totaam · 2016-10-29T05:42:24Z

2016-10-29 05:42:24: antoine commented

A better and easier way of capturing log output, one that does not wrap the lines at 80 characters (which makes them so much harder to parse), is:
Xpra_cmd.exe attach to:host:port -d mouse > C:\xpra-log.txt 2>&1
Then you can just attach "C:\xpra-log.txt".

For my own reference, the code that converts the event's native coordinates is:

for X11 clients: gdkwindow: convert_toplevel_coords_to_window

for win32: [https://github.com/GNOME/gtk/blob/master/gdk/win32/gdkevents-win32.c]:
current_root_x = msg->pt.x + _gdk_offset_x;
current_root_y = msg->pt.y + _gdk_offset_y;
event->motion.x # current_x(gint16) GET_X_LPARAM (msg->lParam);
event->motion.y # current_y(gint16) GET_Y_LPARAM (msg->lParam);
directly from the WM_MOUSEMOVE event message. As of r14323 we will now log both the root coordinates and plain (x,y).
The good news is that the win32 code gives us the raw values. It's quite possible that the window ends up using negative coordinates on win32, which then gets mapped partially offscreen on the server and is therefore unable to receive any events in that region.

@jdawgzim: the mouse in your short log sample is not anywhere near the top of the screen, the y coordinate is near ~1000 which is not where the problem occurs from what I understand:
do_motion_notify_event(<gtk.gdk.Event at 0577E590: GDK_MOTION_NOTIFY x=1684.00, y=1115.00>) \
    wid=9 / focus=9, device=Core Pointer, pointer=(2773, 1431), modifiers=['mod2'], buttons=[]
please try the latest client beta build. It includes a workaround for #1284 which may help, and the extra debug logging should also help narrow things down.
Please make sure to move the mouse in the problematic area, and keep the log sample as small as possible.
Also, it would be easier to parse the output if you add "--desktop-scaling=no" to the command line.
If the coordinates are negative, it may be useful to also have the "-d geometry" log output.
Finnaly, a screenshot showing where the mouse was when the log was capture would be ideal.

totaam · 2016-12-26T09:41:24Z

2016-12-26 09:41:24: antoine changed owner from jdawgzim to afarr

totaam · 2016-12-26T09:41:24Z

2016-12-26 09:41:24: antoine commented

Not heard back, @afarr: can you help with this?

totaam · 2017-01-04T20:30:42Z

2017-01-04 20:30:42: jdawgzim commented

Sorry, I worked around this issue by putting my vertical monitor on the left and switching the vertical monitor to be the primary monitor (start menu).

If I switch back to making the right horizontal monitor the primary monitor the issue returns. If I right or left click the mouse it is correct. It's just a "mouse over scroll wheel action" that is the offset the same amount as the two monitors are offset relative to where the primary monitor (start menu) is. It's like Xpra needs the start menu or primary monitor to be on the far left bottom.

I don't run the Xpra server where I use Xpra so I can't update it. I could update my client, but that probably wouldn't fix it. If I get brave enough in the future I could try to recreate this problem on my home computer and get you more info.

totaam · 2017-01-05T17:58:38Z

2017-01-05 17:58:38: maxmylyn commented

Retrying with a trunk r14619 Win8.1 client against a trunk 14712 Fedora 25 server and am only able to reproduce the scrolling offset. The mouse click actions behave nicely, but the scrolling does not. I'll attach a -d mouse log in a sec.

totaam · 2017-01-05T18:00:36Z

2017-01-05 18:00:36: maxmylyn changed owner from afarr to antoine

totaam · 2017-01-05T18:00:36Z

2017-01-05 18:00:36: maxmylyn commented

There you go, hand it back to me if you want any other logs.

totaam · 2017-01-05T18:01:38Z

2017-01-05 18:01:38: maxmylyn uploaded file `1339 Monitor Layout.png` (44.4 KiB)

Here's a screenshot of the monitor layout I'm using - both are 1080p

totaam · 2017-01-06T05:28:16Z

2017-01-06 05:28:16: antoine changed owner from antoine to maxmylyn

totaam · 2017-01-06T05:28:16Z

2017-01-06 05:28:16: antoine commented

@maxmylin: the log file you attached doesn't have any mouse event debugging in it.

It should look like this with "-d mouse":

for clicks:

_button_action(4, <gtk.gdk.Event at 0x7f99a240bb20: GDK_SCROLL x=253.00, y=121.00, direction=GDK_SCROLL_UP>, False) \
    wid=1 / focus=1 / window wid=1, device=Core Pointer, pointer=(303, 242), modifiers=['mod2'], buttons=[]

for movements:

do_motion_notify_event(<gtk.gdk.Event at 0x7f99a240bb20: GDK_MOTION_NOTIFY x=253.00, y=122.00>) \
    wid=1 / focus=1 / window wid=1, device=Core Pointer, pointer=(303, 243), modifiers=['mod2'], buttons=[]

totaam · 2017-01-06T16:54:23Z

2017-01-06 16:54:23: maxmylyn changed owner from maxmylyn to antoine

totaam · 2017-01-06T16:54:23Z

2017-01-06 16:54:23: maxmylyn commented

Oops It helps to remember to add -d mouse to the command line arguments. That would explain why the file was so small..

Anyways, steps are the same:

Connected

Clicked a few times

Scrolled a few times

Clicked some more

Scrolled some more

The clicks (left, middle, and right mouse) all registered in the correct place, but the scrolls were indeed offset.

totaam · 2017-01-06T16:54:40Z

2017-01-06 16:54:40: maxmylyn uploaded file `1339dmouse.txt` (312.7 KiB)

did some scrolling and clicking - the scrolls were offset but the clicks were not.

totaam · 2017-01-06T17:35:40Z

2017-01-06 17:35:40: antoine changed owner from antoine to maxmylyn

totaam · 2017-01-06T17:35:40Z

2017-01-06 17:35:40: antoine commented

Does the problem go away if you run the client with XPRA_WHEEL=0?

Pointer events are adjusted for the second monitor (964 vs 638, 1339 vs 730):
do_motion_notify_event(<gtk.gdk.Event at 0482FE18: GDK_MOTION_NOTIFY x=638.00, y=730.00>) wid=4 / focus=4 / window wid=4, device=Core Pointer, pointer=(964, 1339), modifiers=['mod2'], buttons=[]
But our wheel event throttling code (#1131) doesn't know about the screen adjustments made by GTK.. and so we send the raw values, and those will be offset:
mousewheel: orientation=vertical distance=-120.0, units=-1, new value=-120.0, keys=0x0, x=964, y=849, client=gtk2.client, wid=4
If you confirm that this is indeed the problem, I'll try to find a solution.
Potential inelegant solutions I can think of:

keep the last position of the mouse as reported by GTK and use that value instead

calculate the GTK offset on every mouse move event and apply the same value to our cooked wheel events

totaam · 2017-01-06T18:04:40Z

2017-01-06 18:04:40: maxmylyn changed owner from maxmylyn to antoine

totaam · 2017-02-16T18:28:44Z

2017-02-16 18:28:44: maxmylyn commented

The 2.X r15030 client works fine.

totaam · 2017-02-16T18:29:38Z

2017-02-16 18:29:38: antoine changed priority from critical to blocker

totaam · 2017-02-16T18:29:38Z

2017-02-16 18:29:38: antoine commented

The fix to test for this multi monitor bug is in the 2.0 branch.

If wheel events no longer work at all in the 1.x branch then this is serious issue - so raising to blocker. That's surprising seeing that your log samples indicate that the wheel events have been sent to the server.

And I have just tested 1.0.3 and I see the events:

client side with "-d mouse"

server side with "-d mouse"

applications respond

xev shows the button events (buttons 4 and 5)

totaam · 2017-02-16T19:30:25Z

2017-02-16 19:30:25: maxmylyn commented

NOTE:

The 1.0.X client only stops scrolling if there more than monitor attached to the client machine - after disconnecting the second monitor, scrolling works again.

totaam · 2017-02-16T19:33:33Z

2017-02-16 19:33:33: maxmylyn commented

Also:

Setting XPRA_WHEEL=0 gets scrolling to work again with both monitors plugged in.

totaam · 2017-02-17T12:34:59Z

2017-02-17 12:34:59: antoine changed priority from blocker to major

totaam · 2017-02-17T12:34:59Z

2017-02-17 12:34:59: antoine commented

Right, so the 1.0.x client is still affected by this bug - which is totally expected since the fix wasn't in there yet.
I was thinking that something broke in 1.x - stand down red alert.

I have uploaded a beta 1.0.4 with this change backported (15100 + 15104, backport was made more difficult by the switch to ctypes #678): [http://xpra.org/beta/windows].

totaam · 2017-02-21T18:10:38Z

2017-02-21 18:10:38: maxmylyn commented

Latest 1.0.X build (r15129) fixed it - connecting against a 2.0 r15129 Fedora 25 server scrolling works fine with multimonitor in a number of configurations.

I'll hold on to this for a few hours to double check nothing else broke, and then close it if everything checks out.

totaam · 2017-02-21T22:10:39Z

2017-02-21 22:10:39: maxmylyn changed status from new to closed

totaam · 2017-02-21T22:10:39Z

2017-02-21 22:10:39: maxmylyn set resolution to fixed

totaam · 2017-02-21T22:10:39Z

2017-02-21 22:10:39: maxmylyn commented

Yep, everything looks good - closing.

totaam · 2019-08-05T13:14:44Z

2019-08-05 13:14:44: antoine uploaded file `context-menu-garbled.png` (346.7 KiB)

context menu shifted up and with pointer offset

totaam · 2019-08-05T13:16:31Z

2019-08-05 13:16:31: antoine changed status from closed to reopened

totaam · 2019-08-05T13:16:31Z

2019-08-05 13:16:31: antoine removed resolution (was fixed)

totaam · 2019-08-05T13:16:31Z

2019-08-05 13:16:31: antoine commented

Following this mailing list post, I am seeing this problem again with 3.0 and many applications:

./tests/xpra/test_apps/test_context_menu.py

eclipse

libreoffice

Looks like the client OS decides to shift the popup to keep it all on-screen, but the server copy is not shifted so the events land in the wrong place?
[[Image(context-menu-garbled.png)]]

totaam · 2019-08-05T15:18:22Z

2019-08-05 15:18:22: antoine changed status from reopened to new

totaam · 2019-08-05T15:18:22Z

2019-08-05 15:18:22: antoine changed owner from maxmylyn to antoine

totaam · 2019-08-05T15:18:22Z

2019-08-05 15:18:22: antoine commented

Downgrading the server all the way to v1.0.14 does not help.
Bisecting the client:

v1.0.13: OK

v2.0.3: OK

v2.2.6: OK

v2.4.3: NOK

v2.3.4: NOK

v2.3: OK

v2.3.1: OK

v2.3.2: NOK

So the bug was backported to the v2.3.x branch between 19350 (v2.3.1) and 19657 (v2.3.2).
The list of changesets is here: [/log/xpra/tags/v2.3.x?rev=19657&stop_rev=&limit=61 rev 19657 - 61 changes]

totaam · 2019-08-05T16:26:46Z

2019-08-05 16:26:46: @totaam changed status from new to assigned

totaam · 2019-08-05T16:26:46Z

2019-08-05 16:26:46: @totaam changed owner from antoine to totaamwin32

totaam · 2019-08-05T16:26:46Z

2019-08-05 16:26:46: @totaam commented

Potential problems in 19651?
Less likely in:

19369

19380

19382, 19389, 19472

19655

totaam · 2019-08-05T17:22:54Z

2019-08-05 17:22:54: @totaam uploaded file `fix-window-offsets.patch` (2.6 KiB)

this hack of a patch fixes the symptoms for this particular case..

totaam · 2019-08-06T16:20:26Z

2019-08-06 16:20:26: @totaam changed status from assigned to closed

totaam · 2019-08-06T16:20:26Z

2019-08-06 16:20:26: @totaam set resolution to fixed

totaam · 2019-08-06T16:20:26Z

2019-08-06 16:20:26: @totaam commented

Fixed in r23459.

totaam closed this as completed Aug 6, 2019

This was referenced Jan 22, 2021

non-opengl backend rendering issues #1874

Closed

xpra shadow dying due to lack of bandwidth #2029

Closed

mouse offset with java applications #2600

Open

gnome-terminal sides not rendered fully #2618

Open

Multimonitor Offset for Mouse Over #1339

Multimonitor Offset for Mouse Over #1339

Comments

totaam commented Oct 12, 2016

2016-10-12 23:42:02: jdawgzim created the issue

totaam commented Oct 12, 2016

2016-10-12 23:42:58: jdawgzim uploaded file xpra_scroll_bug.zip (464.4 KiB)

totaam commented Oct 21, 2016

2016-10-21 13:50:50: antoine uploaded file xp-two-screens.png (714.7 KiB)

totaam commented Oct 21, 2016

2016-10-21 13:54:53: antoine uploaded file rotate-config.png (41.5 KiB)

totaam commented Oct 21, 2016

2016-10-21 13:58:06: antoine changed owner from antoine to jdawgzim

totaam commented Oct 21, 2016

2016-10-21 13:58:06: antoine commented

totaam commented Oct 28, 2016

2016-10-28 17:39:26: jdawgzim uploaded file xpra_scroll_bug_2.zip (401.5 KiB)

totaam commented Oct 28, 2016

2016-10-28 17:43:29: jdawgzim commented

totaam commented Oct 28, 2016

2016-10-28 17:46:07: antoine commented

totaam commented Oct 28, 2016

2016-10-28 18:30:20: jdawgzim commented

totaam commented Oct 28, 2016

2016-10-28 18:40:16: antoine commented

totaam commented Oct 28, 2016

2016-10-28 19:00:07: jdawgzim commented

totaam commented Oct 28, 2016

2016-10-28 19:05:34: jdawgzim uploaded file xpra_scroll_bug.jpg (1489.8 KiB)

totaam commented Oct 28, 2016

2016-10-28 19:30:00: afarr commented

totaam commented Oct 28, 2016

2016-10-28 19:40:16: jdawgzim uploaded file xpra_scroll_bug.txt (20.7 KiB)

totaam commented Oct 29, 2016

2016-10-29 05:42:24: antoine commented

totaam commented Dec 26, 2016

2016-12-26 09:41:24: antoine changed owner from jdawgzim to afarr

totaam commented Dec 26, 2016

2016-12-26 09:41:24: antoine commented

totaam commented Jan 4, 2017

2017-01-04 20:30:42: jdawgzim commented

totaam commented Jan 5, 2017

2017-01-05 17:58:38: maxmylyn commented

totaam commented Jan 5, 2017

2017-01-05 18:00:36: maxmylyn changed owner from afarr to antoine

totaam commented Jan 5, 2017

2017-01-05 18:00:36: maxmylyn commented

totaam commented Jan 5, 2017

2017-01-05 18:01:38: maxmylyn uploaded file 1339 Monitor Layout.png (44.4 KiB)

totaam commented Jan 6, 2017

2017-01-06 05:28:16: antoine changed owner from antoine to maxmylyn

totaam commented Jan 6, 2017

2017-01-06 05:28:16: antoine commented

totaam commented Jan 6, 2017

2017-01-06 16:54:23: maxmylyn changed owner from maxmylyn to antoine

totaam commented Jan 6, 2017

2017-01-06 16:54:23: maxmylyn commented

totaam commented Jan 6, 2017

2017-01-06 16:54:40: maxmylyn uploaded file 1339dmouse.txt (312.7 KiB)

totaam commented Jan 6, 2017

2017-01-06 17:35:40: antoine changed owner from antoine to maxmylyn

totaam commented Jan 6, 2017

2017-01-06 17:35:40: antoine commented

totaam commented Jan 6, 2017

2017-01-06 18:04:40: maxmylyn changed owner from maxmylyn to antoine

totaam commented Feb 16, 2017

2017-02-16 18:28:44: maxmylyn commented

totaam commented Feb 16, 2017

2017-02-16 18:29:38: antoine changed priority from critical to blocker

totaam commented Feb 16, 2017

2017-02-16 18:29:38: antoine commented

totaam commented Feb 16, 2017

2017-02-16 19:30:25: maxmylyn commented

totaam commented Feb 16, 2017

2017-02-16 19:33:33: maxmylyn commented

totaam commented Feb 17, 2017

2017-02-17 12:34:59: antoine changed priority from blocker to major

totaam commented Feb 17, 2017

2017-02-17 12:34:59: antoine commented

totaam commented Feb 21, 2017

2016-10-12 23:42:58: jdawgzim uploaded file `xpra_scroll_bug.zip` (464.4 KiB)

2016-10-21 13:50:50: antoine uploaded file `xp-two-screens.png` (714.7 KiB)

2016-10-21 13:54:53: antoine uploaded file `rotate-config.png` (41.5 KiB)

2016-10-28 17:39:26: jdawgzim uploaded file `xpra_scroll_bug_2.zip` (401.5 KiB)

2016-10-28 19:05:34: jdawgzim uploaded file `xpra_scroll_bug.jpg` (1489.8 KiB)

2016-10-28 19:40:16: jdawgzim uploaded file `xpra_scroll_bug.txt` (20.7 KiB)

2017-01-05 18:01:38: maxmylyn uploaded file `1339 Monitor Layout.png` (44.4 KiB)

2017-01-06 16:54:40: maxmylyn uploaded file `1339dmouse.txt` (312.7 KiB)

2019-08-05 13:14:44: antoine uploaded file `context-menu-garbled.png` (346.7 KiB)

2019-08-05 17:22:54: @totaam uploaded file `fix-window-offsets.patch` (2.6 KiB)