8257414: Drag n Drop target area is wrong on high DPI systems #1907
Conversation
|
👋 Welcome back omikhaltcova! A progress list of the required criteria for merging this PR into |
|
@omikhaltsova The following label will be automatically applied to this pull request:
When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command. |
Webrevs
|
| @@ -613,7 +613,7 @@ private boolean processXdndPosition(XClientMessageEvent xclient) { | |||
| if (xwindow != null) { | |||
| /* Translate mouse position from root coordinates | |||
| to the target window coordinates. */ | |||
| Point p = xwindow.toLocal(x, y); | |||
| Point p = xwindow.toLocal(xwindow.scaleDown(x), xwindow.scaleDown(y)); | |||
There was a problem hiding this comment.
I guess some clarification here is required. It is unclear what the coordinate system is used by the x/y above. Looks like before the fix the device space was always used, and after the fix the mix of device/user's space will be used.
If the user's space should always be used(the scaled version of the device), then I suggest to scale these coordinates here:
x = (int)(xclient.get_data(2) >> 16);
y = (int)(xclient.get_data(2) & 0xFFFF);
But you will need to check that all usage of the new coordinate system will be valid across the method.
There was a problem hiding this comment.
Sergey, thank you! Could you give me a hint please concerning some use case of drag-and-drop to meet the (xwindow == 0) condition in XDnDDropTargetProtocol::processXdndPosition(..)? I'm unable to do it by means of the tests' apps I have.
There was a problem hiding this comment.
Sergey, thank you! Could you give me a hint please concerning some use case of drag-and-drop to meet the (xwindow == 0) condition in XDnDDropTargetProtocol::processXdndPosition(..)? I'm unable to do it by means of the tests' apps I have.
I think it is somehow related to the embedded frame when the event comes from some native window that is not registered as a java peer(but some external native app/window)
There was a problem hiding this comment.
I failed to test branches of XDnDDropTargetProtocol::processXdndPosition(..) under condition (xwindow == null) by means of EmbeddedFrame. In addition none of automatic regression tests (test/jdk/java/awt/dnd/, test/jdk/java/awt/xembed/, test/jdk/sun/awt) gets into these branches.
I also tried to scale coordinates (as you advised) right after:
x = (int)(xclient.get_data(2) >> 16);
y = (int)(xclient.get_data(2) & 0xFFFF);
as follow:
GraphicsConfiguration gc = GraphicsEnvironment
.getLocalGraphicsEnvironment()
.getDefaultScreenDevice()
.getDefaultConfiguration();
gc = SunGraphicsEnvironment.getGraphicsConfigurationAtPoint(gc, x, y);
GraphicsDevice device = gc.getDevice();
if (device instanceof X11GraphicsDevice) {
int scale = ((X11GraphicsDevice) device).getScaleFactor();
x = XlibUtil.scaleDown(x, scale);
y = XlibUtil.scaleDown(y, scale);
}
but found out that getGraphicsConfigurationAtPoint() also requires the already scaled coordinates.
There are 3 different places in XDnDDropTargetProtocol::processXdndPosition(..) where it is necessary to decide whether to scale coordinates or not:
- I'm not sure and I'm unable to test it:
if (xwindow == null)
{
long receiver =
XDropTargetRegistry.getRegistry().getEmbeddedDropSite(
xclient.get_window(), x, y);
- I'm quite sure and I'm able to test it:
if (xwindow != null) {
/* Translate mouse position from root coordinates
to the target window coordinates. */
Point p = xwindow.toLocal(xwindow.scaleDown(x), xwindow.scaleDown(y));
- I think that scaling is necessary but I'm unable to test it (additionally the question raises concerning this case, why the coordinates are not converted to local here similar to the 2 case):
if (xwindow == null) {
if (targetXWindow != null) {
notifyProtocolListener(targetXWindow, x, y,
DnDConstants.ACTION_NONE, xclient,
MouseEvent.MOUSE_EXITED);
Sergey, is it possible at the moment to fix the particular issue and leave the other use cases to work as it works before due to no possibility to test them?
There was a problem hiding this comment.
There are 3 different places in XDnDDropTargetProtocol::processXdndPosition(..) where it is necessary to decide whether to scale coordinates or not:
Unfortunately, this is the purpose of the whole fix to decide when the scale is required and then not, otherwise we can simply break the untested cases.
There was a problem hiding this comment.
Sergey, please, review my last changes! Finally I managed to check all necessary branches in XDnDDropTargetProtocol::processXdndPosition(..). The user's space (the scaled version of the device) must be used everywhere here. I had to devide scaling according to condition (xwindow != null) in order to use more simple and fast realization for most of cases. The scaling for the case (xwindow == null) related to the embedded frame is based on the idea that there is only one pointer (cursor) in the system and it is used for drag-n-drop operation. I tried different ways to get scale in the last case but only this one is workable. If you have some better idea, tell me, please, another way to do it!
| XToolkit.awtLock(); | ||
| try { | ||
| for (int i = 0; i < gdslen; i++) { | ||
| long screenRoot = XlibWrapper.RootWindow(display, i); |
There was a problem hiding this comment.
I am not sure this will work in the multi-monitors configuration when the screen is split across a few monitors(when Xinerama is enabled for example). In such a configuration, only one root window and only one X11 screen exist. And the index of the GraphicsDevice is the index of the "virtual" monitor, so you cannot pass this index to the "XlibWrapper.RootWindow".
A few questions:
- Are you sure that to scale the x/y in the device space you need to find where the pointer is located? And not where the x/y coordinates are located themselves?
- Did you try to iterate over the devices->downscale the bounds of each GD to the device scale and upscale the x/y using the selected device. Such utility method can be added to the SunGraphicsEnvironment
There was a problem hiding this comment.
Sergey, thanks for the hint! Hope I understand you right. Please, review my changes!
I decided to do necessary scaling in place and not to add an utility method to SunGraphicsEnvironment due to the following doubts:
the methods from SunGraphicsEnvironment can be called from different OS and I'm going to use GraphicsConfig::getBounds() but I see that the Unix implementation includes downscaling (X11GraphicsConfig::getBounds(), src/java.desktop/unix/classes/sun/awt/X11GraphicsConfig.java) and the Windows one is not (Win32GraphicsConfig::getBounds(), src/java.desktop/windows/classes/sun/awt/Win32GraphicsConfig.java);
so for Unix I need upscaling before checking the point position but for Windows it is not needed.
Please, correct me if I'm wrong!
Concerning my previous implementation based on the cursor position, would you be so kind to take a look at XMouseInfoPeer::fillPointWithCoords(..) (src/java.desktop/unix/classes/sun/awt/X11/XMouseInfoPeer.java) in order to check whether it should be also fixed?
There was a problem hiding this comment.
Yes, I think the fillPointWithCoords is also broken.
I'll run the tests for this change.
|
@omikhaltsova This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be: You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 817 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@mrserb) but any other Committer may sponsor as well. ➡️ To flag this PR as ready for integration with the above commit message, type |
|
Sergey, thank you for reviewing! /integrate |
|
@omikhaltsova |
|
/sponsor |
openjdk
bot
added
integrated
|
@mrserb @omikhaltsova Since your change was applied there have been 828 commits pushed to the
Your commit was automatically rebased without conflicts. Pushed as commit d339832. 💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored. |
Please, review this small fix for drag-n-drop on Linux with HiDPI turned on!
This bug is due to the following reason: while scaling Java recalculates resolution (W x H) according to sun.java2d.uiScale (W/SCALE x H/SCALE) and works inside these new coordinates but at the same time the events, that come from the system, continue reporting positions in the old coordinates (W x H).
The idea of the suggested fix is in division of coordinates on the scale when they come from the system to Java and multiplying them on the scale when they go back from Java to the system. It is similar to processing events from mouse and buttons.
Testing is quite complicated because for reproducing this bug the following conditions should be met:
The step-by-step guide how to reproduce this bug is added to https://bugs.openjdk.java.net/browse/JDK-8257414.
Progress
Issue
Reviewers
Download
$ git fetch https://git.openjdk.java.net/jdk pull/1907/head:pull/1907$ git checkout pull/1907