8291087: Wrong position of focus of screen reader on Windows with screen scale > 1 #853

kevinrushforth · 2022-07-28T15:01:34Z

When using the Windows Narrator screen reader on a Hi-DPI screen, the visible focus is drawn in the wrong location and with the wrong size. This happens because the the JavaFX Windows accessibility code that returns the screen bounds of the requested UI element does not take the screen scale into account.

The fix is to adjust the bounds by the screen scale before returning it.

You can test it on a Windows machine with scaling set to >= 125% as follows:

Turn on Windows Narrator
Run any JavaFX program with at least UI controls, for example, hello.HelloTextField in apps/toys/Hello
TAB between the controls

Without the fix, Narrator shows the focus indicator in the wrong position and is too small. With the fix, it is correct. While testing this, I discovered an unrelated (and preexisting) bug where the focus indicator for a TextField or TextArea whose content is larger that the control (and is displayed with scroll bars) is not clipped to the visible area. This happens regardless of screen scale. I will file a follow-up bug for this.

Note that this bug is specific to Windows. It does not occur on macOS, which works correctly on a retina or non-retina display.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

JDK-8291087: Wrong position of focus of screen reader on Windows with screen scale > 1

Reviewers

Ajit Ghaisas (@aghaisas - Reviewer)
Ambarish Rapte (@arapte - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx pull/853/head:pull/853
$ git checkout pull/853

Update a local copy of the PR:
$ git checkout pull/853
$ git pull https://git.openjdk.org/jfx pull/853/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 853

View PR using the GUI difftool:
$ git pr show -t 853

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/853.diff

…een scale > 1

kevinrushforth · 2022-07-28T15:02:27Z

/reviewers 2

bridgekeeper · 2022-07-28T15:02:43Z

👋 Welcome back kcr! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2022-07-28T15:05:27Z

@kevinrushforth
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

mlbridge · 2022-07-28T15:10:26Z

Webrevs

01: Full - Incremental (554fae4d)
00: Full (a991c463)

andy-goryachev-oracle · 2022-07-28T15:15:48Z

just curious, does it work when the control spans two monitors with different scales?
when dragging the enclosing window from one monitor to another?

kevinrushforth · 2022-07-28T15:41:25Z

I did some limited testing yesterday with two screens each of which had a different scale, and what I tested works as expected.

aghaisas

I tested this on Windows with 100%, 125%. 150% and 175% scales. I confirm that this fixes the focus rendering problem seen with the screen reader being ON.

modules/javafx.graphics/src/main/java/com/sun/glass/ui/win/WinAccessible.java

arapte

LGTM, Tested with different scale factors on one screen only.

openjdk · 2022-07-29T18:54:49Z

@kevinrushforth This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8291087: Wrong position of focus of screen reader on Windows with screen scale > 1

Reviewed-by: aghaisas, arapte

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 11 new commits pushed to the master branch:

b4d86bd: 8218826: TableRowSkinBase: horizontal layout broken if row has padding
4c1f4c2: Merge
dd30402: 8291589: Update copyright header for files modified in 2022
779365f: Merge
5febaca: 8291502: Mouse or touch presses on a non-focusable region don't clear the focusVisible flag of the current focus owner
8fa8919: 8291588: Update boot JDK to 18.0.2
08ec9c8: 8289605: Robot capture tests fail intermittently on Mac M1
4b4deae: 8289397: Fix warnings: Possible accidental assignment in place of a comparison. A condition expression should not be reduced to an assignment
89d1cf2: 8290530: Bump minimum JDK version for JavaFX to JDK 17
0b74ee8: 8290990: Clear .root style class from a root node that is removed from a Scene/SubScene
... and 1 more: https://git.openjdk.org/jfx/compare/49a2efbbfba017101e5f57b590bf820307863bc3...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

kevinrushforth · 2022-07-30T12:14:44Z

While testing this, I discovered an unrelated (and preexisting) bug where the focus indicator for a TextField or TextArea whose content is larger that the control (and is displayed with scroll bars) is not clipped to the visible area. This happens regardless of screen scale. I will file a follow-up bug for this.

I filed JDK-8291590 to track this.

kevinrushforth · 2022-07-30T14:39:30Z

I did a little more multi-screen testing with different layouts, and saw some odd behavior on the secondary screen that I want to track down and understand better. It look like a different problem, but I want to be sure before I integrate this. If it is a different problem, I'll file another follow-on bug.

kevinrushforth · 2022-08-05T18:38:28Z

Getting back to this, I discovered what the problem was with some multi-screens layouts when two screens have a different scale. Computing the platform x and y by simply multiplying by the scale is not right, and only work for a screen that touches the origin of the virtual screen space (in both X and Y). When the secondary screen is to the left or above the primary screen, the simple (but incorrect) calculation gives the same result as the correct calculation. For a secondary screen below or to the right of the primary screen, it doesn't. Simply scaling the coordinate about the origin produces a value that is either too small or too large when the two screens have a different scale.

The right fix is to use the Screen.toPlatformX and Screen.toPlatformY methods, which take this into account, and translates the coordinate to the right origin, scales it, and then translates it back. In order to do that, I needed to get the Window from which I could get the Screen, which required making the quantum WindowStage class public along with its getWindow method. This is an internal implementation class, so is not a concern.

I've tested this updated version in various combinations of layout and screen scales, and everything now works as expected.

andy-goryachev-oracle · 2022-08-05T18:44:21Z

@kevinrushforth :
on windows, a window can be positioned to span two monitors. you can't do this on Mac - once dropped, only a part of the window remains (on the monitor which contains the title bar, it seems). but on windows, the app continues to work on both monitors. what happens then?

kevinrushforth · 2022-08-05T18:50:54Z

Much of my testing was done with a window straddling two screens. It works as expected in all cases I checked. If a single control spans both screens, the part that is on each screen is correctly highlighted by the Windows Narrator visible focus.

Note that in this mode, the window is considered to be "on" one of the screens, and all scales are relative to that screen (not just for accessibility, this is how HiDPI works on Windows if you are multi-monitor HiDPI aware).

aghaisas

The fix looks good!

kevinrushforth · 2022-08-09T11:28:44Z

/integrate

openjdk · 2022-08-09T11:30:13Z

Going to push as commit 38324a7.
Since your change was applied there have been 11 commits pushed to the master branch:

b4d86bd: 8218826: TableRowSkinBase: horizontal layout broken if row has padding
4c1f4c2: Merge
779365f: Merge
5febaca: 8291502: Mouse or touch presses on a non-focusable region don't clear the focusVisible flag of the current focus owner
8fa8919: 8291588: Update boot JDK to 18.0.2
08ec9c8: 8289605: Robot capture tests fail intermittently on Mac M1
4b4deae: 8289397: Fix warnings: Possible accidental assignment in place of a comparison. A condition expression should not be reduced to an assignment
89d1cf2: 8290530: Bump minimum JDK version for JavaFX to JDK 17
0b74ee8: 8290990: Clear .root style class from a root node that is removed from a Scene/SubScene
... and 1 more: https://git.openjdk.org/jfx/compare/49a2efbbfba017101e5f57b590bf820307863bc3...master

Your commit was automatically rebased without conflicts.

openjdk · 2022-08-09T11:30:25Z

@kevinrushforth Pushed as commit 38324a7.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8291087: Wrong position of focus of screen reader on Windows with scr…

a991c46

…een scale > 1

kevinrushforth requested review from arapte and aghaisas July 28, 2022 15:01

openjdk bot added the rfr Ready for review label Jul 28, 2022

aghaisas approved these changes Jul 29, 2022

View reviewed changes

aghaisas reviewed Jul 29, 2022

View reviewed changes

modules/javafx.graphics/src/main/java/com/sun/glass/ui/win/WinAccessible.java Show resolved Hide resolved

arapte approved these changes Jul 29, 2022

View reviewed changes

openjdk bot added the ready Ready to be integrated label Jul 29, 2022

Fix conversion to platform coords

554fae4

openjdk bot removed the ready Ready to be integrated label Aug 5, 2022

kevinrushforth requested review from aghaisas and arapte August 6, 2022 12:38

aghaisas approved these changes Aug 8, 2022

View reviewed changes

arapte approved these changes Aug 9, 2022

View reviewed changes

openjdk bot added the ready Ready to be integrated label Aug 9, 2022

openjdk bot added the integrated Pull request has been integrated label Aug 9, 2022

openjdk bot closed this Aug 9, 2022

openjdk bot removed ready Ready to be integrated rfr Ready for review labels Aug 9, 2022

kevinrushforth deleted the 8291087-a11y-hidpi branch August 9, 2022 11:50

kevinrushforth mentioned this pull request Jan 3, 2024

8189282: JavaFX: Invalid position of candidate pop-up of InputMethod in Hi-DPI on Windows #1311

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8291087: Wrong position of focus of screen reader on Windows with screen scale > 1 #853

8291087: Wrong position of focus of screen reader on Windows with screen scale > 1 #853

kevinrushforth commented Jul 28, 2022 •

edited by openjdk bot

Loading

kevinrushforth commented Jul 28, 2022

bridgekeeper bot commented Jul 28, 2022

openjdk bot commented Jul 28, 2022

mlbridge bot commented Jul 28, 2022 •

edited

Loading

andy-goryachev-oracle commented Jul 28, 2022

kevinrushforth commented Jul 28, 2022

aghaisas left a comment

arapte left a comment

openjdk bot commented Jul 29, 2022 •

edited

Loading

kevinrushforth commented Jul 30, 2022

kevinrushforth commented Jul 30, 2022

kevinrushforth commented Aug 5, 2022

andy-goryachev-oracle commented Aug 5, 2022

kevinrushforth commented Aug 5, 2022

aghaisas left a comment

kevinrushforth commented Aug 9, 2022

openjdk bot commented Aug 9, 2022

openjdk bot commented Aug 9, 2022

8291087: Wrong position of focus of screen reader on Windows with screen scale > 1 #853

8291087: Wrong position of focus of screen reader on Windows with screen scale > 1 #853

Conversation

kevinrushforth commented Jul 28, 2022 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

kevinrushforth commented Jul 28, 2022

bridgekeeper bot commented Jul 28, 2022

openjdk bot commented Jul 28, 2022

mlbridge bot commented Jul 28, 2022 • edited Loading

Webrevs

andy-goryachev-oracle commented Jul 28, 2022

kevinrushforth commented Jul 28, 2022

aghaisas left a comment

Choose a reason for hiding this comment

arapte left a comment

Choose a reason for hiding this comment

openjdk bot commented Jul 29, 2022 • edited Loading

kevinrushforth commented Jul 30, 2022

kevinrushforth commented Jul 30, 2022

kevinrushforth commented Aug 5, 2022

andy-goryachev-oracle commented Aug 5, 2022

kevinrushforth commented Aug 5, 2022

aghaisas left a comment

Choose a reason for hiding this comment

kevinrushforth commented Aug 9, 2022

openjdk bot commented Aug 9, 2022

openjdk bot commented Aug 9, 2022

kevinrushforth commented Jul 28, 2022 •

edited by openjdk bot

Loading

mlbridge bot commented Jul 28, 2022 •

edited

Loading

openjdk bot commented Jul 29, 2022 •

edited

Loading