Make NVDA DPI aware

[nvaccessAuto]

Reported by jteh on 2014-01-08 06:36
Many newer screens have higher DPI than the traditional 96 DPI. If an application isn't DPI aware, Windows will virtualise and scale the application.

Currently, NVDA isn't DPI aware. Because of the virtualisation, most of the locations we use are scaled appropriately; e.g. Win32 functions. Unfortunately, this doesn't appear to be entirely accurate, so routing the mouse often routes to the wrong position despite the scaling, causing issues such as #3703. Also, apps with custom IAccessible* implementations don't have locations virtualised.

We need to:

1. Mark NVDA as DPI aware. We can do this via the manifest, but that doesn't work for source copies, so we will probably use the user32.SetProcessDPIAware function instead.
2. Make display model handle DPI awareness. If the process isn't DPI aware, any coordinates need to be transposed.
3. Make sure our GUI can cope with DPI awareness. This could be problematic if wx has DPI awareness issues. #356 is a problem we have already despite being DPI unaware.

One other issue is that there may be apps with custom IAccessible* implementations that are DPI unaware. That will be tricky to handle. My feeling is that we should cross that bridge when a specific case arises.
Blocked by #3782
Blocking #3703

[nvaccessAuto]

Comment 3 by jteh on 2014-01-09 12:02
For display model:

• From Windows Vista to Windows 8, we need to use LogicalToPhysicalPoint.
• In Windows 8.1, we need to use LogicalToPhysicalPointForPerMonitorDPI, as LogicalToPhysicalPoint doesn't transform points any more due to additional virtualisation.
• Neither of these functions is present in Windows XP because there was no DPI virtualisation.
  Because of this complexity, I wonder whether it might be easier to handle the coordinate conversion in NVDA itself rather than in-process.

See also Writing DPI-Aware Desktop and Win32 Applications for an in-depth discussion of this high DPI stuff.

[nvaccessAuto]

Comment 4 by James Teh <jamie@... on 2014-01-16 03:05
In [b2ea588]:

Merge branch 't3758' into next

Incubates #3758.

Changes:
Added labels: incubating

[nvaccessAuto]

Comment 6 by James Teh <jamie@... on 2014-01-17 00:02
In [2919a21]:

Merge branch 't3758' into next

Incubates #3758. Fixes #3782.

[nvaccessAuto]

Comment 7 by jukesy1992 on 2014-01-28 15:48
The issue in 3703 has been fixed, but now there is another problem.
In apps like Steam or Demul, that don't really have keyboard focus, if I try routing the mouse to an object with nvda plus slash, the route jumps to the desktop. In Demul's case this means I can't click the file icon, and in Steam's case it means I can't install any new games without sighted help.
The easiest way to see this in action is to try downloading the Demul dreamcast emulator, running it, go into screen review, find file, try routing the mouse to it.
NVDA should normally say file alt plus f when you do this, however since updating to the latest NEXT snapshot the cursor gets confused and just routes itself to the desktop. I tried clicking it and it clicked a desktop icon.
So, weirdly, it's kind of become a reverse situation. I can now click system tray icons but I now can't route the mouse in apps that need it.

[nvaccessAuto]

Comment 8 by jteh on 2014-01-28 22:06
What operating system is this? I'm guessing Windows 8.1 based on #3703.

My guess is that the application isn't DPI aware, but I introduced code which should work around that for screen review and it does seem to work for me in some applications i tested. I'll need to investigate further.

[nvaccessAuto]

Comment 9 by jukesy1992 on 2014-01-29 15:38
Operating system is windows 8.1.

[nvaccessAuto]

Comment 10 by jteh on 2014-01-29 22:06
Investigation reveals that the app isn't DPI aware, but for some reason, Windows refuses to translate coordinates for this app. We don't know why yet. It does work in most apps that aren't DPI aware, though.

[nvaccessAuto]

Comment 11 by jukesy1992 on 2014-01-30 17:17
Is this refering to Demul or Steam?

[nvaccessAuto]

Comment 12 by jteh on 2014-01-30 21:58
Demul, but if Steam behaves in a similar way, it is very probably the same situation.

[nvaccessAuto]

Comment 13 by jukesy1992 on 2014-02-05 12:08
|What should we do about apps such as Demul, if there is anything that can be done?

[nvaccessAuto]

Comment 14 by James Teh <jamie@... on 2014-02-11 03:28
In [d0ecf33]:

On systems with a high DPI display setting (which occurs by default for many modern screens), NVDA no longer routes the mouse to the wrong location in some applications.

NVDA was made DPI aware to fix this.
Fixes #3758, #3703.

Changes:
Removed labels: incubating
State: closed

[nvaccessAuto]

Comment 15 by jteh on 2014-02-11 03:32
Changes:
Milestone changed from next to 2014.1

[nvaccessAuto]

Comment 16 by jteh (in reply to comment 13) on 2014-02-11 04:53
Replying to jukesy1992:

|What should we do about apps such as Demul, if there is anything that can be done?

As far as we can tell, this seems to be a bug in Windows. If we can think of a reasonable solution, we'll implement it, but right now, we haven't come up with anything. I've created #3875 to address this. You should be able to access some of what you need using object navigation; e.g. the menus in Demul.