Respect scale factor on recent Windows versions #52
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
... at least using MSVC 2019. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Due to an off-by-one, the full width or height would have been rejected, even as the block below would fall back to said full width and height. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Recent Windows versions allow to change the "Scale" factor of the display. This is particularly useful on high DPI devices, where displaying windows in the regular pixel size would make it _really_ hard to read (think: 3240x2160 resolution on a regular laptop, where the Scale factor would default to 200%). To maintain compatibility with scale-unaware software, this is an opt-in feature, where applications have to specify via their manifest that they are aware of it. Otherwise, Windows will pretend that the resolution is actually lower. However, this leads to a problem where calling `screen.capture()` _thinks_ that the resolution on above-mentioned laptop is actually 1620x1080, yet when capturing a 1620X1080 rectangle at 0x0, it only captures the upper left quadrant! Let's capture the full screen and downsize it using the current Scale factor. Note: an alternative would be to report the high resolution as desktop size. However, this is incompatible with `screen.highlight()` and `mouse.move()` which are unaware of the scale factor. The easier direction, therefore, seems to be the chosen one: just downscale the high-resolution screen capture and continue to work in the scale-unaware mode. Note, too: yet _another_ alternative would be to change the manifest. But I think that would not work because it would be `node.exe` whose manifest would have to change, this Node module's `.dll` cannot change the application's behavior. For now, we do not bother trying to fall back to using `GetScaleFactorForDevice()` on Windows 8 (`GetScaleFactorForMonitor()` is only available in Windows 8.1 and newer). To allow _running_ on Windows 8 (or older), we only try to load that function at runtime rather than linking to it (because the `.dll` would otherwise not even load on Windows 8). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho
added a commit
to dscho/nut.js
that referenced
this pull request
Feb 15, 2021
When capturing a screen region in a recent Windows version on a high-DPI device, the captured image can have a much higher resolution than scale-unaware applications such as `node.exe` might think (compare also nut-tree/libnut-core#52). For convenience, let's support scaling the "needle" image that `nut.js` is told to find. For example, on my laptop, where the scale factor is 200%, I would capture a rectangular region, save it as a `.png` file, then tell `nut.js` to find it thusly: await screen.find('needle.png', 10000, { scaleNeedle: true }) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
|
The CI failure seems to be a time-out downloading Node, not a bug in my code: |
s1hofmann
requested changes
Mar 4, 2021
There was a problem hiding this comment.
Hi @dscho 👋
Please refer to nut-tree/nut.js#211 (review) regarding the overall review
|
Closing this in favor of #59 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On my laptop, the scale factor is 200% (scale-unaware applications will think that my 3240x2160 display is a 1620x1080 one). One of those scale-unaware applications is
node.exe. Ascreen.capture(path)would therefore capture only the upper-left quadrant of my screen, andnut.jswould therefore be unable toscreen.find()anything outside that area.This PR fixes that by scaling the captured screen, also pretending that there is no scale factor involved.
This fixes nut-tree/nut.js#183.