Skip to content

screen.find neglects offsets when providing a search region #160

@s1hofmann

Description

@s1hofmann

Version
1.4.3-next.20200813090951

Short overview
When providing a search region to screen.find the resulting Region is off.

Issue occurs on

  • Virtual machine
  • Docker container
  • Dev/Host system

Detailed error description
When providing a search region to screen.find, the returned Region is relative to the provided search region.
It does not incorporate offsets provided by the search region, so the result does not properly translate onto desktop space.

Steps to reproduce error

  • Define a new Region containing your target image, e.g. const searchRegion = new Region(300, 200, 500, 500);
  • Search for the target image, providing a search region: const detectedRegion = await screen.find("target_image.png", {searchRegion})
  • Highlight the detected region: highlight(detectedRegion)
  • -> Highlighted region is off by x and y coordinates of our searchRegion
  • Add the offset to the result region:
const correctedRegion = new Region(searchRegion.left + detectedRegion.left, searchRegion.top + detectedRegion.top, detectedRegion.width, detectedRegion.height);
highlight(correctedRegion);
  • -> Correct region is highlighted
    Additional content

Please provide any (mandatory) additional data to reproduce the error (Dockerfiles etc.)

const { screen, Region } = require("@nut-tree/nut-js");

(async () => {
    const searchRegion = new Region(300, 400, 500, 500);
    const detectedRegion = await screen.find("image.png", {searchRegion})
    await screen.highlight(detectedRegion);
    const correctedRegion = new Region(
        searchRegion.left + detectedRegion.left,
        searchRegion.top + detectedRegion.top,
        detectedRegion.width,
        detectedRegion.height
    );
    await screen.highlight(correctedRegion);
})();

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions