Screenshots in macOS Safari: Unable to locate the page area - mark not found.

[htho]

What is your Scenario?

I do visual regression tests. Now I need to test Safari as well.

What is the Current behavior?

When I run the tests, the screenshot is not cropped - so the window chrome (frame, browser-bars etc.) is still included, and the mark is visible in the bottom right corner.

Test "Screenhot" - in Safari

What is the Expected behavior?

The screenshots should be cropped and not include the mark. Also there should be no warning in the log. Like for example in Chrome on macOS.

Test "Screenhot" - in Chrome

What is the public URL of the test page? (attach your complete example)

I created a repository at https://github.com/htho/testcafe-repro-screenshot-mark-macos-14

What is your TestCafe test code?

import { fixture, test } from "testcafe";

const allowedChars = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ-_";
export function mkValidPath(path: string): string {
	return path.split("").map(char => allowedChars.includes(char) ? char : "_").join("");
}

fixture("screenshot")
    .page("about:blank");

test("screenshot", async t => {
    await t.eval(() => document.body.style.backgroundColor = "#dde")
    await t.resizeWindow(640, 480);
    await t.takeScreenshot(mkValidPath(`${t.browser.alias}--${t.test.name}`));
});

Your complete configuration file

none

Your complete test report

% npm run test

> testcafe-repro-screenshot-mark-macos-14@1.0.0 test
> testcafe safari screenshot.tc.ts

(node:5212) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 Running tests in:
 - Safari 17.3.1 / Sonoma 14

 screenshot
 ✓ screenshot (screenshots: /Users/xxx/testcafe-repro-screenshot-mark-macos-14/screenshots/safari--screenshot.png)


 1 passed (3s)

 Warnings (1):
 --
  Unable to locate the page area in the browser window screenshot at /Users/xxx/testcafe-repro-screenshot-mark-macos-14/screenshots/safari--screenshot.png, because the page area
  mark with ID 3194121322 is not found in the screenshot.

Screenshots

No response

Steps to Reproduce

1. % git clone https://github.com/htho/testcafe-repro-screenshot-mark-macos-14.git
2. % cd testcafe-repro-screenshot-mark-macos-14
3. % npm i
4. % npm run test

TestCafe version

3.5.0

Node.js version

21.6.2

Command-line arguments

testcafe safari screenshot.tc.ts

Browser name(s) and version(s)

Safari 17.3.1

Platform(s) and version(s)

macOS 14.3.1

Other

No response

[htho]

I assume it has something to do with macOS rounded corners.
In https://github.com/DevExpress/testcafe/blob/master/src/screenshots/constants.js
I increased MARK_RIGHT_MARGIN from 10 to 15 and it solved the problem.

But I don't know why there is no such problem in Chrome although it also has rounded corners in macOS.
However the Chrome screenshot does not have rounded corners.

[PavelMor25]

Hello @htho,

TestCafe allows you to capture screenshots of the application at any point during a test run or when a test fails - this helps in debugging. However, please note that TestCafe does not include a scenario for visual regression tests. As a result, we cannot guarantee consistent screenshots across different versions.

In theory, you can use TestCafe for screenshot testing, but since Safari and Chrome are two completely different browsers, their HTML rendering and screenshot mechanisms differ. This leads to visual discrepancies. We recommend comparing screenshots within the same browser, not across different ones.

At this moment, we're unable to prioritize the task of adjusting the UI on screenshots for Safari as we have other priorities with our tasks. In the future, if this issue becomes more prominent, we will revisit the consideration of this improvement.

[htho]

Hello @PavelMor25

I am aware that screenshots from two different browsers can not simply be compared.
This bug is not about differences between Chrome and Safari. This issue is about Safari on macOS, where TestCafe produces an error because it can not find the screenshot mark. Chrome is only cited to show how it should be.

Since there is the screenshot-feature in TestCafe, it makes sense for it to work - right?

Anyway, I am willing to create a PR for this.
Would you accept any PR regarding this, although it is not your priority right now?

[PavelMor25]

@htho,

We are always happy to consider all proposed changes. You can submit a pull request and include a description. We will review it and accept it if everything is okay.

Before creating the fix, please take a look at our contributing guide

[htho]

fixed by #8161