[BUG] When different engineers compare screenshots, the resize UI of the textArea is misaligned by 1px.

[daisuke-okada]

System info

• Playwright Version: [v1.29.2]
• Operating System: [macOS 13.2, macOS 11.5.2] ※but, I didn't check other versions.
• Browser: [Chromium]
• Other info:

Source code

• [] I provided exact source code that allows reproducing the issue locally.

Config file

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'], },
    },
});

Test file (self-contained)

import { expect, test } from '@playwright/test';

test('testTitle', async ({ page }) => {
  await page.goto('url');
  await expect(page).toHaveTitle(/pageTitle/);
  await page.evaluate(() => window.scroll({ top: 0 }));
  await expect(page).toHaveScreenshot(); // problem occurred
});

Steps

• Run the test
• and then error occurred

Error: Screenshot comparison failed:
      6 pixels (ratio 0.01 of all image pixels) are different.

Expected
The test should be succeeded no matter who does it.

Actual

The following errors may or may not occur when the same test is performed by different engineers.
There are people who succeed even if the version of mac Os is different, but the error in the attached screenshot occurs

[mxschmitt]

So there are different screenshots based on the macOS version or the mac machine hardware which is used? Is one of them maybe M1 or M2 and one of them Intel?

[daisuke-okada]

@mxschmitt

Thanks! We check OS version and mac machine hardware.

but, we confused because there seems to be no correlation.

• problem did not occurred
  • A
    • OS 12.3.1
    • Intel
  • B
    • OS 13.2.1
    • M1
• problem occurred
  • C
    • OS 13.1
    • Intel
  • D
    • OS 11.5.2
    • Intel

[mxschmitt]

Its a known issue that Chromium will sometimes generate different screenshots on different macOS versions or if the hardware is different. We recommend creating all these screenshots on e.g. the CI server to ensure hardware and OS version is the same. Since its a Chromium issue, we can only recommend that or filing an issue at https://crbug.com to make them creating more consistent screenshots.