[BUG] Windows + Webkit test fails with 'SSL peer certificate or SSH remote key was not OK'

[puagarwa]

Playwright example tests failed for webkit project when ran on azure windows VM with SSL error
Few points

• Test do pass on my local windows laptop
• Also i can set ignoreHTTPSErrors to true to ignore the issue
• I tried with some more test endpoint and it sems like it fail for some endpoints

System info

• Playwright Version: [v1.36]
• Operating System: [Windows Server 2022]
• Browser: [WebKit]
• Other info:

Source code

• I provided exact source code that allows reproducing the issue locally.

Config file

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  projects: [
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'], },
    },
});

Test file (self-contained)

test('has title', async ({ page }) => {
  await page.goto('https://playwright.dev/');

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/Playwright/);
});

test('get started link', async ({ page }) => {
  await page.goto('https://playwright.dev/');

  // Click the get started link.
  await page.getByRole('link', { name: 'Get started' }).click();

  // Expects the URL to contain intro.
  await expect(page).toHaveURL(/.*intro/);
});

Steps

• Create azure VM with windows 2022 server
• install node and npm
• npm init playwright
• npx playwirght test

Expected
Test should pass

Actual
Webkit Tests failed with SSL error
Error: page.goto: SSL peer certificate or SSH remote key was not OK
navigating to "https://playwright.dev/", waiting until "load"

[mxschmitt]

Filed upstream: https://bugs.webkit.org/show_bug.cgi?id=259767

[mxschmitt]

Looks like a workaround running this file, thats why we didn't experience it on GitHub Actions: https://github.com/actions/runner-images/blob/4ec9fdae13222a0b2b278ae1ddcdc9e8b44901ed/images/win/scripts/Installers/Install-RootCA.ps1

@puagarwa could you try doing that?

[puagarwa]

Sure, i will give a try.

[puagarwa]

@mxschmitt workaround works for us but I assume you will still followup upstream for root cause.

[mxschmitt]

Amazing, we discussed it in the team meeting and will wait for more users who run into it before we proceed with making changes to the current implementation.

[mxschmitt]

Closing for now as working as intended, for other customers who run into it, please re-file. Thanks!

Workaround if you are facing this issue:

1. Open PowerShell as an Administrator: Right-click on the Start button, search for PowerShell, right-click on it, and select "Run as administrator."

2. Allow Script Execution: By default, PowerShell restricts the execution of scripts. You can allow the execution of scripts by running the following command:

   Set-ExecutionPolicy RemoteSigned -Scope CurrentUser

   Answer 'Y' or 'A' when prompted.

3. Download the Script: You can use the Invoke-WebRequest cmdlet to download the script:

   Invoke-WebRequest -Uri "https://github.com/actions/runner-images/blob/4ec9fdae13222a0b2b278ae1ddcdc9e8b44901ed/images/win/scripts/Installers/Install-RootCA.ps1" -OutFile "Install-RootCA.ps1"

4. Run the Script: If everything looks good, you can execute the script by running:

   .\Install-RootCA.ps1

[rstarkov]

If you're going to run this workaround script on your dev machine, be aware that it disables the root certificate auto-update mechanism in your Windows install. In other words, it's a fairly invasive workaround. To be fair it also manually updates the root certificate store from Windows Update on every run.

I think I'd rather just skip webkit entirely when running tests interactively, and only run those tests in CI.

[Stan-Stani]

If you're going to run this workaround script on your dev machine, be aware that it disables the root certificate auto-update mechanism in your Windows install. In other words, it's a fairly invasive workaround. To be fair it also manually updates the root certificate store from Windows Update on every run.

I think I'd rather just skip webkit entirely when running tests interactively, and only run those tests in CI.

Any ideas of a less invasive workaround?

Ah, if I read the original post more closely, the below works for me.

 const context = await browser.newContext({
    cache: "disabled",
    ignoreHTTPSErrors: true
  })