[BUG] New Playwright releases break GitHub Actions cache

[ericmatthys]

Context:

• Playwright Version: Latest (14.1 now)
• Operating System: Linux (GitHub Actions Ubuntu runners)
• Node.js version: 16x
• Browser: Chromium

Code Snippet

GitHub Actions setup steps:

jobs:
  test:
    name: Run tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        if: ${{ github.event_name != 'pull_request' }}

      - uses: actions/checkout@v3
        if: ${{ github.event_name == 'pull_request' }}
        with:
          ref: ${{ github.event.pull_request.head.sha }}

      - uses: pnpm/action-setup@v2.2.2
        with:
          version: 7.1.7

      - uses: actions/setup-node@v3
        with:
          node-version: '16'
          cache: 'pnpm'

      - name: Install packages
        run: pnpm install --frozen-lockfile

      - name: Install Playwright
        run: pnpx playwright install --with-deps chromium

Describe the bug

When a new Playwright version is released, it seems like we run into problems with our GitHub Actions cache. Maybe the timing is coincidental and it's not actually related, but I think it probably is. We'll see errors like this:

browserType.launch: Executable doesn't exist at /home/runner/.cache/ms-playwright/chromium-1012/chrome-linux/chrome%0A

If I delete cache entries via the GitHub API and retry, this will fix the issue. If I upgrade Playwright to the latest, this will also fix the issue until a new version comes out again. I think this could be more of a GitHub Actions issue / question, but I'm asking here in case there's a problem with the setup steps that someone can spot. It is strange that the cache could be seemingly affected by a new release before we ever even open up a PR with that upgrade.

[pavelfeldman]

I can't think of anything on our side that would cause it. You do run pnpx playwright install --with-deps chromium, which is installing the browser. That action should have a log saying whether it installed browser at the path mentioned in the error message.

[ericmatthys]

Could it be that pnpx is fetching the latest, which may not match the playwright version in package.json? Perhaps we should be using pnpm exec instead.

Run pnpx playwright install --with-deps chromium
.../share/pnpm/store/v3/tmp/dlx-1[8](https://github.com/plexinc/website/runs/7545740746?check_suite_focus=true#step:7:9)03     | Progress: resolved 1, reused 0, downloaded 0, added 0
.../share/pnpm/store/v3/tmp/dlx-1803     |   +2 +
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: /home/runner/.local/share/pnpm/store/v3

[ericmatthys]

That would explain the coincidental timing with new releases. Thanks for the response. I'll keep an eye on it with this change.

         run: pnpm install --frozen-lockfile
 
       - name: Install Playwright
-        run: pnpx playwright install --with-deps chromium
+        run: pnpm exec playwright install --with-deps chromium
 
       - name: Check types
         run: pnpm run tsc