[Question]: How to cache on github actions? #7249
Comments
|
General recommendation is to not cache them, since they are downloaded in less than 10 seconds. If you are still want to cache them, you find the locations here. The cache key should contain the OS and Playwright version. |
|
It's taking ages to install playwright. But on the actual tests run step, I get the following error: When I list the folder, I can actually see the files but with Is there a way to install a certain webkit version? |
|
Are you using self-hosted runners? If this problem is persistent on the hosted-runners, it might make sense to contact the GitHub Actions support. We run 50-100 jobs daily and didn't experience such big delays yet. Also you are using a private NPM Registry, which might add delay. But the browsers are not getting downloaded over NPM, thats a normal http request then. Running |
|
We are using the gha runners. We have no self hosted stuff. The strange thing is - the Mac machines run consistently slower in everything. The tests run roughly 50% longer and running Anyway, I've managed to solve the issue. On the Mac machine running the tests I've added this code: Now it has a cache for playwright only. I runs a lot faster than the actual After the change: I will check what might be the issue with gha on mac for us as we thought it is a common issue for everyone. Thanks a lot! |
|
@YonatanKra 8 minutes! This is too much! We ourselves run installation on GHA-based macs, and it's never that bad. The whole Is it a public repo? Would you mind sharing a link to the GHA action so that I can look into logs? |
|
Closing since the issue seems stale as part of the triage process. Please create a new issue if you still face any issues. 8 Minutes is definitely not as-designed but sounds more a infra issue. |
Hi, |
|
I extended the above caching logic to support new browser binaries as the version of Playwright changes. This works for - name: Install dependencies
run: yarn
# Figures out the version of playwright that's installed.
# 1. Because we don't know what version yarn will resolve it to, we have
# to use `yarn why` to get the actually installed version.
# 2. Because we're in a workspace, we need to make sure we get the version
# for the root and not any children, hence the `grep`. If not using
# workspaces, this can be skipped.
# 3. jq comes pre-installed in the Ubuntu runner, so we use that to get
# the correct version string.
# 4. Finally, we use sed to extract just the version number (eg; '1.22.0')
# The result is stored in steps.playwright-version.outputs.version
- name: Get installed Playwright version
id: playwright-version
run: echo "::set-output name=version::$(yarn why --json @playwright/test | grep -h 'workspace:.' | jq --raw-output '.children[].locator' | sed -e 's/@playwright\/test@.*://')"
# Attempt to restore the correct Playwright browser binaries based on the
# currently installed version of Playwright (The browser binary versions
# may change with Playwright versions).
# Note: Playwright's cache directory is hard coded because that's what it
# says to do in the docs. There doesn't appear to be a command that prints
# it out for us.
- uses: actions/cache@v3
id: playwright-cache
with:
path: '~/.cache/ms-playwright'
key: '${{ runner.os }}-playwright-${{ steps.playwright-version.outputs.version }}'
# As a fallback, if the Playwright version has changed, try use the
# most recently cached version. There's a good chance that at least one
# of the browser binary versions haven't been updated, so Playwright can
# skip installing that in the next step.
# Note: When falling back to an old cache, `cache-hit` (used below)
# will be `false`. This allows us to restore the potentially out of
# date cache, but still let Playwright decide if it needs to download
# new binaries or not.
restore-keys: |
${{ runner.os }}-playwright-
# If the Playwright browser binaries weren't able to be restored, we tell
# paywright to install everything for us.
- name: Install Playwright's dependencies
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps |
|
It might be my version of yarn but echo "::set-output name=version::$(cat yarn.lock | grep "^\"@playwright\/test" --after-context 1 | tail -n 1 | sed 's/\s*version "\([0-9.]\+\)"/\1/')"Furthermore, I found that while the above caches the downloaded versions, the playwright runtime dependencies still need to be installed. Hence, I ended up with the following for the last step: - name: Install Playwright with dependencies
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright's dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps |
This is crossplatform version of this: node -e "console.log(require('playwright/package.json').version)" |
|
@NikolayMakhonin that only works if |
I think we would need to use
{
"version": "0.1.0",
"name": "your-project-name",
"dependencies": {
"@playwright/test": {
"version": "1.24.0",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.24.0.tgz"
}
}
} |
|
@andykenward |
@chrisvariety Thank you. That's working for me.
|
|
@jesstelford Thanks for sharing your config! Just like @birtles, I'm wondering how this manages installing / caching the ubuntu dependencies. If I just cache |
|
Hmm, I'm not sure about how GitHub Actions caches OS deps. Perhaps if it's done in a seperate step, GitHub will cache the generated Docker image, making subsequent installs faster? |
You can separate the two parts and cache the binaries but still install the host system dependencies. https://justin.poehnelt.com/posts/caching-playwright-in-github-actions/ |
|
if someone needs to set a cleaner version using npm: output |
|
does any of this work anymore??? I've tried and it just keeps saying cache not found nor does it ever create it..... |
|
Also wondering what @gselsidi asked. |
This worked for us after seeing some timeouts installing playwright today. Confirmed we're hitting the cache in our github actions. - name: Cache Playwright
uses: actions/cache@v3
id: playwright-cache
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-1.26.1
- name: Install Playwright
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: pnpm dlx playwright@1.26.1 install --with-depsyou could probably use the suggestion here to make it a little more robust and replace the hardcoded version # with whatever version you have in |
|
I got it to work @zkrzyzanowski thing is how can we cache all the ubuntu library stuff? What's the path for that? This still takes massive amounts of time:
|
|
Looks like all this caching stuff is pretty annoying and should probably use containers, I tried the default container and it's done super quickly. |
We had the ubuntu stuff download on the first run, then pulled from cache every run after. Either way it sounds like you found a solution based on your comment here |
|
@rafaelbiten thanks |
|
@rafaelbiten This doesn't seem to cache anymore. I don't see anything wrong with your logic, though. Any chance you can take a look? |
|
@ArmandAgopian I don't know if you're still having issues with this, but I just checked and the cache is still working on my side. From the example I left, make sure you have the path to be cached properly set ( My first run, the
Second run, it pulled from cache and skipped that step altogether:
And you should see it cached: |
|
@rafaelbiten Hi, after copying your code again it works now. Weirdly enough there's no difference between the original and now copied code, I'm not sure what went wrong. Thank you though, it works perfectly! |
|
This isn't working for me. I've used your setup @rafaelbiten and my cache got properly restored but I still get |
|
I had to use this part of @jpoehnelt's suggestion: - run: npx playwright install --with-deps
if: steps.playwright-cache.outputs.cache-hit != 'true'
- run: npx playwright install-deps
if: steps.playwright-cache.outputs.cache-hit == 'true'While this still includes a |
|
@electrovir solution is required if you need to run your tests on Webkit as it needs tons of OS level deps which are not cached |
|
Wish there was a straightforward solution for GitHub actions via After reading this entire thread I've decided to try just using the Playwright container from Microsoft, which is helpfully documented here: https://playwright.dev/docs/ci#via-containers |
|
Yes, same here using Playwright container in github and it's working like a charm 🎉 for all browsers including webkit: |
Thanks a lot :) |
|
Just my 2 cents for getting playwright version in a very simple and short manner: npx playwright -V | awk '{ print $2 }'Or you can use |
|
@avetisk |
|
Yarn edition with disabled colors because of the following phenomenon:
- name: Store Playwright's Version
run: |
PLAYWRIGHT_VERSION=$(YARN_ENABLE_COLORS=false yarn info @playwright/test | grep @playwright | sed 's/.*://')
echo "Playwright's Version: $PLAYWRIGHT_VERSION"
echo "PLAYWRIGHT_VERSION=$PLAYWRIGHT_VERSION" >> $GITHUB_ENV
- name: Cache Playwright Browsers for Playwright's Version
id: cache-playwright-browsers
uses: actions/cache@v3
with:
path: ~/.cache/ms-playwright
key: playwright-browsers-${{ env.PLAYWRIGHT_VERSION }}
- name: Setup Playwright Browsers
if: steps.cache-playwright-browsers.outputs.cache-hit != 'true'
run: yarn playwright install --with-deps |
|
Hmm it is not working.. Edit: solution was here: #7249 (comment) |
|
I confirmed this script works for my case: - name: Get Playwright version
run: echo "PLAYWRIGHT_VERSION=$(jq '.devDependencies["@playwright/test"]' package.json -r)" >> $GITHUB_ENV
- name: Cache Playwright Browsers
id: cache-playwright-browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-browsers-${{ runner.os }}-${{ env.PLAYWRIGHT_VERSION }}
- name: Install Playwright with dependencies
if: steps.cache-playwright-browsers.outputs.cache-hit != 'true'
run: bun playwright install --with-deps
- name: Install Playwright dependencies
if: steps.cache-playwright-browsers.outputs.cache-hit == 'true'
run: bun playwright install-deps
- name: Download test build
uses: actions/download-artifact@v4
with:
name: test-build
path: dist/
- name: Run Playwright tests
run: bun playwright test -c playwright.service.config.ts --workers=20
env:
DOTENV_KEY: ${{ secrets.DOTENV_KEY }}
PLAYWRIGHT_SERVICE_ACCESS_TOKEN: ${{ secrets.PLAYWRIGHT_SERVICE_ACCESS_TOKEN }}
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
PLAYWRIGHT_SERVICE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ github.sha }}
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
path: playwright-report/ |
Citation needed ;) I don't want to sound entitled, but it would really be great if we had a first-party |
Can someone more clearly explain why both of these are needed? I think the documentation makes it seem like those commands should be equivalent... Shouldn't it be sufficient to just do I'm quite confused. Perhaps the docs could be updated to clarify exactly all the different pieces involved (playwright library itself, browsers, browser deps, playwright os deps, whatever), and which get installed (and where) with different commands. |
|
I tried to get the browsers caching working on Ubuntu but ran into errors regarding missing OS-level libs so living without caching for now. The docs state that downloading browsers is as fast as loading them from cache; it takes 1 minute to finish just the Update: After a couple of attempts I'm continuing with the Docker image provided by Playwright. It just does not make sense to manually do the caching. Also found this: https://radekmie.dev/blog/on-playwright-in-github-actions/ |
and others
Hi,
I've setup github actions on Linux and it works fine.
I'm now running the same tests on MacOS and the binaries are not the same, so caching the installation is not working (for obvious reasons).
What is the recommended way to cache the playwright binaries for MacOS on github actions?
The text was updated successfully, but these errors were encountered: