Cache the .tox
directory on GitHub Actions
#1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This can speed up CI on pull requests significantly as we avoid re-installing all the Python packages.
It also means that new releases of our Python dependencies won't cause CI to break because CI won't be using them: it'll be using its cached versions. So this effectively pins our Python dependencies on CI. Our scheduled nightly CI run bypasses this pinning and updates the cache with the latest versions of all dependencies so that our packages do get tested against new versions of their dependencies.
The cached
.tox
directory will not be used under certain circumstances:If your PR has changed
tox.ini
(compared tomain
) then tox will automatically recreate the venvsIf your PR has changed
setup.cfg
(compared tomain
) then our tox-recreate plugin will automatically recreate the venvsWhen CI is run on a schedule (every night at midnight) rather than on a PR an
rm -rf .tox/<ENV>
command is run to delete the venv and therefore force its recreation.The recreated venv will have the latest versions of all the packages installed in it and will be cached so that future PRs retrieve the latest versions from the cache. The cache will only be updated if CI passed, GHA won't update the cache if the scheduled nightly run failed due to a problem with a new version of a dependency.
An alternative implementation of this could be if the branch is
main
(or the default branch), rather than if we're running on a schedule.If your PR's branch name ends with
-rm-tox-dir
that will trigger therm -rf .tox/<ENV>
command. This allows you to send PRs that won't use the cached.tox
dirs (for example if fixing a bug with a new version of a dependency). Such PRs will not update the cache for other PRs or for scheduled nightly runs due to how GHA's caching works: workflows running on PR branches see files cached on that branch or onmain
but they don't see files cached on other PR branches (and workflows running onmain
only see files cached onmain
)When running CI manually you can check a checkbox to enable the
rm -rf .tox/<ENV>
command. If you do this on a PR branch you won't update the cache for other PRs or scheduled nightly runs. If you do it onmain
you will.