Skip to content

Commit

Permalink
ci: Speed up pip install by caching deps install output
Browse files Browse the repository at this point in the history
Based on the benchmark, it speeds up by about 2x for the Ubuntu instances. While extracting the cache tarball for the Windows instance could be further sped up, with a `tar` binary that supports `zstd`.
See rht#12.

The cache should be automatically cleared every month. Or alternatively, create a lockfile (either via Poetry of `pip-tools`)  that is automatically updated via Dependabot. Regardless, this is out of the scope of this PR.
  • Loading branch information
rht authored and tpike3 committed Dec 2, 2023
1 parent 76229a5 commit 838e216
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 3 deletions.
24 changes: 21 additions & 3 deletions .github/workflows/build_lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,30 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- if: ${{ runner.os == 'Windows' }}
# This is needed so that restoring cache on Windows is fast.
# See until https://github.com/actions/cache/issues/752 is resolved.
name: Use GNU tar
shell: cmd
run: |
echo "Adding GNU tar to PATH"
echo C:\Program Files\Git\usr\bin>>"%GITHUB_PATH%"
- uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('Pipfile.lock') }}
path: ${{ env.pythonLocation }}
key: ${{ runner.os }}-pip-${{ env.pythonLocation }}-${{ hashFiles('setup.py') }}
- name: Install dependencies
run: pip install wheel && pip install .[dev]
# Only if the cache misses
# Based on https://github.com/pypa/pip/issues/8049#issuecomment-633845028
# read_requirements.py should be removed once
# https://github.com/pypa/pip/issues/11440 is resolved.
if: steps.cache.outputs.cache-hit != 'true'
run: |
pip install toml
python tests/read_requirements.py > requirements.txt
pip install -r requirements.txt
- name: Install Mesa
run: pip install --no-deps .
- name: Test with pytest
run: pytest --cov=mesa tests/ --cov-report=xml
- if: matrix.os == 'ubuntu'
Expand Down
8 changes: 8 additions & 0 deletions tests/read_requirements.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import toml

# This file reads the pyproject.toml and prints out the
# dependencies and dev dependencies.
# It is located in tests/ folder so as not to pollute the root repo.
c = toml.load("pyproject.toml")
print("\n".join(c["project"]["dependencies"]))
print("\n".join(c["project"]["optional-dependencies"]["dev"]))

0 comments on commit 838e216

Please sign in to comment.