Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add caching to CI environment setup #320

Merged
merged 7 commits into from
Oct 17, 2022
Merged

Conversation

rhugonnet
Copy link
Contributor

@rhugonnet rhugonnet commented Oct 14, 2022

Mirroring GlacioHack/geoutils#321 in GeoUtils, this PR enables the caching of conda environment using the methods of conda-incubator/setup-miniconda and github/actions/cache to avoid long running CI times, especially on Mac and Windows.

In this PR, the functions environment_yml_nopy and diff_environment_yml are fixed for a couple issues when passing pip dependencies, that wen't unnoticed because they weren't needed in GeoUtils. I will open a PR to fix those in GeoUtils as well.

In short

CI durations before:

  • ubuntu: 15-20 min, with 3-8 min for environment setup.
  • mac: 23-30 min, with 7-15 min for environment setup.
  • windows: 23-30 min, with 7-15 min for environment setup.

CI durations after:

All OSs take 11-18 min, with less than 1 min for environment setup (when uncaching) on Linux and Mac 馃コ!
On Windows, the uncaching can take longer (up to 7 min) due to the archive extraction issue mentioned in GlacioHack/geoutils#321, that should be solved by GitHub Actions eventually.

Details

See GlacioHack/geoutils#321

Resolves #308

@rhugonnet
Copy link
Contributor Author

Ready to merge! 馃槃

Copy link
Member

@adehecq adehecq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks complex... Nice work !

@rhugonnet rhugonnet merged commit 26a3037 into GlacioHack:main Oct 17, 2022
@rhugonnet rhugonnet deleted the add_cache branch October 17, 2022 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add caching of conda environments to speed up CI
2 participants