Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache Psydac dependencies in CI #411

Merged
merged 32 commits into from
Aug 20, 2024
Merged

Cache Psydac dependencies in CI #411

merged 32 commits into from
Aug 20, 2024

Conversation

kvrigor
Copy link
Contributor

@kvrigor kvrigor commented Jun 12, 2024 •
edited
Loading

#408 observed failures on the macOS test, presumably related to the recent changes in the macOS runner. This PR will attempt to replicate this issue and fix it.

August 2024 update: As ba0f47a already contains the macOS runner fixes, the scope of this PR has shifted to caching of Psydac dependencies, e.g. PETSc, Python packages, and Ubuntu apt packages.

@kvrigor kvrigor marked this pull request as draft June 12, 2024 16:14
@kvrigor
Cache PETSc installation
d4fd26c
@kvrigor
Cache petsc4py
4180538
@kvrigor
More understandable way of setting of platform-specific FFLAGS
5f22f68
@kvrigor
Fixed wrong petsc cache dir
589c64f
@kvrigor
Cached Ubuntu packages and fixed typo
a3e047c
@kvrigor
h5py fix: always install numpy (instead of accessing its cached version)
8b71a04
@kvrigor
Workarounds for problematic cached packages: libhdf5-openmpi-dev and …
6f71e7c 
…h5py
@kvrigor
Reverted caching of pip and apt packages
16c4c5f 
Too many weird errors during psydac installation step!
@kvrigor
igakit build fix: Downgrade numpy to <2.0
e0838ba
@kvrigor
Don't lock numpy version and reenabled pip cache
5da7239 
Let igakit handle the numpy<2.0.0 requirement.
@kvrigor
Disabled pip cache
784877b 
Pip caching triggers an error at the "Check h5py installation" step:

> $ python -c "from h5py import File; print(File)"
>
>   File "h5py/_conv.pyx", line 1, in init h5py._conv
>   File "h5py/h5r.pyx", line 1, in init h5py.h5r
>   File "h5py/h5p.pyx", line 1, in init h5py.h5p
> ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
@kvrigor
Test igakit bugfix
1e09452
@kvrigor kvrigor changed the title macos-14.5 CI test fix CI test fixes + PETSc caching Jun 19, 2024
@kvrigor
Copy link
Contributor Author

kvrigor commented Jun 19, 2024 •
edited
Loading

Update 19.06

  • Fix to new macOS runner applied to settings.py
  • Implemented PETSc caching in CI. This shaves ~2.5 minutes off the total test time.

Update 24.06

kvrigor added a commit to campospinto/psydac_dev that referenced this pull request Jun 19, 2024
@kvrigor
Fixes for CI failures caused by new macOS runner version and numpy 2.0
9995e21 
See pyccel#411
This was referenced Jun 19, 2024
Closed
Merged
@kvrigor kvrigor marked this pull request as ready for review June 24, 2024 15:11
yguclu pushed a commit that referenced this pull request Jul 31, 2024
@kvrigor @yguclu
Fixes for CI failures caused by new macOS runner version and numpy 2.0
77ca184 
See #411
yguclu pushed a commit that referenced this pull request Aug 1, 2024
@kvrigor @yguclu
Fixes for CI failures caused by new macOS runner version and numpy 2.0
f65b21e 
See #411
@kvrigor kvrigor changed the title CI test fixes + PETSc caching PETSc caching Aug 7, 2024
@kvrigor kvrigor marked this pull request as draft August 12, 2024 11:39
@kvrigor kvrigor marked this pull request as ready for review August 17, 2024 12:42
@kvrigor
Copy link
Contributor Author

kvrigor commented Aug 20, 2024 •
edited
Loading

@yguclu While we're at speeding the CI workflow, do you think we should enable fail-fast: true here? Perhaps it makes sense to immediately abort all tests as soon as one fails.

psydac/.github/workflows/continuous-integration.yml

Lines 15 to 16 in fe14755

strategy:
fail-fast: false

@yguclu
Copy link
Member

yguclu commented Aug 20, 2024

@yguclu While we're at speeding the CI workflow, do you think we should enable fail-fast: true here? Perhaps it makes sense to immediately abort all tests as soon as one fails.

psydac/.github/workflows/continuous-integration.yml

Lines 15 to 16 in fe14755

strategy:
fail-fast: false

The fail-fast option applies to the entire job matrix. If it is set to true, GitHub will cancel all in-progress and queued jobs in the matrix as soon as one job fails. For example, in the case that our tests fail on Ubuntu, it will also stop the macOS jobs... This makes it harder to catch a platform-specific problem IMHO.

@kvrigor kvrigor changed the title PETSc caching Cache Psydac dependencies in CI Aug 20, 2024
@kvrigor kvrigor requested a review from yguclu August 20, 2024 12:08
yguclu
yguclu requested changes Aug 20, 2024
View reviewed changes
Copy link
Member

@yguclu yguclu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! This looks very good to me!

There is just one duplication error in a comment, which I pointed out before.

@yguclu yguclu merged commit 1f338bc into devel Aug 20, 2024
7 checks passed
@yguclu yguclu deleted the macos-runner-fix branch August 20, 2024 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yguclu yguclu yguclu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kvrigor @yguclu