Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin max versions of all major dependencies for future compatibility #462

Merged
merged 14 commits into from Feb 1, 2024

Conversation

rhugonnet
Copy link
Contributor

@rhugonnet rhugonnet commented Jan 31, 2024

Pinning max major version only for future compatibility when they release a new version, we shouldn't have to update the max versions very often (and when we do, our functionalities will very likely require updating too because of breaking changes of major versions, so it's beneficial in any case! 馃槃 ).

Fixing pytest to 7.* to avoid errors from pytest extension packages (lazy fixtures).

EDIT: The change below is done in #458
Also mirroring the script update in xDEM to parse package versions in CI to avoid failure in Windows and be more flexible in the future.
Reminder of the original reason we have to do all this: Unfortunately the Python version cannot be pinned easily when building a conda cached environment in CI from the command line options (and having a cache save us up to 15min of build time on Mac/Windows, so we keep that for sure!). To circumvent the issue, we need to re-create a temporary YAML file that pins the python version (previously done by parsing the YAML argument into command line, but failed on Windows for symbols such as < that we need to pin max versions).

Resolves #426

Copy link
Member

@adehecq adehecq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, this should make things more stable in the longer term. But is there an easy way to test whether or not a newer version that specifies exist? Apart from checking each module individually...
Should we have an environment file with no max version pinned, to be run as tests occasionally?

Arf, segmentation faults are not nice... Is it related to this PR only?

@rhugonnet
Copy link
Contributor Author

Found the cause, pytest just released 8.0 and it breaks things in pytest-lazy-fixtures... Because they did not pin their max major version of pytest!
This is very much in the theme of this PR 馃槄

@rhugonnet
Copy link
Contributor Author

rhugonnet commented Jan 31, 2024

Actually, I realized that the parsing of versions such as rasterio>1.3,<2 doesn't work on CI with Windows because of command line package version parsing issues.
This was fixed in xDEM in GlacioHack/xdem#439 by generating a temporary YAML during CI instead of parsing the package versions from the command line when building the mamba environment, I'm mirroring the changes here.

@rhugonnet
Copy link
Contributor Author

Made a mistake, mirroring the xDEM change in #458 instead. Doesn't change much, it's independent from the changes here.

@rhugonnet
Copy link
Contributor Author

Looks like it's a dependency problem upstream from testing packages...
Let's wait it out a couple days and see again

@rhugonnet
Copy link
Contributor Author

rhugonnet commented Feb 1, 2024

Finally found it!
Two causes: #473, and the latest geopandas, I forced an earlier version. Opened #474.

@rhugonnet rhugonnet merged commit cde8ea6 into GlacioHack:main Feb 1, 2024
10 checks passed
@rhugonnet rhugonnet deleted the pin_max_versions branch February 1, 2024 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pin max major version number for dependencies to ensure durable compatibility
2 participants