-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make the config versioning/compatibility independent of the butido version and improve the config checks #334
Merged
christophprokop
merged 10 commits into
science-computing:master
from
primeos-work:improve-config-versioning-logic
Jan 11, 2024
Merged
Make the config versioning/compatibility independent of the butido version and improve the config checks #334
christophprokop
merged 10 commits into
science-computing:master
from
primeos-work:improve-config-versioning-logic
Jan 11, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ideally, the butido version should be bumped when there are breaking changes but this new approach offers the following advantages: - The compatibility setting doesn't need to be changed when we release a new butido version (e.g., with breaking changes that are unrelated to the configuration). - An independent configuration version is more meaningful: We can update it immediately when we add a breaking change (no need to make a new release), inform the user of the necessary changes, and don't "have" to bundle breaking changes. - One minor issue was that, e.g., configuration version 0.4.1 wouldn't be compatible with butido 0.4.0 (but that can be avoided by always setting `compatibility` to `x.y.0`, which is compatible to butido version `x.y.*`). - The new type is stricter (`semver::VersionReq` is a SemVer version requirement and can be "*", ">=1.2.3, <1.8", etc.). This fixes science-computing#187. PS: We should also start collecting "upcoming" breaking/relevant changes in the changelog so that we don't have to go through the commit history (all of the commits since the last tag) when preparing a new release. PPS: I chose to start with a `compatibility` setting of `1` so that we could use `0` for a "smoother transition", if necessary. Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
Otherwise the config loading (type checking) will likely fail before we can output the appropriate error message with the required changes. Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
This should make it much easier to update butido's configuration, if required after a butido update due to breaking configuration changes. Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
This is important to avoid unnecessary runtime failures (those could easily get missed as the relevant code is conditional). Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
It'd be embarrassing if the example config file wouldn't be valid so let's better add a test for it. Skip the filesystem checks (if the Butido directories exist) when validating the example configuration for the test. Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
I did previously set `compatibility=0` as I expected that this configuration file could easily become outdated (i.e., to signal it isn't really up-to-date) but with this test we can finally ensure that it'll remain valid :) Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
This helps catching errors/typos in butido's configuration as well as to cleanup settings that don't exist anymore or got renamed. It already revealed a misnamed key in our example configuration file (that caused the corresponding test to fail)! We already implemented this for package definitions in b075e97. The error output contains the name of the unknown field and a list of the valid/expected fields. Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
I already made a mistake (accidentally introduced a `!` - luckily it was caught by CI for other reasons) in one of the three checks when introducing `skip_filesystem_checks`. This simplifies the code and ensures that all checks behave the same and I already noticed another mistake that I'll fix next (not part of this commit to keep it a refactoring). Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
Otherwise a missing log directory would result in an error at the end of a build command (despite the build being successful). Signed-off-by: Michael Weiss <michael.weiss@eviden.com>
primeos-work
force-pushed
the
improve-config-versioning-logic
branch
from
January 8, 2024 18:55
f7cd656
to
34716b1
Compare
christophprokop
approved these changes
Jan 11, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested. Thank you!
Merged
via the queue into
science-computing:master
with commit Jan 11, 2024
8943e61
13 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
tl;dr:
Note: This makes breaking configuration changes that require users to update their configuration (should be trivial though as one only has to set
compatibility=1
and remove invalid settings, if present).Example outputs
Before this PR:
If config upgrades are required:
Trying to load an old configuration:
Loading a configuration with invalid settings:
Invalid env vars: