143 configyaml input validation #185
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…allation part of #151.
Codecov Report
@@ Coverage Diff @@
## main #185 +/- ##
==========================================
- Coverage 88.37% 86.74% -1.63%
==========================================
Files 25 27 +2
Lines 1591 1396 -195
==========================================
- Hits 1406 1211 -195
Misses 185 185
|
SeanBryan51
reviewed
Oct 17, 2023
SeanBryan51
reviewed
Oct 17, 2023
SeanBryan51
reviewed
Oct 17, 2023
SeanBryan51
reviewed
Oct 18, 2023
SeanBryan51
reviewed
Oct 18, 2023
SeanBryan51
reviewed
Oct 18, 2023
SeanBryan51
reviewed
Oct 18, 2023
SeanBryan51
reviewed
Oct 18, 2023
SeanBryan51
reviewed
Oct 18, 2023
SeanBryan51
requested changes
Oct 18, 2023
There was a problem hiding this comment.
Nice 😃
I've added a couple corrections on the schema but I may have missed some things, can you check the schema is consistent with the latest documentation on the config options?
I've added a few comments comments on the code in there too.
ccarouge
requested changes
Oct 18, 2023
I'll check it out and confirm. |
I'll get you an example of failed validation. Stand by. |
|
OK, the validation will throw an exception, with a list of the offending entries like so. I think it would be useful to have a CLI routine to validate a config prior to attempting to run benchcab. I'll raise an issue for that. |
|
@ccarouge, @SeanBryan51. OK, I think I have implemented all of your changes. I note the coverage has gone down by virtue of simplified config tests. Re-requesting reviews. Thanks for the feedback! |
SeanBryan51
reviewed
Oct 18, 2023
SeanBryan51
reviewed
Oct 18, 2023
There was a problem hiding this comment.
Just a small comment regarding constants in internal.py.
What sort of integration tests are you doing? E.g. I usually cook up a small bash script that verifies things are working (see #182 (comment))
ccarouge
approved these changes
Oct 18, 2023
OK, I will take a look at your integration test example. It might be worthwhile coding this up as something that can be triggered by a developer. Perhaps including a canned example in the data/tests directory? Regarding the internal.py example. It is a touch out of scope of this issue and I'm not familiar with the system enough yet to remove things wholesale. Perhaps we could raise issue(s) to clean up some of these items along the way to keep things clean. I'll get back to you with those integration tests. |
Updating with main changes.
|
Integration tests have now been added to the data/tests directory. Just a pre canned script from @SeanBryan51's example. PBS output attached to verify correct execution. |
|
Nice. I'll try run the integration test on Gadi. I suspect I'd have to replace |
SeanBryan51
requested changes
Oct 24, 2023
| @@ -0,0 +1,38 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
Should we set the file to be executable? Prior to running the script, I run chmod +x benchcab/data/test/integration.sh.
There was a problem hiding this comment.
This isn't a huge issue, so I'm happy either way (i.e. regular file vs executable).
Co-authored-by: Sean Bryan <39685865+SeanBryan51@users.noreply.github.com>
SeanBryan51
approved these changes
Oct 25, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merged PR for input validation. It appears that the merge has added all of the recent changes as "changes" on this branch - that being said, GitHub is smart enough to put only the files I've touched into the PR. Yay!
I've put you both in as reviewers as it is quite the structural change for configuration loading/validation...
Please pay particular attention to the config validation - I think I've covered all of the tests using the schema validation, but fresh eyes would really help in case I've missed anything.
Given the adjustment to the conda meta.yml, we may consider an action or local test to validate the conda install (if we don't have it already?) - I'll open an issue for this.
Any questions, please plop it in the discussion.