SlurmScheduler: Parse the NODE_FAIL state
#5866
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If a job fails due to a node failure, SLURM will set the job's state to `NODE_FAIL`. The `SlurmScheduler.parse_output` method is updated to check for this state, in which case the `ERROR_SCHEDULER_NODE_FAILURE` exit code is returned. This is a new exit code defined on the `CalcJob` base class.
ltalirz
approved these changes
Jan 23, 2023
tests/schedulers/test_slurm.py
Outdated
| @@ -431,6 +431,20 @@ def test_parse_out_of_memory(): | |||
| assert exit_code == CalcJob.exit_codes.ERROR_SCHEDULER_OUT_OF_MEMORY # pylint: disable=no-member | |||
|
|
|||
|
|
|||
| def test_parse_node_failure(): | |||
| """Test that `ERROR_SCHEDULER_NODE_FAILURE` code is returne if `STATE`.""" | |||
There was a problem hiding this comment.
Suggested change
| """Test that `ERROR_SCHEDULER_NODE_FAILURE` code is returne if `STATE`.""" | |
| """Test that `ERROR_SCHEDULER_NODE_FAILURE` code is returned if `STATE == 'NODE_FAIL'`.""" |
sphuber
force-pushed
the
feature/5865/node-failure-parsing
branch
from
e268e54
to
65ff640
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #5865
If a job fails due to a node failure, SLURM will set the job's state to
NODE_FAIL. TheSlurmScheduler.parse_outputmethod is updated to check for this state, in which case theERROR_SCHEDULER_NODE_FAILUREexit code is returned. This is a new exit code defined on theCalcJobbase class.