[Tests] Fix test_large_job_queue occasional failure
#3023
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Michaelvll
reviewed
Jan 24, 2024
tests/test_smoke.py
Outdated
| @@ -1215,7 +1215,7 @@ def test_large_job_queue(generic_cloud: str): | |||
| f'sky launch -y -c {name} --cpus 8 --cloud {generic_cloud}', | |||
| f'for i in `seq 1 75`; do sky exec {name} -n {name}-$i -d "echo $i; sleep 100000000"; done', | |||
| f'sky cancel -y {name} 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16', | |||
| 'sleep 75', | |||
| 'sleep 150', | |||
There was a problem hiding this comment.
We should keep this small. It demonstrates how fast our job queue operates.
There was a problem hiding this comment.
What is the lowest number we can set for this?
There was a problem hiding this comment.
Tried 90s, and it successfully passed for 6 runs, and 85s failed at the second run. I changed to 90s instead.
Michaelvll
approved these changes
Jan 31, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The smoke test
test_large_job_queueoccasionally failed on a code with no related bug. After an observation I trying to debug #2434 , it is because there is not enough time for those jobs to get executed on the cluster. In a successful run, the smoke test started to query the job statuses right after job 32 was executed on the cluster. This PR fixes the problem.Tested (run the relevant ones):
bash format.shpytest tests/test_smoke.pypytest tests/test_smoke.py::test_fill_in_the_namebash tests/backward_comaptibility_tests.sh