Use globally insatlled installed packages for GPU tests #1011
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Documentation preview |
edknv
changed the title
edknv
commented
Mar 1, 2023
| @@ -15,9 +15,9 @@ commands = | |||
| deps = | |||
| -rrequirements/test.txt | |||
| setenv = | |||
| CUDA_VISIBLE_DEVICES=0 | |||
There was a problem hiding this comment.
Without setting CUDA_VISIBLE_DEVICES=0, we get an error:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Device ordinals must be set for all virtual devices or none. But the device_ordinal is specified for 1 while previous devices didn't have any set.
We didn't have to set this with TF 2.9. Is it specific to TF 2.10? Is there a better way to handle this?
There was a problem hiding this comment.
If would like to run these tests with only 1 GPU, we can also change the runner we use in the workflow config. Currently set to 2GPU
models/.github/workflows/gpu-ci.yml
Line 15 in e2276f4
edknv
commented
Mar 1, 2023
Comment on lines
-65
to
-66
| _ = model.evaluate(valid) | ||
|
|
There was a problem hiding this comment.
Removed because this is unnecessary for these tests, and evaluate() is already tested in one of the other unit tests.
marcromeyn
approved these changes
Mar 1, 2023
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For GPU tests, we should use globally installed packages as we don't install cudf, cupy, etc. in the test environment. By setting
sitepackages=truein tox, tox will use the packages in the CI container if not available in the environment. We hadsitepackages=truebefore but it got dropped at some point. This PR restoressitepackages=truein the tox testpy38-gpu.