add sharding option to test framework #45988
Conversation
janeyx99
changed the title
There was a problem hiding this comment.
@janeyx99 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
| docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda9.2-cudnn7-py3-gcc7" | ||
| - pytorch_linux_test: | ||
| name: pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc7_test1 | ||
| name: pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc7_test |
There was a problem hiding this comment.
is cu92 significantly faster than cuda10/11?
There was a problem hiding this comment.
I'm not sure, but that particular test is not run in CI (not marked important in the CONFIG TREE)
There was a problem hiding this comment.
It is run in CI, but only on master (i.e. important tests run on both master and PRs, while others run only on master), but we don't need to shard it, as users usually do not wait for the test results on master.
janeyx99
force-pushed
the
add-sharding-as-option
branch
from
a5787f1 to
7741264
Compare
There was a problem hiding this comment.
@janeyx99 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
Codecov Report
@@ Coverage Diff @@
## master #45988 +/- ##
=======================================
Coverage 68.25% 68.25%
=======================================
Files 410 410
Lines 53246 53246
=======================================
Hits 36343 36343
Misses 16903 16903 Continue to review full report at Codecov.
|
| docker_image: "308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-xenial-cuda9.2-cudnn7-py3-gcc7" | ||
| - pytorch_linux_test: | ||
| name: pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc7_test1 | ||
| name: pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc7_test |
There was a problem hiding this comment.
It is run in CI, but only on master (i.e. important tests run on both master and PRs, while others run only on master), but we don't need to shard it, as users usually do not wait for the test results on master.
4 participants
Adding a sharding node to our python CONFIG_TREE