[CI] Move gha artifact download before xml parsing for test stat uploads #125609
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/125609
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 2eacc85 with merge base 7bf6ed0 ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
| for path in s3_paths: | ||
| unzip(path) | ||
|
|
||
| artifact_paths = download_gha_artifacts( |
There was a problem hiding this comment.
We probably have them all on S3 now with ODIC, but plz double check edge cases like ROCm
There was a problem hiding this comment.
I'm confused, I thought all the artifacts would eventually get uploaded to s3 by the step that uploads from gha artifacts to s3?
There was a problem hiding this comment.
Yup, that's my thought too. Although the workflow needs to be added there explicitly https://github.com/pytorch/pytorch/blob/main/.github/workflows/upload-test-stats.yml#L5
|
@pytorchbot merge -f "no trunk needed" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
3 participants
Move gha artifact download to before any xml parsing is done for uplaod-test-stats
Do not download gha artifacts during xml parsing since got uploaded to s3 in the above and will be downloaded when all the artifacts are downloaded from s3
The previous method resulted in dups if you run the script again
TODO: write a deduper so we don't have to worry at all