Skip to content

shard linux-xenial-py3.7-clang7-asan / test (default from 4 to 5 #77084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Parent: Add extra debug info for the super common "index out of bounds" error.
wants to merge 1 commit into from
Closed

shard linux-xenial-py3.7-clang7-asan / test (default from 4 to 5 #77084

wants to merge 1 commit into from

Conversation

@clee2000
Copy link
Contributor

@clee2000 clee2000 commented May 9, 2022 •
edited
Loading

Notes

  • still over 2 hrs even with 4 shards so try 5 shards
  • i think 6 shards was a bit unbalanced, will need to test some other time

sharding spreadsheet: https://docs.google.com/spreadsheets/d/1BdtVsjRr0Is9LXMNilR02FEdPXNq7zEWl8AmR3ArsLQ/edit#gid=1242752407

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 9, 2022 •
edited
Loading

🔗 Helpful links

✅ No Failures (0 Pending)

As of commit f5bad5a (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@clee2000 clee2000 force-pushed the clee2000/shard-asan branch from 4d24ded to f5bad5a Compare May 9, 2022 20:17
@clee2000 clee2000 changed the title shard linux-xenial-py3.7-clang7-asan / test (default from 4 to 6 shard linux-xenial-py3.7-clang7-asan / test (default from 4 to 5 May 9, 2022
@clee2000 clee2000 marked this pull request as ready for review May 10, 2022 16:57
@clee2000 clee2000 requested a review from suo May 10, 2022 16:58
@clee2000
Copy link
Contributor Author

@pytorchbot merge this please

@github-actions
Copy link
Contributor

Hey @clee2000.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

facebook-github-bot pushed a commit that referenced this pull request May 13, 2022
@clee2000 @facebook-github-bot
shard linux-xenial-py3.7-clang7-asan / test (default from 4 to 5 (#…
d47fa96 
…77084) (#77084)

Summary:
Notes
- still over 2 hrs even with 4 shards so try 5 shards
- i think 6 shards was a bit unbalanced, will need to test some other time

sharding spreadsheet: https://docs.google.com/spreadsheets/d/1BdtVsjRr0Is9LXMNilR02FEdPXNq7zEWl8AmR3ArsLQ/edit#gid=1242752407

Pull Request resolved: #77084
Approved by: https://github.com/janeyx99

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/ecab4fb057fac50ff041d35a11d933fe6b2c93ea

Reviewed By: atalman

Differential Revision: D36299698

Pulled By: clee2000

fbshipit-source-id: 02829ffa50303660cdb6170f773c7b5cd10f10b6
@clee2000
Copy link
Contributor Author

six shards is unbalanced because some times files (specifically test_ops_jit) are too big, if we want to get more sharding from asan, we should split this file up

@clee2000 clee2000 deleted the clee2000/shard-asan branch May 16, 2022 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janeyx99 janeyx99 janeyx99 approved these changes

@seemethere seemethere Awaiting requested review from seemethere

@atalman atalman Awaiting requested review from atalman

@suo suo Awaiting requested review from suo

Assignees

No one assigned

Labels

cla signed Merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@clee2000 @facebook-github-bot @janeyx99 @pytorchmergebot