Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codemod service to disable and mark slow tests directly in PyTorch code base #4981

Open
huydhn opened this issue Mar 2, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@huydhn
Copy link
Contributor

huydhn commented Mar 2, 2024

The mechanism to disable a test is sometimes "abused" to keep a test disable for a long time without any owner to work on fixing it. Although, this helps mitigate an flaky test well, it has several drawback:

Ideally, we want to keep the current mechanism but limit it only to recently flaky tests, i.e. within 2 weeks. Anything older than that and has not yet been addressed should be disabled directly in the code. Doing this manually doesn't scale, so we need a better tool for this like a codemod service of sort.

cc @clee2000 @malfet @kit1980

@huydhn huydhn added the enhancement New feature or request label Mar 2, 2024
@huydhn
Copy link
Contributor Author

huydhn commented Mar 12, 2024

AI: It's better to be able to keep the test as XFAIL if possible or a special decorator like skipifFlaky

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

No branches or pull requests

1 participant