Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enroll more backends into retries for failed tests #20378

Closed
8 tasks
lilatomic opened this issue Jan 8, 2024 · 1 comment · Fixed by #20606
Closed
8 tasks

Enroll more backends into retries for failed tests #20378

lilatomic opened this issue Jan 8, 2024 · 1 comment · Fixed by #20606
Labels
backend: Go Go backend-related issues backend: Helm Helm backend-related issues backend: JavaScript JavaScript backend-related issues backend: JVM JVM backend-related issues backend: Shell Shell backend-related issues enhancement

Comments

@lilatomic
Copy link
Contributor

lilatomic commented Jan 8, 2024

Is your feature request related to a problem? Please describe.
We now have machinery to retry failed tests. Each backend need to opt into this feature.

Describe the solution you'd like
It is done by wrapping their test processes in a ProcessWithRetries:

    results = await Get(
        ProcessResultWithRetries,
        ProcessWithRetries(setup.process, test_subsystem.attempts_default),
    )

and then using the last result if needed

last_result = results.last
...
        coverage_snapshot = await Get(
            Snapshot, DigestSubset(last_result.output_digest, PathGlobs([".coverage"]))
        )

Relevant backends

Potentially relevant files (searching for -> TestResult and filtering out some irrelevant ones):

  • src/python/pants/backend/helm/test/unittest.py
  • src/python/pants/backend/javascript/goals/test.py
  • src/python/pants/backend/scala/test/scalatest.py
  • src/python/pants/backend/shell/goals/test.py
  • src/python/pants/backend/shell/shunit2_test_runner.py
  • src/python/pants/backend/go/goals/test.py
  • src/python/pants/jvm/test/testutil.py
  • src/python/pants/jvm/test/junit.py
@huonw
Copy link
Contributor

huonw commented Jan 9, 2024

I've done a search for -> TestResult and put in what seem like the relevant files for updating. Please feel free to fix that up.

@huonw huonw added backend: Go Go backend-related issues backend: JVM JVM backend-related issues backend: Helm Helm backend-related issues backend: Shell Shell backend-related issues backend: JavaScript JavaScript backend-related issues labels Jan 13, 2024
benjyw pushed a commit that referenced this issue Mar 23, 2024
Test retries are opt-in, so this MR opts the remaining backends in.

closes #20378
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend: Go Go backend-related issues backend: Helm Helm backend-related issues backend: JavaScript JavaScript backend-related issues backend: JVM JVM backend-related issues backend: Shell Shell backend-related issues enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants