Skip to content

Fix xdist collisions for captured_info artifacts and preserve CI debug logs#45645

Open
stationeros wants to merge 5 commits intohuggingface:mainfrom
stationeros:main
Open

Fix xdist collisions for captured_info artifacts and preserve CI debug logs#45645
stationeros wants to merge 5 commits intohuggingface:mainfrom
stationeros:main

Conversation

@stationeros
Copy link
Copy Markdown

What does this PR do?

Fixes #45561.

This PR fixes a race in the test failure reporting flow when the suite is run with pytest-xdist.

Right now every worker writes debug output to the same captured_info.txt file. In parallel runs that means workers can overwrite each other’s output, and in some cases one worker can remove the file another worker was still relying on. The end result is flaky or incomplete failure logs, which makes CI failures much harder to debug.

The fix keeps the current behavior for non-xdist runs, but switches to worker-specific files when PYTEST_XDIST_WORKER is set. So instead of multiple workers all writing to the same file, each one writes its own captured_info_<worker>.txt.

Along with that, this PR also updates the surrounding plumbing so the debug info still shows up where people expect it:

  • CI log collection now prints all matching captured_info*.txt files instead of assuming there is only one.
  • notification artifact handling aggregates worker-specific captured info files back into the existing reporting flow.
  • stale captured_info*.txt files are cleaned up before a new collection starts so repeated runs do not pick up leftovers from earlier failures.

I also added tests covering:

  • worker-specific path selection under xdist
  • legacy single-file behavior outside xdist
  • cleanup of stale captured info files
  • aggregation of worker-specific artifacts in notification handling

Code Agent Policy

  • I confirm that this is not a pure code agent PR.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

This touches test infrastructure / CI behavior, so @ydshieh seems like the most relevant reviewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] pytest-xdist workers race on captured_info.txt in patched testing utils

1 participant