How to debug worker 'gwN' crashed while running ...

[dimaqq]

We're facing an issue that some times our unit test suite fails with worker 'gwN' crashed while running test_mmm with varying N and M.

The test suite contains some 1171 tests and completes in about 1 minute.

The failure rate is very small, less than 1 per 100 runs.

Current observations, library versions:

• pytest-xdist 3.7.0 affected
• pytest-xdist 3.6.1 affected
• pytest-xdist 3.5.0 possibly not affected

Python versions:

• common failures: python 3.8 and 3.9
• uncommon or no failures: 3.10 and 3.12

Environments:

• able to hit the error in github actions (4 workers; test repeated 10 times in a job, 10x jobs run in parallel)
• not able to hit the error in local setup (so far)

Given how time consuming hitting the error is, I'm looking for advice on how to possibly instrument or log more or any other approach really to help pinpoint the problem.

Thank you!

[RonnyPfannschmidt]

The next step would be to try and trigger the issue with pytest_DEBUG ENV VAR being set

Also there's possibly a need to cross examine the execmodel

[dimaqq]

I've come to suspect that it's a bug in Python.
However, I was only able to repro against py 3.8 and 3.9 and those are closed for bugfixes.
I'm going to close this and

cross-referencing:
canonical/operator#1771 (this problem)
python/cpython#122529 (seems similar)
open-telemetry/opentelemetry-python#2030 (possibly same very problem)