Unexpectedly no active workers available

[mikicz]

Hello!

I've started seeing a weird error on some of my test runs, where I get a RuntimeError: Unexpectedly no active workers available raised in pytest-xdist, after a in TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' from pytest.

What is strangest is that this is on a successful run (from the looks of it), judging by the summary

=== 9605 passed, 100 skipped, 20 xpassed, 6043 warnings in 993.43s (0:16:33) ===

I've not really managed to figure out why it happens, and why it only happens sometimes. What is especially puzzling to me is that it only happens sometimes, and that it does happen on succesful runs as well.

Any tips for debugging this? Any specific functions or hooks I should look at, to see if me overriding sometime causes issues?

Here's the full exception:

INTERNALERROR> def worker_internal_error(
INTERNALERROR>         self, node: WorkerController, formatted_error: str
INTERNALERROR>     ) -> None:
INTERNALERROR>         """
INTERNALERROR>         pytest_internalerror() was called on the worker.
INTERNALERROR>     
INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr, which can't
INTERNALERROR>         be serialized, so we go with a poor man's solution of raising an exception
INTERNALERROR>         here ourselves using the formatted message.
INTERNALERROR>         """
INTERNALERROR>         self._active_nodes.remove(node)
INTERNALERROR>         try:
INTERNALERROR> >           assert False, formatted_error
INTERNALERROR> E           AssertionError: Traceback (most recent call last):
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 283, in wrap_session
INTERNALERROR> E                 session.exitstatus = doit(config, session) or 0
INTERNALERROR> E                                      ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 337, in _main
INTERNALERROR> E                 config.hook.pytest_runtestloop(session=session)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/logging.py", line 803, in pytest_runtestloop
INTERNALERROR> E                 return (yield)  # Run all the tests.
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/terminal.py", line 673, in pytest_runtestloop
INTERNALERROR> E                 result = yield
INTERNALERROR> E                          ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/xdist/remote.py", line 174, in pytest_runtestloop
INTERNALERROR> E                 self.run_one_test()
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/xdist/remote.py", line 195, in run_one_test
INTERNALERROR> E                 self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/warnings.py", line 112, in pytest_runtest_protocol
INTERNALERROR> E                 return (yield)
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/assertion/__init__.py", line 176, in pytest_runtest_protocol
INTERNALERROR> E                 return (yield)
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/unittest.py", line 429, in pytest_runtest_protocol
INTERNALERROR> E                 res = yield
INTERNALERROR> E                       ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/faulthandler.py", line 88, in pytest_runtest_protocol
INTERNALERROR> E                 return (yield)
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 89, in pytest_runtest_protocol
INTERNALERROR> E                 self.runner.pytest_runtest_protocol(item, nextitem)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
INTERNALERROR> E                 runtestprotocol(item, nextitem=nextitem)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/runner.py", line 132, in runtestprotocol
INTERNALERROR> E                 reports.append(call_and_report(item, "call", log))
INTERNALERROR> E                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 152, in call_and_report
INTERNALERROR> E                 report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR> E                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR> E                 raise exception.with_traceback(exception.__traceback__)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR> E                 teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR> E                 ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/tmpdir.py", line 310, in pytest_runtest_makereport
INTERNALERROR> E                 rep = yield
INTERNALERROR> E                       ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR> E                 teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR> E                 ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/skipping.py", line 269, in pytest_runtest_makereport
INTERNALERROR> E                 rep = yield
INTERNALERROR> E                       ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/runner.py", line 368, in pytest_runtest_makereport
INTERNALERROR> E                 return TestReport.from_item_and_call(item, call)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/reports.py", line 376, in from_item_and_call
INTERNALERROR> E                 longrepr = item.repr_failure(excinfo)
INTERNALERROR> E                            ^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/python.py", line 1669, in repr_failure
INTERNALERROR> E                 return self._repr_failure_py(excinfo, style=style)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/nodes.py", line 456, in _repr_failure_py
INTERNALERROR> E                 return excinfo.getrepr(
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 690, in getrepr
INTERNALERROR> E                 return fmt.repr_excinfo(self)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 1058, in repr_excinfo
INTERNALERROR> E                 reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR> E                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 991, in repr_traceback
INTERNALERROR> E                 entries = [
INTERNALERROR> E                           ^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 992, in <listcomp>
INTERNALERROR> E                 self.repr_traceback_entry(entry, excinfo if last == entry else None)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 930, in repr_traceback_entry
INTERNALERROR> E                 source = self._getentrysource(entry)
INTERNALERROR> E                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 828, in _getentrysource
INTERNALERROR> E                 source = entry.getsource(self.astcache)
INTERNALERROR> E                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 263, in getsource
INTERNALERROR> E                 self.lineno, source, astnode=astnode
INTERNALERROR> E                 ^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 213, in lineno
INTERNALERROR> E                 return self._rawentry.tb_lineno - 1
INTERNALERROR> E                        ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
INTERNALERROR> E             TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
INTERNALERROR> E           assert False
INTERNALERROR> 
INTERNALERROR> /venv/lib/python3.11/site-packages/xdist/dsession.py:232: AssertionError
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 283, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 337, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR>     teardown.throw(outcome._exception)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/logging.py", line 803, in pytest_runtestloop
INTERNALERROR>     return (yield)  # Run all the tests.
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR>     teardown.throw(outcome._exception)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/terminal.py", line 673, in pytest_runtestloop
INTERNALERROR>     result = yield
INTERNALERROR>              ^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/xdist/dsession.py", line 138, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/xdist/dsession.py", line 152, in loop_once
INTERNALERROR>     raise RuntimeError("Unexpectedly no active workers available")
INTERNALERROR> RuntimeError: Unexpectedly no active workers available

Full run including plugin and python versions and the full command

pytest --reuse-db --cov --cov-report= -n 30 --force-flaky --max-runs=5 --dist worksteal --junitxml=test-results-split-1.xml --splits 3 --group 1
============================= test session starts ==============================
platform linux -- Python 3.11.11, pytest-8.3.5, pluggy-1.5.0
django: version: 5.1.8, settings: platformapi.settings_for_tests (from env)
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
rootdir: /data/jenkins-/workspace/6df25ff4a0ed11f92fceb02c70fa9f5b
configfile: pytest.ini
plugins: devtools-0.12.2, Faker-37.1.0, flaky-3.8.1, cov-6.1.1, django-4.10.0, env-1.1.5, lazy-fixtures-1.1.2, mock-3.14.0, picked-0.5.1, random-order-1.1.1, repeat-0.9.4, split-0.10.0, timeout-2.3.1, unused-fixtures-0.2.0, xdist-3.6.1, xdist-worker-stats-0.3.0, requests-mock-1.12.1, xelix-testing-utils-2.6.0
timeout: 300.0s
timeout method: signal
timeout func_only: True
created: 30/30 workers
30 workers [9728 items]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
........................................................................ [  2%]
........................................................................ [  3%]
........................................................................ [  4%]
........................................................................ [  5%]
........................................................................ [  5%]
........................................................................ [  6%]
........................................................................ [  7%]
........................................................................ [  8%]
........................................................................ [  8%]
........................................................................ [  9%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 11%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 22%]
...............................................X........................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 27%]
........................................................................ [ 28%]
...........................X..................X..................X...... [ 28%]
................X.....................X..................X..........X... [ 29%]
........................X......................X..............X......... [ 30%]
............X...................X....................................... [ 31%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 33%]
........................................................................ [ 34%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 37%]
.............................................................ss......... [ 38%]
s.s......s.s.s.......................................................... [ 39%]
........................................................................ [ 39%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 42%]
.............................................s.............s............ [ 42%]
......s......ss.s.....ss.......ss.s.....................s.............s. [ 43%]
...........s................................................s.s.....s.ss [ 44%]
.s....ss.ss.s....s.s...s................................................ [ 45%]
................................................................s.s..... [ 45%]
.....s.s..ss.s.......................................................... [ 46%]
..............s..s........s.s..sss...................................... [ 47%]
........................................................................ [ 48%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 51%]
........................................................................ [ 51%]
..............................................s.sssssss.s.ssss.......... [ 52%]
........................................................................ [ 53%]
........................................................................ [ 54%]
........................................................................ [ 54%]
........................................................................ [ 55%]
......................................ss.........................s.s.... [ 56%]
........................................................................ [ 56%]
......................ss.................ss............................. [ 57%]
..s.s................................................................... [ 58%]
........................................................................ [ 59%]
........................................................................ [ 59%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 62%]
...s..s................................................................. [ 63%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 70%]
....................ss.................................................. [ 71%]
........................................................................ [ 71%]
.....................................................X.....X........X... [ 72%]
.......X...X............X............X.................................. [ 73%]
......................................................ss..s............. [ 74%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 76%]
........................................................................ [ 76%]
........................................................................ [ 77%]
.....................................................s..s............... [ 78%]
............ss..................................ss...................... [ 79%]
.s...s.............s...............................................s.... [ 79%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 93%]
.....................................s.................................. [ 93%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 96%]
........................................................................ [ 97%]
........................................................................ [ 98%]
...s...................ssssss..................ss....................... [ 99%]
.....s.................................................................. [ 99%]
.....INTERNALERROR> def worker_internal_error(
INTERNALERROR>         self, node: WorkerController, formatted_error: str
INTERNALERROR>     ) -> None:
INTERNALERROR>         """
INTERNALERROR>         pytest_internalerror() was called on the worker.
INTERNALERROR>     
INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr, which can't
INTERNALERROR>         be serialized, so we go with a poor man's solution of raising an exception
INTERNALERROR>         here ourselves using the formatted message.
INTERNALERROR>         """
INTERNALERROR>         self._active_nodes.remove(node)
INTERNALERROR>         try:
INTERNALERROR> >           assert False, formatted_error
INTERNALERROR> E           AssertionError: Traceback (most recent call last):
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 283, in wrap_session
INTERNALERROR> E                 session.exitstatus = doit(config, session) or 0
INTERNALERROR> E                                      ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 337, in _main
INTERNALERROR> E                 config.hook.pytest_runtestloop(session=session)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/logging.py", line 803, in pytest_runtestloop
INTERNALERROR> E                 return (yield)  # Run all the tests.
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/terminal.py", line 673, in pytest_runtestloop
INTERNALERROR> E                 result = yield
INTERNALERROR> E                          ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/xdist/remote.py", line 174, in pytest_runtestloop
INTERNALERROR> E                 self.run_one_test()
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/xdist/remote.py", line 195, in run_one_test
INTERNALERROR> E                 self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/warnings.py", line 112, in pytest_runtest_protocol
INTERNALERROR> E                 return (yield)
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/assertion/__init__.py", line 176, in pytest_runtest_protocol
INTERNALERROR> E                 return (yield)
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/unittest.py", line 429, in pytest_runtest_protocol
INTERNALERROR> E                 res = yield
INTERNALERROR> E                       ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> E                 teardown.throw(outcome._exception)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/faulthandler.py", line 88, in pytest_runtest_protocol
INTERNALERROR> E                 return (yield)
INTERNALERROR> E                         ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 89, in pytest_runtest_protocol
INTERNALERROR> E                 self.runner.pytest_runtest_protocol(item, nextitem)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
INTERNALERROR> E                 runtestprotocol(item, nextitem=nextitem)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/runner.py", line 132, in runtestprotocol
INTERNALERROR> E                 reports.append(call_and_report(item, "call", log))
INTERNALERROR> E                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 152, in call_and_report
INTERNALERROR> E                 report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR> E                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR> E                 raise exception.with_traceback(exception.__traceback__)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR> E                 teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR> E                 ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/tmpdir.py", line 310, in pytest_runtest_makereport
INTERNALERROR> E                 rep = yield
INTERNALERROR> E                       ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR> E                 teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR> E                 ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/skipping.py", line 269, in pytest_runtest_makereport
INTERNALERROR> E                 rep = yield
INTERNALERROR> E                       ^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E                       ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/runner.py", line 368, in pytest_runtest_makereport
INTERNALERROR> E                 return TestReport.from_item_and_call(item, call)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/reports.py", line 376, in from_item_and_call
INTERNALERROR> E                 longrepr = item.repr_failure(excinfo)
INTERNALERROR> E                            ^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/python.py", line 1669, in repr_failure
INTERNALERROR> E                 return self._repr_failure_py(excinfo, style=style)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/nodes.py", line 456, in _repr_failure_py
INTERNALERROR> E                 return excinfo.getrepr(
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 690, in getrepr
INTERNALERROR> E                 return fmt.repr_excinfo(self)
INTERNALERROR> E                        ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 1058, in repr_excinfo
INTERNALERROR> E                 reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR> E                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 991, in repr_traceback
INTERNALERROR> E                 entries = [
INTERNALERROR> E                           ^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 992, in <listcomp>
INTERNALERROR> E                 self.repr_traceback_entry(entry, excinfo if last == entry else None)
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 930, in repr_traceback_entry
INTERNALERROR> E                 source = self._getentrysource(entry)
INTERNALERROR> E                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 828, in _getentrysource
INTERNALERROR> E                 source = entry.getsource(self.astcache)
INTERNALERROR> E                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 263, in getsource
INTERNALERROR> E                 self.lineno, source, astnode=astnode
INTERNALERROR> E                 ^^^^^^^^^^^
INTERNALERROR> E               File "/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 213, in lineno
INTERNALERROR> E                 return self._rawentry.tb_lineno - 1
INTERNALERROR> E                        ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
INTERNALERROR> E             TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
INTERNALERROR> E           assert False
INTERNALERROR> 
INTERNALERROR> /venv/lib/python3.11/site-packages/xdist/dsession.py:232: AssertionError
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 283, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/main.py", line 337, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR>     teardown.throw(outcome._exception)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/logging.py", line 803, in pytest_runtestloop
INTERNALERROR>     return (yield)  # Run all the tests.
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR>     teardown.throw(outcome._exception)
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/_pytest/terminal.py", line 673, in pytest_runtestloop
INTERNALERROR>     result = yield
INTERNALERROR>              ^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/xdist/dsession.py", line 138, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/venv/lib/python3.11/site-packages/xdist/dsession.py", line 152, in loop_once
INTERNALERROR>     raise RuntimeError("Unexpectedly no active workers available")
INTERNALERROR> RuntimeError: Unexpectedly no active workers available

=== 9605 passed, 100 skipped, 20 xpassed, 6043 warnings in 993.43s (0:16:33) ===

[RonnyPfannschmidt]

at frist glance it seems like a error in traceback handling breaks things (that lineno in the tb shouldnt be None)

--tb=native might be able to help with that as the traceback errors when making a teardown report

theres need for extra detail to figure if this is a bug in a c library or if ptest needs to get more defensive

[mikicz]

Ok, I will throw in --tb=native to see if I catch it! Will reply back if I do. Going on vacation after tomorrow so might not provide more details for a while, unless it happens tomorrow

[mikicz]

I think if anything, it's primarily a pytest bug rather than pytest-xdist bug, where the main issue is the TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'. I just experienced that error while running tests without xdist.

Maybe xdist doesn't need to die in the case, but seemingly primary issue is in pytest itself. I saw it locally when I aborted a testrun with Control+C.

Should I raise this as a bug in pytest instead?

Also a side note, I tried with --tb native and I got the same exact exception above with the same level of details, no changes there with --tb native...

The critical parts of the exception - it says line is -1 for some reason! Maybe that's the problem?

  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/options.py", line -1, in _expire_cache
KeyboardInterrupt

...

  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 213, in lineno
    return self._rawentry.tb_lineno - 1
           ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

Full

Traceback (most recent call last):
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/main.py", line 283, in wrap_session
    session.exitstatus = doit(config, session) or 0
                         ^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/main.py", line 337, in _main
    config.hook.pytest_runtestloop(session=session)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/logging.py", line 803, in pytest_runtestloop
    return (yield)  # Run all the tests.
            ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/terminal.py", line 673, in pytest_runtestloop
    result = yield
             ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/main.py", line 362, in pytest_runtestloop
    item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/warnings.py", line 112, in pytest_runtest_protocol
    return (yield)
            ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/assertion/__init__.py", line 176, in pytest_runtest_protocol
    return (yield)
            ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/unittest.py", line 429, in pytest_runtest_protocol
    res = yield
          ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/faulthandler.py", line 88, in pytest_runtest_protocol
    return (yield)
            ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 89, in pytest_runtest_protocol
    self.runner.pytest_runtest_protocol(item, nextitem)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/runner.py", line 113, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/runner.py", line 132, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 149, in call_and_report
    call = _call_runtest_hook(item, when, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 145, in _call_runtest_hook
    return runner.CallInfo.from_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/flaky/flaky_pytest_plugin.py", line 146, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/capture.py", line 898, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/.pyenv/versions/3.11.8/lib/python3.11/unittest/mock.py", line 1375, in patched
    return func(*newargs, **newkeywargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/src/demo_data/tests/commands/test_make_demo_data.py", line 124, in test_s3_identifier_and_celery_combination_no_test_org
    call_command("make_demo_data", s3_identifier=organisation.s3_identifier, celery=True)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django_custom_commands/management.py", line 118, in call_command
    return original_call_command(command, *args, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 194, in call_command
    return command.execute(*args, **defaults)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/xelix_tenanted_command/tenanted_command.py", line 123, in execute
    super().execute(*args, **options)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/src/statement_automation/error_handling/should_raise.py", line 24, in decorated
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/src/demo_data/management/commands/make_demo_data.py", line 113, in handle
    self.setup_demo_organisation_count(organisations, options["organisations"])
  File "/home/miki/xelix/platform-api/src/demo_data/management/commands/make_demo_data.py", line 68, in setup_demo_organisation_count
    factories.DemoOrganisationFactory.create_batch(size=change_count, base_currency=CurrencyChoices.GBP)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/xelix_factory_utils/django.py", line 36, in create_batch
    return [cls.create(**kwargs) for _ in range(size)]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/xelix_factory_utils/django.py", line 36, in <listcomp>
    return [cls.create(**kwargs) for _ in range(size)]
            ^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/factory/base.py", line 539, in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/factory/django.py", line 122, in _generate
    return super()._generate(strategy, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/factory/base.py", line 468, in _generate
    return step.build()
           ^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/factory/builder.py", line 274, in build
    instance = self.factory_meta.instantiate(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/factory/base.py", line 320, in instantiate
    return self.factory._create(model, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/src/public/organisation/tests/factories/organisation.py", line 26, in _create
    return super()._create(model_class, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/factory/django.py", line 175, in _create
    return manager.create(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/query.py", line 679, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/xelix_platform_db/public/organisation/models/organisation.py", line 109, in save
    return super().save(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django_tenants/models.py", line 111, in save
    self.create_schema(check_if_exists=True, verbosity=verbosity)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/xelix_platform_db/public/organisation/models/organisation.py", line 115, in create_schema
    super().create_schema(check_if_exists, sync_schema, verbosity)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django_tenants/models.py", line 198, in create_schema
    call_command('migrate_schemas',
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 194, in call_command
    return command.execute(*args, **defaults)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django_tenants/management/commands/migrate_schemas.py", line 77, in handle
    executor.run_migrations(tenants=tenants)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/xelix_platform_db/common/db/executor.py", line 93, in run_migrations
    run_migrations(self.args, self.options, self.codename, schema_name, idx=idx, count=len(tenants))
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django_tenants/migration_executors/base.py", line 59, in run_migrations
    migrate_command_class(stdout=stdout, stderr=stderr).execute(*args, **options)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 357, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 255, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/migration.py", line 118, in apply
    operation.state_forwards(self.app_label, project_state)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/operations/models.py", line 83, in state_forwards
    state.add_model(
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 124, in add_model
    self.reload_model(*model_key)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 400, in reload_model
    self._reload(related_models)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 435, in _reload
    self.apps.render_multiple(states_to_be_rendered)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 665, in render_multiple
    model.render(self)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/migrations/state.py", line 959, in render
    return type(self.name, bases, body)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/base.py", line 194, in __new__
    new_class.add_to_class(obj_name, obj)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/base.py", line 371, in add_to_class
    value.contribute_to_class(cls, name)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/fields/__init__.py", line 951, in contribute_to_class
    cls._meta.add_field(self, private=private_only)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/options.py", line 358, in add_field
    self._expire_cache(reverse=False)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/django/db/models/options.py", line -1, in _expire_cache
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/miki/xelix/platform-api/venv/bin/pytest", line 8, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 201, in console_main
    code = main()
           ^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 175, in main
    ret: ExitCode | int = config.hook.pytest_cmdline_main(config=config)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/main.py", line 330, in pytest_cmdline_main
    return wrap_session(config, _main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/main.py", line 297, in wrap_session
    config.hook.pytest_keyboard_interrupt(excinfo=excinfo)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/terminal.py", line 932, in pytest_keyboard_interrupt
    self._keyboardinterrupt_memo = excinfo.getrepr(funcargs=True)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 690, in getrepr
    return fmt.repr_excinfo(self)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 1058, in repr_excinfo
    reprtraceback = self.repr_traceback(excinfo_)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 991, in repr_traceback
    entries = [
              ^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 992, in <listcomp>
    self.repr_traceback_entry(entry, excinfo if last == entry else None)
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 930, in repr_traceback_entry
    source = self._getentrysource(entry)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 828, in _getentrysource
    source = entry.getsource(self.astcache)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 263, in getsource
    self.lineno, source, astnode=astnode
    ^^^^^^^^^^^
  File "/home/miki/xelix/platform-api/venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 213, in lineno
    return self._rawentry.tb_lineno - 1
           ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute

[RonnyPfannschmidt]

As per the exception a frame has a line number of none

My current understanding is that only a broken c extension can trigger that

My understand is that every valid frame object returns a number there

The note about a type not having a module hardens that suspicion

[RonnyPfannschmidt]

I just looked up the details and Indeed cpython maps line numbers 0 and -1 in c to None in python

This needs a pytest bugs

We also need to create a warning when this case triggers