Skip to content

stuck replacing workers - maybe is related with pytest-timeout plugin? #1094

Open
@andreabisello

Description

@andreabisello

Take a look at that execution

C:\ec4a9867-4d56\qa-lsd-webservices>pytest --tc-file=configurations/utenze/eu-it1.ini --tc-file=configurations/puntamenti/legal.ini --dist=loadgroup -n auto --alluredir=allure-results/legal --force-flaky
parametro -> configurations/utenze/eu-it1.ini
parametro -> configurations/puntamenti/legal.ini
============================================================================================= test session starts ==============================================================================================
platform win32 -- Python 3.11.6, pytest-7.4.4, pluggy-1.3.0
Suite di test dei webservices
rootdir: C:\ec4a9867-4d56\qa-lsd-webservices
configfile: pytest.ini
plugins: allure-pytest-2.13.2, anyio-4.3.0, flaky-3.7.0, base-url-2.0.0, html-4.0.2, metadata-3.0.0, playwright-0.5.0, testconfig-0.2.0, timeout-2.2.0, xdist-3.6.1
timeout: 900.0s
timeout method: thread
timeout func_only: False
6 workers [1420 items] skipped
scheduling tests via LoadGroupScheduling

test/test_api/test_metodi_conservazione.py::test_getAllSoggettiParam_e_count_disponibili_solo_per_amministratori_Sdc@serialize
test/test_browser/test_admin/test_document_type/test_document_type.py::TestDocumentType::test_create_document_type_with_document_type_script[chromium]
test/test_api/test_metodi_registrazione_admin_cliente.py::test_registraCliente_FatturazioneElettronicaB2B[chromium-contratto_senza_barcode.pdf]
test/test_api/test_lta_rest_api.py::test_open_swagger_page[chromium]
test/test_browser/test_admin/test_document_type/test_document_type.py::TestDocumentType::test_create_document_type_with_index_completion_script[chromium]
test_upload_documento_esaw.py::TestEsawArchivePlugin::test_send_and_sign_a_document[chromium]
[gw0] [ 0%] PASSED test/test_api/test_metodi_conservazione.py::test_getAllSoggettiParam_e_count_disponibili_solo_per_amministratori_Sdc@serialize
test/test_api/test_metodi_conservazione.py::test_getAllAziende_getAzienda_UpdateAzienda@serialize
[gw0] [ 0%] PASSED test/test_api/test_metodi_conservazione.py::test_getAllAziende_getAzienda_UpdateAzienda@serialize
test/test_api/test_metodi_conservazione.py::test_visibilita_e_funzionalita_getUserByUserName@serialize
[gw0] [ 0%] PASSED test/test_api/test_metodi_conservazione.py::test_visibilita_e_funzionalita_getUserByUserName@serialize
test/test_api/test_metodi_conservazione.py::test_getAllPdvParamCount_confermami_lacount_dei_pdv@serialize

[a lot of log here]

[gw5] [ 40%] SKIPPED test/test_api/test_lta_rest_api.py::test_pre_pdv_combinazioni[True---False-False]
test/test_api/test_lta_rest_api.py::test_pre_pdv_combinazioni[False-5ddfd582163b9b2e789d7563-5ddfd582163b9b2e789d7563-False-False]
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++

PDF created successfully.

~~~~~~~~~~~~~~~~~~~~~~~~~~ Stack of asyncio_0 (16712) ~~~~~~~~~~~~~~~~~~~~~~~~~~
  File "C:\Python311\Lib\threading.py", line 1002, in _bootstrap
    self._bootstrap_inner()
  File "C:\Python311\Lib\threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "C:\Python311\Lib\threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python311\Lib\concurrent\futures\thread.py", line 81, in _worker
    work_item = work_queue.get(block=True)

~~~~~~~~~~~~~~~~~~~~~~~~~~ Stack of <unknown> (14536) ~~~~~~~~~~~~~~~~~~~~~~~~~~
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 411, in _perform_spawn
    reply.run()
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 341, in run
    self._result = func(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 1160, in _thread_receiver
    msg = Message.from_io(io)
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 567, in from_io
    header = io.read(9)  # type 1, channel 4, payload 4
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 534, in read
    data = self._read(numbytes - len(buf))

~~~~~~~~~~~~~~~~~~~~~~~~~ Stack of MainThread (26152) ~~~~~~~~~~~~~~~~~~~~~~~~~~
  File "C:\Python311\Lib\site-packages\playwright\sync_api\_context_manager.py", line 56, in greenlet_main
    self._loop.run_until_complete(self._connection.run_as_sync())
  File "C:\Python311\Lib\asyncio\base_events.py", line 640, in run_until_complete
    self.run_forever()
  File "C:\Python311\Lib\asyncio\windows_events.py", line 321, in run_forever
    super().run_forever()
  File "C:\Python311\Lib\asyncio\base_events.py", line 607, in run_forever
    self._run_once()
  File "C:\Python311\Lib\asyncio\base_events.py", line 1884, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\Python311\Lib\asyncio\windows_events.py", line 444, in select
    self._poll(timeout)
  File "C:\Python311\Lib\asyncio\windows_events.py", line 817, in _poll
    status = _overlapped.GetQueuedCompletionStatus(self._iocp, ms)

+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++

[gw3] [ 40%] PASSED test/test_api/test_lta_rest_api.py::test_pre_pdv_combinazioni[True---False-True]
test/test_api/test_lta_rest_api.py::test_pre_pdv_combinazioni[False-5ddfd582163b9b2e789d7563-5ddfd582163b9b2e789d7563-True-False]
[gw0] [ 40%] PASSED test/test_api/test_lta_rest_api.py::test_pre_pdv_combinazioni[True--5ddfd582163b9b2e789d7563-False-True]
test/test_api/test_lta_rest_api.py::test_pre_pdv_combinazioni[True---True-False]
[gw1] node down: Not properly terminated
[gw1] [ 40%] FAILED test_upload_documento_esaw.py::TestEsawArchivePlugin::test_send_and_sign_a_document[chromium]

replacing crashed worker gw1
collecting: 6/7 workers[gw2] [ 40%] PASSED test/test_api/test_lta_rest_api.py::test_pre_pdv_combinazioni[False-5ddfd582163b9b2e789d7563-5ddfd582163b9b2e789d7563-True-True]

[a lot of log here]


[gw6] [100%] PASSED test/test_api/test_lta_rest_api_sensible_document.py::test_cerca_per_metadata_equal_to
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Captured stdout ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PDF created successfully.

~~~~~~~~~~~~~~~~~~~~~~~~~~ Stack of asyncio_0 (14848) ~~~~~~~~~~~~~~~~~~~~~~~~~~
  File "C:\Python311\Lib\threading.py", line 1002, in _bootstrap
    self._bootstrap_inner()
  File "C:\Python311\Lib\threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "C:\Python311\Lib\threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python311\Lib\concurrent\futures\thread.py", line 81, in _worker
    work_item = work_queue.get(block=True)

~~~~~~~~~~~~~~~~~~~~~~~~~~ Stack of <unknown> (6052) ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 411, in _perform_spawn
    reply.run()
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 341, in run
    self._result = func(*args, **kwargs)
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 1160, in _thread_receiver
    msg = Message.from_io(io)
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 567, in from_io
    header = io.read(9)  # type 1, channel 4, payload 4
  File "C:\Python311\Lib\site-packages\execnet\gateway_base.py", line 534, in read
    data = self._read(numbytes - len(buf))

~~~~~~~~~~~~~~~~~~~~~~~~~~ Stack of MainThread (4196) ~~~~~~~~~~~~~~~~~~~~~~~~~~
  File "C:\Python311\Lib\site-packages\playwright\sync_api\_context_manager.py", line 56, in greenlet_main
    self._loop.run_until_complete(self._connection.run_as_sync())
  File "C:\Python311\Lib\asyncio\base_events.py", line 640, in run_until_complete
    self.run_forever()
  File "C:\Python311\Lib\asyncio\windows_events.py", line 321, in run_forever
    super().run_forever()
  File "C:\Python311\Lib\asyncio\base_events.py", line 607, in run_forever
    self._run_once()
  File "C:\Python311\Lib\asyncio\base_events.py", line 1884, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\Python311\Lib\asyncio\windows_events.py", line 444, in select
    self._poll(timeout)
  File "C:\Python311\Lib\asyncio\windows_events.py", line 817, in _poll
    status = _overlapped.GetQueuedCompletionStatus(self._iocp, ms)

+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++

[gw0] node down: Not properly terminated
[gw0] [100%] FAILED test_upload_documento_esaw.py::TestEsawArchivePlugin::test_send_and_sign_a_document[chromium]

replacing crashed worker gw0
8 workers [1420 items]

at some points, after a timeout (https://pypi.org/project/pytest-timeout/) a worker were replaced, but for pytest now workers are 8, not 6 as the beginning, and nothing happening for 30 minutes

image

what other informations could i share to make this situation actionable? thanks for help

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions