Race condition in async tests

[timobrembeck]

Describe the Bug

In some cases there seems to be a race condition in asynchronous tests again.

Steps to Reproduce

No deterministic way of reproducing this, but at least two workflow runs showed this behaviour:

• https://app.circleci.com/pipelines/github/digitalfabrik/integreat-cms/7877/workflows/4cc87299-07f9-4b8a-83f5-3d75bd9ad5c7/jobs/80573
• https://app.circleci.com/pipelines/github/digitalfabrik/integreat-cms/7877/workflows/d0d6d4ca-310d-4363-89bc-6bf66cc5e139/jobs/80617

Expected Behavior

Async tests should not influence the sync tests

Actual Behavior

Something goes wrong when specific tests are parallelized together with the async SUMM.AI test

Additional Information

Traceback

============================= test session starts ==============================
platform linux -- Python 3.9.16, pytest-7.2.2, pluggy-1.0.0
django: settings: integreat_cms.core.circleci_settings (from option)
rootdir: /home/circleci/project
plugins: circleci-parallelized-0.1.0, cov-4.0.0, django-4.5.2, icdiff-0.6, testmon-1.4.5, xdist-3.2.0, requests-mock-1.10.0
collecting 2499 items                                                          No timing found for "test_recurrence_rule.TestCreatingIcalRule"
collected 2499 items                                                           
running 69 items due to CircleCI parallelism

tests/summ_ai_api/summ_ai_test.py ......................                 [ 31%]
tests/api/test_api_result.py .....................................       [ 85%]
tests/cms/views/test_public_view_status_code.py ....                     [ 91%]
tests/core/management/commands/test_summ_ai_bulk.py ..

Too long with no output (exceeded 10m0s): context deadline exceeded

This is probably a similar problem like the one fixed in #1787.

[timobrembeck]

The underlying problem: summ_ai_enabled is set to True in async mode while other sync tests expect the value to be False (like in the test data) and the sync tests start before the test db is reset after the async test is finished.

This is a problem when these two tests are assigned to the same container on CircleCI.

Approaches I tried that all didn't work:

• Rewrite the async tests as sync (didn't work because aiohttp didn't work with asyncio.run(aiohttp_raw_server(fake_summ_ai_server)))
• Wrap the async tests in sync wrappers (didn't work as well, not sure why)
• Wrapping the async code in async.Lock (didn't work because the sync code can't aquire the lock)
• Wrapping the async tests in asgiref.sync.async_to_sync (the tests were just ignored)

I temporarily disabled the corresponding test case in d912d60 to prevent further CI errors for the moment. Enable them again when you want to reproduce the problem.

[seluianova]

Maybe we should deal with such issues by separating the data?
Let's say: each async test should only use data, that is not used by other tests (separate regions, separate pages, etc.)

If we move our SUMM AI tests to a new region, test_summ_ai_bulk_disabled_region will work.
We could also fix test_summ_ai_bulk_disabled_region by using nurnberg instead of augsburg, but I think it's better to change SUMM AI tests to avoid similar problems in the future.