Make tests using asyncio timers deterministic

[leandro-lucarella-frequenz]

What's needed?

We have quite a few tests that are very dependent on timing. This have 2 main (serious) problems:

• It leads to flaky tests
• It leads to slow tests

Both issues are interrelated, the longer one sleep() between test that depends on time passing by, the slower the tests are. If this safety time is reduced, then the chances of flakiness increases.

Proposed solution

Try async-solipsism, a deterministic event loop:

async-solipsism provide a Python asyncio event loop that does not interact with the outside world at all. This is ideal for writing unit tests that intend to mock out real-world interactions. It makes for tests that are reliable (unaffected by network outages), reproducible (not affected by random timing effects) and portable (run the same everywhere).

Use cases

Tests for many actors have this issue, basically anything using a timer is affected.

Alternatives and workarounds

Only finding a good balance between slowness and flakiness that I know of.

Additional context

We are already using time_machine to fake time, but this is not enough, as it doesn't really play nicely with the event loop. If one advances the time it doesn't necessarily mean events will be fired in the event loop.