--durations gets tricked by freezegun

[mgedmin]

I've a test suite that takes >800 seconds to run, and I'm poking at it to see why.

pytest --durations=10

produces this output for me:

======================================================== slowest 10 test durations =========================================================
118839304.13s teardown statusreports/test_skatteverket.py::test_fetch_times_out[db]
118839304.10s teardown statusreports/test_skatteverket.py::test_fetch_times_out[qvarn]
930.00s call     statusreports/test_skatteverket.py::test_fetch_times_out[db]
930.00s call     statusreports/test_skatteverket.py::test_fetch_times_out[qvarn]
11.21s call     views/test_companies.py::test_admin_user
8.36s setup    services/test_reports.py::test_company_report_keys[qvarn]
7.71s call     views/test_companies.py::test_company_list_fixtures
7.12s call     views/test_companies.py::test_filter_by_status
7.10s call     views/test_companies.py::test_search_by_id
6.55s call     views/test_companies.py::test_search_by_title
=========================================== 2133 passed, 1 skipped, 4 xfailed in 818.45 seconds ============================================

Those numbers are absurd.

The test in question looks like this:

import freezegun
import pytest

...

@pytest.yield_fixture
def frozen_time(mocker):
    with freezegun.freeze_time("2016-12-12") as frozen_time:
        mocker.patch('time.sleep',
                     side_effect=lambda s: frozen_time.tick(datetime.timedelta(seconds=s)))
        yield frozen_time


def test_fetch_times_out(..., mocker, frozen_time):
    ...
    with pytest.raises(TimedOut):
        ... call code under test ...

and I think what happens here is pytest calls time.time() or a similar monkey-patched function while the frozen_time fixture is still alive.

It would be nice if pytest would insulate itself from freezegun (or raw monkey-patching) shenaningans by keeping local references to the real time.time/whatever it uses.

(Although if you say "you pointed the (freeze)gun at your foot and pulled the trigger, what did you expect", I won't be too mad.)

---

This is pytest version 4.1.1, imported from .../env/lib/python3.8/site-packages/pytest.py
setuptools registered plugins:
  requests-mock-1.8.0 at .../env/lib/python3.8/site-packages/requests_mock/contrib/_pytest_plugin.py
  pytest-mock-1.10.0 at .../env/lib/python3.8/site-packages/pytest_mock.py
  pytest-cov-2.6.1 at .../env/lib/python3.8/site-packages/pytest_cov/plugin.py
  celery-4.4.6 at .../env/lib/python3.8/site-packages/celery/contrib/pytest.py

[nicoddemus]

Hi @mgedmin,

We have actually counter measures against monkeypatching time functions and freezegun in latest versions, please try a more recent version.

[mgedmin]

Oops, you're absolutely right, I'm sorry for not checking that myself! pytest 6.0.2 cannot be tricked this way.

(Knee-deep in yaks, you know how it goes.)

[nicoddemus]

No worries! 👍