[Question] How do I globally increase the page timeout when running via pytest

[seancolsen]

Problem

• I have a number of tests that succeed but require longer than the default timeout, especially when running on CI where compute resources are scarce.

• I have a tried using a file conftest.py with a fixture to set the default timeout to 30 seconds like so:

  @pytest.fixture
  def page(page):
      timeout = 30000
      page.set_default_navigation_timeout(timeout)
      page.set_default_timeout(timeout)
      yield page

  With this fixture, setting timeout to small values like 100 seems to work, but using large values like 30000 does not seem to work.

Desired solution

• I would like a way to globally increase the page timeout to 30 seconds.

More details

• When one of my problematic tests fails, I see this message among its output:

  playwright._impl._api_types.TimeoutError: Timeout 5000ms exceeded.

  That's helpful because I can see the timeout value is 5000ms.

• To simulate this behavior consistently, here's a test that always fails, printing the error message which tells me the timeout is 5000ms.

  def test_should_fail(page: Page, live_server):
      page.goto(f"{live_server}")
      expect(page.locator("text=This text does not exist")).to_be_visible()

• If I modify that test as follows, manually setting the page timeout to 100ms...

  def test_should_fail(page: Page, live_server):
      page.set_default_timeout(100)
      page.goto(f"{live_server}")
      expect(page.locator("text=This text does not exist")).to_be_visible()

  ...then I observe 100ms in the error message as follows:

  playwright._impl._api_types.TimeoutError: Timeout 100ms exceeded.

  That's good because it gives me the impression that I have some control over the timeout.

• However what I want to do is raise the timeout -- not lower it. If I change the timeout in my code to 6000 and re-run the test, I expect to see 6000 in the output. But instead I see 5000 in the output. Strange.

• I have tried using page.set_default_timeout and page.set_default_navigation_timeout (and both). Same effect.

• Interestingly, setting a timeout of 300 (or below) works, but setting a timeout of 400 (or above) does not.

• So there appears to be a threshold at which my manually-set timeout becomes too large to take effect. Exactly where this threshold lies seems a bit slippery. I have a vague impression that it may be larger for some tests which are more computationally intensive, but I only have a loose grasp on its behavior, having been troubleshooting this problem somewhat haphazardly.

• The unpredictability leads me to suspect there may be a race condition at play.

Any help would be greatly appreciated!!

Environment

• Playwright 1.22.0
• Python 3.9.8
• pytest 0.2.3
• Ubuntu 20.04

[mxschmitt]

The reason for that:

expect(page.locator("text=This text does not exist")).to_be_visible()

is not working is that the web-first assertions as of today have a fixed timeout of 5s. We have a feature request for it which makes it configurable: microsoft/playwright#12824

[mxschmitt]

Folding into #1358