New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest's testdir fixture may end up creating flaky tests #2751

Closed
tarcisiofischer opened this Issue Sep 5, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@tarcisiofischer

tarcisiofischer commented Sep 5, 2017

While trying to investigate a flaky test on my code, I discovered a problem in the testdir fixture.
I was able to reach the problem while running a test using the testdir fixture too many times and in parallel. The code is something like:

@pytest.mark.parametrize('n', range(1000))
def test_bla(testdir, ..., n):
  # Use testdir here

report1

The problem seems to be here (Note that if the for loop ends with never setting the variable, the problem should happen):
https://github.com/pytest-dev/pytest/blob/master/_pytest/pytester.py#L424

My proposal is to change the whole code block with something like

self.tmpdir = tmpdir_factory.mktemp(name, numbered=True)

I'll make a PR soon.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Sep 5, 2017

Sounds good, thanks @tarcisiofischer!

tarcisiofischer pushed a commit to tarcisiofischer/pytest that referenced this issue Sep 5, 2017

tarcisiofischer pushed a commit to tarcisiofischer/pytest that referenced this issue Sep 5, 2017

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Sep 6, 2017

Resolved by #2752

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment