Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest's testdir fixture may end up creating flaky tests #2751

Closed
tarcisiofischer opened this issue Sep 5, 2017 · 2 comments
Closed

Pytest's testdir fixture may end up creating flaky tests #2751

tarcisiofischer opened this issue Sep 5, 2017 · 2 comments

Comments

@tarcisiofischer
Copy link

@tarcisiofischer tarcisiofischer commented Sep 5, 2017

While trying to investigate a flaky test on my code, I discovered a problem in the testdir fixture.
I was able to reach the problem while running a test using the testdir fixture too many times and in parallel. The code is something like:

@pytest.mark.parametrize('n', range(1000))
def test_bla(testdir, ..., n):
  # Use testdir here

report1

The problem seems to be here (Note that if the for loop ends with never setting the variable, the problem should happen):
https://github.com/pytest-dev/pytest/blob/master/_pytest/pytester.py#L424

My proposal is to change the whole code block with something like

self.tmpdir = tmpdir_factory.mktemp(name, numbered=True)

I'll make a PR soon.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Sep 5, 2017

Sounds good, thanks @tarcisiofischer!

tarcisiofischer pushed a commit to tarcisiofischer/pytest that referenced this issue Sep 5, 2017
tarcisiofischer pushed a commit to tarcisiofischer/pytest that referenced this issue Sep 5, 2017
@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Sep 6, 2017

Resolved by #2752

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.