New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest function scoped fixtures should run once per example, not once per test #377

Open
DRMacIver opened this Issue Oct 18, 2016 · 7 comments

Comments

Projects
None yet
3 participants
@DRMacIver
Member

DRMacIver commented Oct 18, 2016

Because of the way Hypothesis just exposes a single test function, pytest just runs its fixtures once for that function.

This is a long running problem with Hypothesis's interaction with pytest and there is an open bug about it on the pytest side too.

Fixing this is tricky. The fixtures-integration branch has a partial prototype which makes some progress towards making it work, but it's very far from complete and has a number of issues which are likely to need substantial work to fix.

I am currently seeking funding to work on this ticket and it is unlikely to make much progress without said funding.

@Zac-HD

This comment has been minimized.

Member

Zac-HD commented May 31, 2017

Related to #59

@Zac-HD

This comment has been minimized.

Member

Zac-HD commented Apr 12, 2018

Note that the pytest-subtesthack package by @untitaker provides a decent workaround for this case.

@Zac-HD

This comment has been minimized.

Member

Zac-HD commented Apr 26, 2018

Actually... it might be possible to incorporate a version of pytest-subtesthack into our pytest plugin, that activates this logic only for Hypothesis tests. @untitaker, do you think this could work?

@untitaker

This comment has been minimized.

Contributor

untitaker commented Apr 26, 2018

Maybe but the version would look very differently, assuming you want this to be the new default behavior of "given"

@DRMacIver

This comment has been minimized.

Member

DRMacIver commented Apr 26, 2018

Actually... it might be possible to incorporate a version of pytest-subtesthack into our pytest plugin, that activates this logic only for Hypothesis tests

I think we need to be quite careful about this. There are a bunch of problems with pytest-subtesthack (or were last I looked) and it's a fairly major change in behaviour. If we're going to support pytest function scoped fixtures I think we need to do it properly.

@Zac-HD

This comment has been minimized.

Member

Zac-HD commented Apr 26, 2018

Fair enough! I didn't expect it to be that easy, but thought it was worth mentioning.

@untitaker

This comment has been minimized.

Contributor

untitaker commented Apr 26, 2018

I think @ossronny can judge better than me how bad the hack really is. I'm not really sure about adding this to hypothesis but I am not sure if it will be fixed in pytest somehow either

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment