New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wish documentation was clearer on recommended way to structure tests and fixtures across files #1505

Closed
chrish42 opened this Issue Apr 7, 2016 · 8 comments

Comments

Projects
None yet
4 participants
@chrish42

chrish42 commented Apr 7, 2016

I've been (re-)learning pytest (used it last in the days where you had to get a snapshot of "py") and really like the new pytest, its documentation, and especially fixtures. However, there's one aspect of the documentation that I personally found lacking and really wish was clearer: how to reuse tests and fixtures across multiple test_*.py files. Other than the recommendation not to create a __init__.py file in the root directory of the tests, there's little else (at least that I could find) on how to share test data and fixtures across python files.

Does pytest always add the root of the tests directory to sys.path (so we can use imports to pull in test data, etc. from a common module)? Should we put stuff in conftest.py instead? Googling these questions produces some hits, but they are sometimes unclear and/or contradictory. I wish there was some kind of official recommendation about this in the doc. I think it would be helpful.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Apr 7, 2016

Thanks for taking the time to write this!

Answering your question, to share fixtures in a project the recommended way is using conftest files.

I agree the documentation could be more clear on how to structure the tests, and trade-offs between having __init__.py files in the test hierarchy or not.

@chrish42

This comment has been minimized.

chrish42 commented Apr 8, 2016

Thanks for the answer and pointer to the documentation. I think this topic really deserves its own section in the documentation (as a bullet point under the "scales from simple unit to complex functional testing" heading in the home page), instead of being scattered throughout multiple other documentation sections.

Also, two quick notes. I've searched, and I wasn't able to find a dedicated section on conftest files. When searching for "pytest conftest" on Google, the first hit (for me at least) is to the "Installing and Using Plugins" page for the documentation of the latest version (which doesn't talk at all about the other things you can do with conftest files). And the second hit is to "Working with plugins and conftest files" for version 2.2.4 of the pytest doc... which is a bit old. Since this is an important feature of pytest (when using it for more than a few simple tests), it would be really helpful if there was a dedicated page of documentation that talked about conftest files and all their features.

Also, for the link you gave me, if you look just a bit below, at the "tests file structure" of the "Override a fixture on a folder (conftest) level" section, there is a top-level __init__.py file in that structure, which seems to go against recommendations made elsewhere... :-)

(Let me know if you'd prefer separate issues filed for those two small things.)

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Apr 8, 2016

Excellent suggestions, thanks!

(Let me know if you'd prefer separate issues filed for those two small things.)

We have #1112 which is related, but keeping them separate for now is OK I guess. 😁

Btw, PRs on docs are really welcome!

@DuncanBetts

This comment has been minimized.

Contributor

DuncanBetts commented Nov 26, 2016

@chrish42 if you have a bullet point in mind, I can edit the documentation - or if you'd like to commit it yourself, see @lwm's suggestion.

@bilderbuchi

This comment has been minimized.

Contributor

bilderbuchi commented Oct 18, 2017

It seems a section about conftest.py could best fit immediately before https://docs.pytest.org/en/latest/fixture.html#sharing-a-fixture-across-tests-in-a-module-or-class-session. There the concept of conftest.py is "first" introduced, so that part could be isolated under its own heading. Conftest.py is in fact earlier mentioned in passing here, which should probably add a reference to later.

I can do the part about sharing fixtures via conftest.py, but what about "how to share test data" in the OP - what is the best practice there? Load the test data via a fixture and provide that way? Is there even another way to get at that reusably? So far I have mostly put the datafiles into the tests folder and loaded in the test files directly, and this worked, but my setups were not too complex.

@bilderbuchi

This comment has been minimized.

Contributor

bilderbuchi commented Oct 18, 2017

also relevant: #1770 (intro docs for conftest.py), could probably kill two birds with one stone, here.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Oct 18, 2017

@bilderbuchi

but what about "how to share test data" in the OP - what is the best practice there? Load the test data via a fixture and provide that way? Is there even another way to get at that reusably?

If you can load the data into a fixture, then that is a good way to do it because of the automatic caching (even using scope="session" if the data is immutable).

Adding the data files in the tests folder as you mention is also a very good approach and I use it extensively. We might even mention a few plugins which help in this regard:

@bilderbuchi

This comment has been minimized.

Contributor

bilderbuchi commented Oct 19, 2017

Ok, thanks, I'll take a shot at this.

bilderbuchi added a commit to bilderbuchi/pytest that referenced this issue Oct 23, 2017

Introduce a dedicated section about conftest.py
Also reorganize conftest-related content a bit.
Closes pytest-dev#1505, closes pytest-dev#1770.

bilderbuchi added a commit to bilderbuchi/pytest that referenced this issue Oct 23, 2017

Introduce a dedicated section about conftest.py
Also reorganize conftest-related content a bit.
Closes pytest-dev#1505, closes pytest-dev#1770.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment