test: Discover pytest fixtures during doctest

[matthewfeickert]

Description

• Resolves GitHub Actions only error: AttributeError("'NoneType' object has no attribute '_sorted_indices'") #2015
• Needed for PR feat: Add autoscan for upper limit using TOMS Algorithm 748 #1274

Symlink the tests/conftest.py under the src tree to allow for the autouse pytest fixtures to be discovered and used during doctest. This is needed as the backends are not currently being automatically reset between doctests and the fixtures will not be discovered during doctest runs invoked with pytest src/ as noted in the pytest docs:

Note that like the normal conftest.py, the fixtures are discovered in the directory tree conftest is in. Meaning that if you put your doctest with your source code, the relevant conftest.py needs to be in the same directory tree. Fixtures will not be discovered in a sibling directory tree!

• c.f. https://docs.pytest.org/en/7.1.x/how-to/doctest.html#doctest-namespace-fixture
• https://docs.pytest.org/en/7.1.x/explanation/pythonpath.html

Exclude src/conftest.py from the MANIFEST.in so as to not include it in the sdist build.

Checklist Before Requesting Reviewer

• Tests are passing
• "WIP" removed from the title of the pull request
• Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

• Summarize commit messages into a comprehensive review of the PR

* Symlink the tests/conftest.py under the src/ tree to allow for the
  autouse pytest fixtures to be discovered and used during doctest.
  This is needed as the backends are not currently being automatically
  reset between doctests and the fixtures will not be discovered during
  doctest runs invoked with `pytest src/` as noted in the pytest docs:

> Note that like the normal conftest.py, the fixtures are discovered in the
> directory tree conftest is in. Meaning that if you put your doctest with
> your source code, the relevant conftest.py needs to be in the same directory
> tree. Fixtures will not be discovered in a sibling directory tree!

   - c.f. https://docs.pytest.org/en/7.1.x/how-to/doctest.html#doctest-namespace-fixture
   - https://docs.pytest.org/en/7.1.x/explanation/pythonpath.html

* Exclude src/conftest.py from the MANIFEST.in so as to not include it
  in the sdist build.

[codecov]

Codecov Report

Base: 98.28% // Head: 98.28% // No change to project coverage 👍

Coverage data is based on head (6b64e89) compared to base (f7e974f).
Patch has no changes to coverable lines.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2017   +/-   ##
=======================================
  Coverage   98.28%   98.28%           
=======================================
  Files          68       68           
  Lines        4482     4482           
  Branches      730      730           
=======================================
  Hits         4405     4405           
  Misses         45       45           
  Partials       32       32           

Flag	Coverage Δ
contrib	27.64% <ø> (ø)
doctest	61.13% <ø> (-0.18%)	⬇️
unittests-3.10	96.22% <ø> (ø)
unittests-3.7	96.20% <ø> (ø)
unittests-3.8	96.25% <ø> (ø)
unittests-3.9	96.27% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[kratsg]

Does this work if we move conftest.py up one level instead of having a symlink?

[matthewfeickert]

Does this work if we move conftest.py up one level instead of having a symlink?

I think so (testing now) edit: NOPE. Assuming so, you'd prefer to have conftest.py be a top level file? If so, that's fine with me.

[kratsg]

I think so (testing now). Assuming so, you'd prefer to have conftest.py be a top level file? If so, that's fine with me.

It feels a bit weird and hacky but I guess I see why. I wonder if pytest tests/conftest.py src/ would fix this as well. I think I might prefer the symlink as it's more explicit, but the problem is we might forget why it's there later...

[matthewfeickert]

It feels a bit weird and hacky but I guess I see why. I wonder if pytest tests/conftest.py src/ would fix this as well.

Yeah, it is not ideal I agree. Though turns out we don't want to move conftest.py to the top level as it causes additional problems (that I think are somewhat related to pytest-dev/pytest#2269). So I think we'd have to either keep the symlink or during the doctest do some on the fly injection/copying (which I think isn't a good idea).

c.f. the pytest docs for more details on the mechanics of conftest.py and import mechanisms: https://docs.pytest.org/en/7.1.x/explanation/pythonpath.html

I think I might prefer the symlink as it's more explicit, but the problem is we might forget why it's there later...

Yes, that's one of the reasons I wanted to split this out instead of putting it in PR #1274 so that git blame will be more useful.