support testsetup

[jab]

I saw in the README that support for testsetup is planned but didn't see an issue to track it, so thought I'd create one and register interest to boot.

Is there a workaround you would recommend in the meantime?

Thanks for maintaining pytest-sphinx!

[twmr]

No there is unfortunately no workaround. In the coming days I'll clean up the existing implementation first and then I'll implement support for testsetup. Since you are also interested in support for more flags, maybe you can try to contibute and add support for pyversion and skipif. That would be great.

[jab]

(see pytest-dev/pytest#4274 (comment))

[nathanshammah]

I was wondering if there were any updates.

Maybe I need much less (I'm not so familiar with sphinx+doctest): My issue is that I have a function that is defined in the guide *.rst file, and I'd like to use it. Right now I am including a module with such function in the code base, including such module in the conf.py doctest_global_setup feature and then importing it in a testsetup block that I'd like to provide it for the subsequent blocks in the same .rst file. Is there some easy fix I am not looking? make doctest passes, but pytest --doctest-glob fail, and these are set in the CI on Github workflow (similar to Travis CI). Do you have any suggestion?

[twmr]

Is there some easy fix I am not looking?

No currently not. I'll have a look at the testsetup code again, once I'm done with the support for ":skipif:" option.

[nathanshammah]

@Thisch thank you. Sorry, just to understand:
Now I am using testcode + testoutput and I get some errors with pytest --doctest-glob='*.rst' (with the pytest-sphinx plugin installed) but not with make doctest. The issue seems to be that I am defining some functions in a testcode block, and using them in a subsequent testcode block, in the same *.rst file. Is this an intended behavior, a bug, or I am not setting it up correctly?

[twmr]

Your described behavior is definitely a bug. I don't yet have an idea how to implement/fix it, because it requires that we share data between pytest test-items.

[twmr]

@nathanshammah Please take a look at #23. You can move the code block from conf.py to conftest.py and adapt it s.t. uses the doctest_namespace fixture. See https://docs.pytest.org/en/latest/doctest.html#doctest-namespace-fixture

Does this solve some of you issues?

[jab]

The doctest_namespace workaround works in some cases, but not all. For example, if you try to reference an injected variable from a :skipif:, you get a NameError.

# conftest.py
@pytest.fixture(autouse=True)
def add_doctest_globals(doctest_namespace):
    doctest_namespace['foo'] = False

# test.rst

.. testcode::
    :skipif: not foo

    42

$ pytest --doctest-glob='*.rst'
...
ERROR test.rst - NameError: name 'foo' is not defined

[bveeramani]

Hey @Thisch! We're considering using pytest-sphinx for https://github.com/ray-project/ray. Do you know if support for testsetup is still planned?

[twmr]

Hi! That's great news! Currently support for testsetup is still wanted, but it is not planned to be implemented anytime soon. I've just seen that https://github.com/astropy/pytest-doctestplus has "support" for testsetup, but it just works by accident by using the >>> in the testsetup block, s.t. the code is run by the doctest plugin that is part of pytest. (scientific-python/pytest-doctestplus#164)