Document the autotestdict directive (#387)

[robertwb]

Document the functionality implemented in http://trac.cython.org/ticket/387.

Migrated from http://trac.cython.org/ticket/388

[robertwb]

@dagss changed summary from

Document the doctest hack directive (http://trac.cython.org/ticket/387)

to

Document the autotestdict directive (http://trac.cython.org/ticket/387)
commented

[embray]

I was going to open an issue about this as well. I was recently bitten by this because working with the pytest-cython plugin this actually causes every test to be run twice.

It's actually a pretty nice feature of Cython and it would be worth incorporating explicit support for it into pytest-cython, but in the meantime I needed to find a way to disable it, and since it's not documented it was a bit mysterious at first. I'll make a PR for it if you want. Just wasn't sure until I found this issue if there was an explicit reason it was an undocumented feature.

[robertwb]

Yes, a PR would be appreciated.

[embray]

I'm actually trying to better understand exactly what problem this feature was intended to solve and how it's expected to work? At least in current versions of Cython, unless compiling with docstrings stripped out, for functions accessible from Python this isn't entirely needed because their doctests can still be introspected from the Python level.

To me, this seems most useful for docstrings that would not otherwise be accessible from Python, such as for cdef functions. This can be enabled by setting autotestdict.cdef = True, but this is not even the default. It also explicitly excludes special methods like __cinit__ and __dealloc__ on the basis that they are "not accessible from Python", but this is also true for any cdef functions, and it still seems like a case could be made for wanting to introspect the docstrings/doctests for these methods as well.

@dagss Do you have any idea?

[da-woods]

cython/runtests.py

Lines 901 to 902 in 5d7d0db

	if sys.version_info >= (3, 4):
	Options._directive_defaults['autotestdict'] = False

I think it's necessary to make the doctests work on earlier versions of Python. Doctests definitely don't work for Python 2.7 without it (but I don't know the exact mechanism that the doctest module uses). For more recent versions it's probably pretty unnecessary.

[scoder]

Should we remove the feature in 3.1 then?

[da-woods]

Should we remove the feature in 3.1 then?

I don't see why it should be on by default there. Potentially autotestdict.cdef might still be useful though? We don't look to use it in our test suite though, and practically you'd always need some Python-visible wrapper function to actually run the tests (so the docstrings could just go in the wrapper function).

Probably remove it unless autotestdict.cdef has real uses?

[embray]

One reason I was looking into this is thinking about the possibility of switching SageMath to using pytest as its test runner. For that, being able to document cdef functions would be useful (I agree that in most cases cdef functions are still tested implicitly by tests for wrapper code, but Sage still has tons of doctests in cdef functions for historic reasons).

However, I found that there is no combination of settings that allow creating a __test__ dict just for cdef functions. So maybe I would propose disabling this by default, but making it possible to test cdef functions if, e.g. autotestdict.all == False and autotestdict.cdef == True.