doctests in python open source: Position paper

[tony]

I want to improve how tests work in Python in regards to our documentation.

doctest is genius. docutils and sphinx is ubiquitous in our open source projects.

Despite that, in our own documentation (.rst and .md files), doctest does not run out of the box. This leads to documentation examples that are untested.

There is an essential thing open source projects need for good doctests: Readily available test objects in scope.

In the case of libtmux, that'd mean have a new test-friendly session = Session() ready-to-go for every doctest.

>>> session.new_window()
...

In pytest we achieve that through creating a fixture injecting doctest_namespace:

https://github.com/tmux-python/libtmux/blob/v0.15.0a1/libtmux/conftest.py#L107-L117

This works well in pytest 7.1.0 when ran against .py files - but not against .md, .rst etc.

My doctest_docutils module parses .md and .rst, and my pytest plugin pytest_doctest_docutils which implements it, aims to remedy it.

Status: An issue with conftest.py and fixtures was resolved by moving conftest.py on libtmux to a pytest plugin (tmux-python/libtmux#411, tmux-python/libtmux#412)