bpo-36597: fix random doctest failure

[methane]

https://bugs.python.org/issue36597

[vstinner]

Would you mind to close/reopen the PR to trigger a second Travis CI job, to confirm that it's not a random failure?

[vstinner]

@vstinner vstinner removed the needs backport to 3.7 label just now

The bot cannot cherry-pick directly the change, since I didn't disable doctest in the 3.7 branch.

[methane]

1st build: https://travis-ci.org/python/cpython/jobs/518656385
2nd build: https://travis-ci.org/python/cpython/jobs/518661338

[vstinner]

When I test your PR locally, I still get random failures. Commands:

cd Doc
make clean
make venv
LANG= PATH=./venv/bin:$PATH sphinx-build -b doctest -d build/doctrees -D latex_elements.papersize= -q -W -j4 -W . build/doctest ^C

Example of failure:

Warning, treated as error:
**********************************************************************
File "library/statistics.rst", line 126, in default
Failed example:
    fmean([3.5, 4.0, 5.25])
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib64/python3.7/doctest.py", line 1329, in __run
        compileflags, 1), test.globs)
      File "<doctest default[0]>", line 1, in <module>
        fmean([3.5, 4.0, 5.25])
    NameError: name 'fmean' is not defined

My venv has these versions:

$ venv/bin/python -m pip freeze
alabaster==0.7.12
Babel==2.6.0
blurb==1.0.7
certifi==2019.3.9
chardet==3.0.4
docutils==0.14
idna==2.8
imagesize==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
packaging==19.0
Pygments==2.3.1
pyparsing==2.4.0
python-docs-theme==2018.7
pytz==2019.1
requests==2.21.0
six==1.12.0
snowballstemmer==1.2.1
Sphinx==2.0.1
sphinxcontrib-applehelp==1.0.1
sphinxcontrib-devhelp==1.0.1
sphinxcontrib-htmlhelp==1.0.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.2
sphinxcontrib-serializinghtml==1.1.3
urllib3==1.24.1

[methane]

make venv is wrong. You used Python 3.7.
make PYTHON=../python venv is needed.

[vstinner]

I'm very surprised that the job succeed on Travis CI but fail locally. Why Travis CI doesn't get the library/statistics.rst bug for example? Extract of library/statistics.rst where the fail occurred:

.. function:: fmean(data)

   Convert *data* to floats and compute the arithmetic mean.

   This runs faster than the :func:`mean` function and it always returns a
   :class:`float`.  The result is highly accurate but not as perfect as
   :func:`mean`.  If the input dataset is empty, raises a
   :exc:`StatisticsError`.

   .. doctest::

      >>> fmean([3.5, 4.0, 5.25])
      4.25

   .. versionadded:: 3.8

This file doesn't contain "import statistics". How is doctest support to know that fmean() comes from statistics?

[methane]

There is "testsetup" directive In library/statistics.rst:

.. testsetup:: *

   from statistics import *
   __name__ = '<doctest>'

[vstinner]

This file doesn't contain "import statistics". How is doctest support to know that fmean() comes from statistics?

Ah sorry, I looked for "import statistics", but the file contains:

.. testsetup:: *

   from statistics import *
   __name__ = '<doctest>'

I didn't know this markup. The doc can be found at:
https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

Doc/conf.py also contains:

extensions = [..., 'sphinx.ext.doctest', ...]

doctest_global_setup = '''
try:
    import _tkinter
except ImportError:
    _tkinter = None
'''

[vstinner]

make venv is wrong. You used Python 3.7.
make PYTHON=../python venv is needed.

Aaaaaah, ok :-) Sorry, I completely misunderstood how sphinx-build picks its Python.

I was confused by this line from .travis.yml:

xvfb-run make -C Doc/ PYTHON=../python SPHINXOPTS="-q -W -j4" doctest

The PYTHON=../python argument is misleading: it's useless, since sphinx-build uses its "own" Python from the virtual environment...

Sorry for the noise.

[vstinner]

Thanks @methane for the quick fix!