CI should not depend on gmane response

[terryjreedy]

BPO	37605
Nosy	@terryjreedy, @vstinner, @zware, @pablogsal, @nsiregar, @eriker-fsc

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-07-16.15:49:35.655>
labels = ['3.11', 'type-bug', 'tests', 'docs']
title = 'CI should not depend on gmane response'
updated_at = <Date 2021-05-04.11:11:54.618>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2021-05-04.11:11:54.618>
actor = 'terry.reedy'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Tests']
creation = <Date 2019-07-16.15:49:35.655>
creator = 'terry.reedy'
dependencies = []
files = []
hgrepos = []
issue_num = 37605
keywords = []
message_count = 8.0
messages = ['348023', '348024', '348025', '348249', '348383', '348387', '392865', '392875']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'vstinner', 'docs@python', 'zach.ware', 'pablogsal', 'nsiregar', 'eriker']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue37605'
versions = ['Python 3.11']

[terryjreedy]

On Travis, 'Documentation tests' passing is apparently required.  Line 23 of nntplib.rst is ">>> s = NNTP('news.gmane.org')"  I use gmane and occasional failures to respond are routine.  Being able to merge should not depend on this external site.

In addition to a real failure, the following happen on PR 14675 earlier today.

https://travis-ci.org/python/cpython/jobs/559289634

Warning, treated as error:
**********************************************************************
File "library/nntplib.rst", line ?, in default
Failed example:
    s = NNTP('news.gmane.org')
Exception raised:
    Traceback (most recent call last):
      File "/home/travis/build/python/cpython/Lib/doctest.py", line 1328, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest default[0]>", line 1, in <module>
        s = NNTP('news.gmane.org')
      File "/home/travis/build/python/cpython/Lib/nntplib.py", line 1050, in __init__
        _NNTPBase.__init__(self, file, host,
      File "/home/travis/build/python/cpython/Lib/nntplib.py", line 331, in __init__
        self.welcome = self._getresp()
      File "/home/travis/build/python/cpython/Lib/nntplib.py", line 456, in _getresp
        raise NNTPTemporaryError(resp)
    nntplib.NNTPTemporaryError: 400 load at 16.59, try later

Try later is what I do. This exception should somehow be skipped. I don't know anything about Travis and doctest tests and how flakey doctests are supposed to be handled. If nothing else, put the whole example is a try-except block?

>>> try:  # News sites can fail to respond.
...     s = NNTP('news.gmane.org')
...     ...
... except: # If site failure, try later.
...     pass

Since output varies, I presume output checking is suppressed.

[vstinner]

On Travis, 'Documentation tests' passing is apparently required. Line 23 of nntplib.rst is ">>> s = NNTP('news.gmane.org')" I use gmane and occasional failures to respond are routine. Being able to merge should not depend on this external site.

Python test suite uses many external services:
https://pythondev.readthedocs.io/infra.html#services-used-by-unit-tests

You should all be skipped if the "network" resource is not used, which is the default.

Many tests were modified to catch network failures and ignore the test in this case. test_nntplib is known to be very fragile: it doesn't handle network issues.

I would suggest to remove functional tests which depend on external servers, since these tests fail often and nobody seems to care of test_nntplib.

[zware]

Victor, note that this issue is about the nntplib doctests in Doc/library/nntplib.rst. I agree that any examples in that file that actually hit an external server should be marked as not doctests.

[terryjreedy]

gmane was down awhile yesterday, so another merge (temporarily) blocked because of this. If either of you knows how to fix this, please do so.

[nsiregar]

is it allowed to mock news.gmane.org response?

if it is okay i would like to help

[zware]

For doctests, I would prefer to just disable some or all of the doctests in Doc/library/nntplib.rst rather than go to the effort of trying to mock things.

[eriker-fsc]

In other news, news.gmane.org is now news.gmane.io; see also https://bugs.python.org/issue39343

[terryjreedy]

nntp.rst, line 23, has already been updated to "s = NNTP('news.gmane.io'). But that also occasionally fails to respond. I have not seen any test failures recently, so maybe something else has changes. Should this be closed pending current failures?

[hugovk]

One year later:

nntp.rst, line 23, has already been updated to "s = NNTP('news.gmane.io'). But that also occasionally fails to respond. I have not seen any test failures recently, so maybe something else has changes. Should this be closed pending current failures?

Yes, let's close and re-open if necessary.

For doctests, I would prefer to just disable some or all of the doctests in Doc/library/nntplib.rst rather than go to the effort of trying to mock things.

Yes, if it comes up again, that sounds sensible, especially given the nntplib module is deprecated in 3.11 and set for removal in 3.13 (see PEP 594 – Removing dead batteries from the standard library and #91217).