test_pydoc hangs sometimes on 3.6 and master branches on ARMv7 Ubuntu 3.6/3.x

[vstinner]

BPO	30758
Nosy	@gpshead, @vstinner

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-05-29.22:01:55.937>
created_at = <Date 2017-06-26.09:39:54.825>
labels = ['3.7', 'tests']
title = 'test_pydoc hangs sometimes on 3.6 and master branches on ARMv7 Ubuntu 3.6/3.x'
updated_at = <Date 2018-05-29.22:01:55.937>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2018-05-29.22:01:55.937>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-05-29.22:01:55.937>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2017-06-26.09:39:54.825>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 30758
keywords = ['buildbot']
message_count = 9.0
messages = ['296861', '296953', '297600', '297601', '298320', '298948', '301875', '302212', '318118']
nosy_count = 2.0
nosy_names = ['gregory.p.smith', 'vstinner']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue30758'
versions = ['Python 3.6', 'Python 3.7']

[vstinner]

./python ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --timeout=900 -j2 -j8

faulthandler was supposed to kill test_pydoc after 15 minutes, and regrtest is supposed to log "running: pydoc (since xxx seconds)" every 30 seconds. Something gone bad. Buildbot killed regrtest because it didn't produce output during 20 minutes.

http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.x/builds/968/steps/test/logs/stdio

running: test_pydoc (420 sec), test_datetime (234 sec), test_tools (258 sec)
0:11:42 load avg: 3.90 [404/406] test_datetime passed (240 sec) -- running: test_pydoc (428 sec), test_tools (266 sec)
running: test_pydoc (458 sec), test_tools (296 sec)
0:12:13 load avg: 3.15 [405/406] test_tools passed (296 sec) -- running: test_pydoc (459 sec)

command timed out: 1200 seconds without output running ['make', 'buildbottest', 'TESTOPTS=-j2 -j8', 'TESTPYTHONOPTS=', 'TESTTIMEOUT=900'], attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=1936.512343

[vstinner]

Similar issue:

http://buildbot.python.org/all/builders/x86%20Windows7%203.x/builds/800/steps/test/logs/stdio

...
1:49:04 [403/406/1] test_compile passed -- running: test_subprocess (4969 sec)
1:49:10 [404/406/1] test_strptime passed -- running: test_subprocess (4975 sec)
1:49:14 [405/406/1] test_unicode_file passed -- running: test_subprocess (4980 sec)

command timed out: 1200 seconds without output running ['Tools\\buildbot\\test.bat', '-j2', '--timeout', '900'], attempting to kill
program finished with exit code 1
elapsedTime=7768.278000

[vstinner]

Maybe the commit cdee3f1 fixed test_subprocess on the x86 Windows7 3.x buildbot.

[vstinner]

I have no idea for the test_pydoc bug, but it was only seen once. I close the issue, but I may reopen it if I see again these bugs.

[vstinner]

Another fail, so I reopen the issue:

http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.6/builds/563

...
0:11:30 load avg: 4.45 [403/405] test_lib2to3 passed (248 sec) -- running: test_tools (274 sec), test_pydoc (411 sec)
running: test_tools (304 sec), test_pydoc (441 sec)
running: test_tools (334 sec), test_pydoc (471 sec)
0:12:34 load avg: 2.83 [404/405] test_tools passed (338 sec) -- running: test_pydoc (476 sec)

command timed out: 1200 seconds without output ...

[vstinner]

It seems like test_pydoc only hangs on ARMv7 Ubuntu 3.x. Why faulthandler doesn't dump a traceback? :-(

faulthandler is supposed to dump a traceback after 900 seconds. Here, regrtest was killed after 1200 seconds and test_pydoc was already running for 423 seconds.

http://buildbot.python.org/all/builders/ARMv7%20Ubuntu%203.x/builds/1094/steps/test/logs/stdio

./python ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 -j8
...
0:10:28 load avg: 6.21 [403/406] test_venv passed (34 sec) -- running: test_pydoc (400 sec), test_pickle (47 sec), test_tools (338 sec)
0:10:32 load avg: 5.95 [404/406] test_pickle passed (51 sec) -- running: test_pydoc (405 sec), test_tools (343 sec)
0:10:51 load avg: 4.83 [405/406] test_tools passed (361 sec) -- running: test_pydoc (423 sec)

command timed out: 1200 seconds without output running ['make', 'buildbottest', 'TESTOPTS=-j2 -j8', 'TESTPYTHONOPTS=', 'TESTTIMEOUT=900'], attempting to kill
process killed by signal 9
program finished with exit code -1