pythoninfo collect_gdb() blows up when gdb fails to run

[hroncok]

BPO	40436
Nosy	@vstinner, @hroncok, @miss-islington
PRs	bpo-40436: Fix code parsing gdb version #19792 [3.8] bpo-40436: Fix code parsing gdb version (GH-19792) #19795 [3.7] bpo-40436: Fix code parsing gdb version (GH-19792) #19796

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-04-29.15:16:02.547>
created_at = <Date 2020-04-29.11:36:45.559>
labels = ['3.8', '3.7', 'tests', '3.9']
title = 'pythoninfo collect_gdb() blows up when gdb fails to run'
updated_at = <Date 2020-06-11.13:23:37.447>
user = 'https://github.com/hroncok'

bugs.python.org fields:

activity = <Date 2020-06-11.13:23:37.447>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-29.15:16:02.547>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2020-04-29.11:36:45.559>
creator = 'hroncok'
dependencies = []
files = []
hgrepos = []
issue_num = 40436
keywords = ['patch']
message_count = 8.0
messages = ['367641', '367647', '367654', '367657', '367661', '367662', '367663', '371276']
nosy_count = 3.0
nosy_names = ['vstinner', 'hroncok', 'miss-islington']
pr_nums = ['19792', '19795', '19796']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40436'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[hroncok]

We had this weird traceback when running pythoninfo in Fedora build with Python 3.9.0a6:

+ /builddir/build/BUILD/Python-3.9.0a6/build/optimized/python -m test.pythoninfo
ERROR: collect_gdb() failed
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/pythoninfo.py", line 761, in collect_info
    collect_func(info_add)
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/pythoninfo.py", line 383, in collect_gdb
    version = version.splitlines()[0]
IndexError: list index out of range

I have debugged the problem and it is:

>>> subprocess.run(["gdb", "-nx", "--version"])
CompletedProcess(args=['gdb', '-nx', '--version'], returncode=-11)

There is a segfault. Possibly because gdb was linked to libpython from 3.9.0a5 and we run it trough subprocess from 3.9.0a6.

The code in pythoninfo is:

try:
    proc = subprocess.Popen(["gdb", "-nx", "--version"],
                            stdout=subprocess.PIPE,
                            stderr=subprocess.PIPE,
                            universal_newlines=True)
    version = proc.communicate()[0]
except OSError:
    return

That means it is designed to ignore errors. But it only ignores some errors. Should it only attempt to parse the version when proc.returncode is 0?

I don't know yet if the tests will fail as well, maybe the problem will be bigger.

[hroncok]

BTW The test gdb also crashes in the same way:

test test_gdb crashed -- Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/libregrtest/runtest.py", line 221, in _runtest_inner2
    the_module = importlib.import_module(abstest)
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/test_gdb.py", line 41, in <module>
    gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version()
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/test_gdb.py", line 38, in get_gdb_version
    raise Exception("unable to parse GDB version: %r" % version)
Exception: unable to parse GDB version: ''

1 test failed again:
test_gdb

[vstinner]

New changeset ec9bea4 by Victor Stinner in branch 'master':
bpo-40436: Fix code parsing gdb version (GH-19792)
ec9bea4

[vstinner]

I fixed test.pythoninfo to ignore gdb output if the command failed. I also enhanced test_gdb error message: display stdout, stderr and the exit code. Backport to 3.8 and 3.7 will be merged as soon as the CI pass.

[miss-islington]

New changeset d9e9049 by Miss Islington (bot) in branch '3.8':
bpo-40436: Fix code parsing gdb version (GH-19792)
d9e9049

[miss-islington]

New changeset beba1a8 by Miss Islington (bot) in branch '3.7':
bpo-40436: Fix code parsing gdb version (GH-19792)
beba1a8

[vstinner]

Thanks Miro for the bug report, it's now fixed ;-)

In PR 19792, Miro proposed to skip test_gdb is gdb is available but exit with non-zero exit code. I chose to raise a hard exception instead, to notify CI owners that something is wrong. I prefer to not make gdb error "silent". Otherwise, we may miss that python-gdb.py is no longer tested on a CI.

[vstinner]

I marked bpo-29685 "test_gdb failed" as duplicate of this issue. Extract:

(...)
File "/home/studio/Python-3.6.0/Lib/test/test_gdb.py", line 46, in <module>
gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version()
File "/home/studio/Python-3.6.0/Lib/test/test_gdb.py", line 43, in get_gdb_version
raise Exception("unable to parse GDB version: %r" % version)
Exception: unable to parse GDB version: ''