test_gdb should better detect when Python is optimized

[vstinner]

BPO	40019
Nosy	@vstinner, @miss-islington
PRs	bpo-40019: Skip test_gdb if Python was optimized #19081 [3.8] bpo-40019: Skip test_gdb if Python was optimized (GH-19081) #19255 [3.7] bpo-40019: Skip test_gdb if Python was optimized (GH-19081) #19256

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-03-31.17:31:29.760>
created_at = <Date 2020-03-19.23:20:34.169>
labels = ['tests', '3.9']
title = 'test_gdb  should better detect when Python is optimized'
updated_at = <Date 2020-03-31.17:31:29.759>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-03-31.17:31:29.759>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-03-31.17:31:29.760>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2020-03-19.23:20:34.169>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40019
keywords = ['patch']
message_count = 7.0
messages = ['364640', '364650', '364660', '365400', '365404', '365405', '365407']
nosy_count = 2.0
nosy_names = ['vstinner', 'miss-islington']
pr_nums = ['19081', '19255', '19256']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40019'
versions = ['Python 3.9']

[vstinner]

On my PR 19077 which changes Python/ceval.c, test_gdb fails on Travis CI with Python compiled with clang -Og.

The -Og optimization level is a compromise between performance and the ability to debug Python. The problem is that gdb fails to retrieve some information and so test_gdb fails.

I proposed bpo-38350 "./configure --with-pydebug should use -O0 rather than -Og", but the status quo is to continue to use -Og by default.

See examples of test_gdb failures from PR 19077 below.

I propose to skip a test if one of the follow pattern is found in gdb output:

• '<optimized out>',
• '(frame information optimized out)',
• 'Unable to read information on python frame',

======================================================================
FAIL: test_basic_command (test.test_gdb.PyListTests)
Verify that the "py-list" command works
----------------------------------------------------------------------
(...)

AssertionError: (...)

'Breakpoint 1 at 0x5aabf1: file Python/bltinmodule.c, line 1173.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, builtin_id (self=, v=42) at Python/bltinmodule.c:1173
1173\t PyObject *id = PyLong_FromVoidPtr(v);
Unable to read information on python frame
'

did not end with

' 5
6 def bar(a, b, c):
7 baz(a, b, c)
8
9 def baz(*args):
>10 id(42)
11
12 foo(1, 2, 3)
'

======================================================================
FAIL: test_bt (test.test_gdb.PyBtTests)
Verify that the "py-bt" command works
----------------------------------------------------------------------
(...)
AssertionError:

'Breakpoint 1 at 0x5aabf1: file Python/bltinmodule.c, line 1173.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, builtin_id (self=, v=42) at Python/bltinmodule.c:1173
1173\t    PyObject *id = PyLong_FromVoidPtr(v);
Traceback (most recent call first):
  <built-in method id of module object at remote 0x7ffff7f87050>
  (frame information optimized out)
  File "/home/travis/build/python/cpython/Lib/test/gdb_sample.py", line 7, in bar
    baz(a, b, c)
  File "/home/travis/build/python/cpython/Lib/test/gdb_sample.py", line 4, in foo
    bar(a, b, c)
  (frame information optimized out)
'

did not match

'^.*
Traceback \\(most recent call first\\):
<built-in method id of module object .*>
File ".*gdb_sample.py", line 10, in baz
id\\(42\\)
File ".*gdb_sample.py", line 7, in bar
baz\\(a, b, c\\)
File ".*gdb_sample.py", line 4, in foo
bar\\(a, b, c\\)
File ".*gdb_sample.py", line 12, in <module>
foo\\(1, 2, 3\\)
'

[vstinner]

New changeset 7bf069b by Victor Stinner in branch 'master':
bpo-40019: Skip test_gdb if Python was optimized (GH-19081)
7bf069b

[vstinner]

The fix works as expected: test_gdb passed on PR 19077. Well, likely because a few tests have been skipped. I don't think that in test_gdb can do better than skipping the test if gdb fails to read debug information.

[vstinner]

We have this issue in Fedora Rawhide. test_gdb of Python 3.8 fails on s390x and armv7hl architectures with GCC 10:
https://bugzilla.redhat.com/show_bug.cgi?id=1818857

I tested manually that my change fix test_gdb: tests are skipped as expected:
https://src.fedoraproject.org/rpms/python3/pull-request/182

So I backported the change to 3.7 and 3.8.

This change only fix test_gdb. To get a fully working python-gdb.py, IMHO the best is to disable all compiler optimization using -O0 optimization level.

[miss-islington]

New changeset 4ced9a7 by Miss Islington (bot) in branch '3.8':
bpo-40019: Skip test_gdb if Python was optimized (GH-19081)
4ced9a7

[miss-islington]

New changeset a764a1c by Miss Islington (bot) in branch '3.7':
bpo-40019: Skip test_gdb if Python was optimized (GH-19081)
a764a1c

[vstinner]

Ok, test_gdb should now be more reliable with various compilers and compiler optimization levels.

I chose to not skip a test if '<optimized out>' is found in the gdb output. If it becomes an issue, test_gdb can easily be modified to also be skipped in this case.