test_capi.test_subinterps() failed on OpenBSD (powerpc)

[rpointel]

BPO	12588
Nosy	@pitrou

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-11-06.18:27:07.410>
created_at = <Date 2011-07-19.21:36:41.346>
labels = ['tests']
title = 'test_capi.test_subinterps() failed on OpenBSD (powerpc)'
updated_at = <Date 2020-11-06.18:27:07.409>
user = 'https://bugs.python.org/rpointel'

bugs.python.org fields:

activity = <Date 2020-11-06.18:27:07.409>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2020-11-06.18:27:07.410>
closer = 'iritkatriel'
components = ['Tests']
creation = <Date 2011-07-19.21:36:41.346>
creator = 'rpointel'
dependencies = []
files = []
hgrepos = []
issue_num = 12588
keywords = []
message_count = 10.0
messages = ['140696', '140705', '140759', '140762', '140764', '140909', '140921', '140986', '222819', '222827']
nosy_count = 5.0
nosy_names = ['pitrou', 'rpetrov', 'neologix', 'rpointel', 'landry']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue12588'
versions = ['Python 3.2']

[rpointel]

Hello,
the test test_subinterps failed on OpenBSD on powerpc architecture.
It works fine on amd64 and sparc64.
(The test_pendingcalls_threaded has been skipped because it blocks on OpenBSD).

Don't hesitate if you need more informations.
Details:

Re-running test 'test_capi' in verbose mode
test_instancemethod (test.test_capi.CAPITest) ... ok
test_memoryview_from_NULL_pointer (test.test_capi.CAPITest) ... ok
test_no_FatalError_infinite_loop (test.test_capi.CAPITest) ... ok
test_pendingcalls_non_threaded (test.test_capi.TestPendingCalls) ... ok
test_pendingcalls_threaded (test.test_capi.TestPendingCalls) ... skipped 'blocking on OpenBSD'
test (test.test_capi.Test6012) ... ok
test_subinterps (test.test_capi.EmbeddingTest) ... test test_capi failed
FAIL

======================================================================
FAIL: test_subinterps (test.test_capi.EmbeddingTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/obj/ports/Python-3.2.1/Python-3.2.1/Lib/test/test_capi.py", line 170, in test_subinterps
    (p.returncode, err))
AssertionError: -11 != 0 : bad returncode -11, stderr is b''

---

Ran 7 tests in 3.797s

FAILED (failures=1, skipped=1)

[pitrou]

test_subinterps merely runs ./Modules/_testembed, so perhaps you could launch it manually and see what it does?
(chances are it crashes, then can you please post the backtrace using a debugger?)

[landry]

(gdb) r
Starting program: /usr/obj/ports/Python-3.2.1/Python-3.2.1/Modules/_testembed
--- Pass 0 ---
interp 0x90ae7300, thread state 0x87b48a00: id(modules) = 2363177552
interp 0x90ae7a40, thread state 0x87b48a80: id(modules) = 2243881408
interp 0x90ae7cc0, thread state 0x87b48100: id(modules) = 2216467744
interp 0x90ae7ec0, thread state 0x87b48180: id(modules) = 2221503376
interp 0x90ae7300, thread state 0x87b48a00: id(modules) = 2363177552
--- Pass 1 ---
interp 0x0, thread state 0x87b48a00:
Program received signal SIGSEGV, Segmentation fault.
0x8cb70094 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
(gdb) bt
#0 0x8cb70094 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
#1 0x8cb70070 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
#2 0x8cb70070 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
#3 0x8cb70070 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
#4 0x8cb70070 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
#5 0x8cb70070 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
#6 0x8cb70070 in PyImport_GetModuleDict () from /usr/local/lib/libpython3.2m.so.1.0
Previous frame inner to this frame (corrupt stack?)

[rpetrov]

Hi Landry , what is result if you move _testembed to upper directory and run it with environment so that shared python library from upper to be loaded (LD_LIBRARY_PATH=.... )?

[landry]

You get the same issue :

[23:08] mikey:/usr/obj/ports/Python-3.2.1/Python-3.2.1/ $cp Modules/_testembed .
[23:08] mikey:/usr/obj/ports/Python-3.2.1/Python-3.2.1/ $LD_LIBRARY_PATH=. gdb _testembed
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "powerpc-unknown-openbsd4.9"...(no debugging symbols found)

(gdb) r
Starting program: /usr/obj/ports/Python-3.2.1/Python-3.2.1/_testembed
--- Pass 0 ---
interp 0x90d4a440, thread state 0x81855380: id(modules) = 2201418320
interp 0x90d4a580, thread state 0x81855600: id(modules) = 2259298752
interp 0x90d4a880, thread state 0x81855480: id(modules) = 2246904544
interp 0x90d4acc0, thread state 0x81855500: id(modules) = 2328741616
interp 0x90d4a440, thread state 0x81855380: id(modules) = 2201418320
--- Pass 1 ---
interp 0x0, thread state 0x81855380:
Program received signal SIGSEGV, Segmentation fault.
0x8eb95094 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
(gdb) bt
#0 0x8eb95094 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
#1 0x8eb95070 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
#2 0x8eb95070 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
#3 0x8eb95070 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
#4 0x8eb95070 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
#5 0x8eb95070 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
#6 0x8eb95070 in PyImport_GetModuleDict () from ./libpython3.2m.so.1.0
Previous frame inner to this frame (corrupt stack?)

[neologix]

Previous frame inner to this frame (corrupt stack?)

Interesting.

Could you post the result of:

$ valgrind --tool=memcheck --suppressions=Misc/valgrind-python.supp Modules/_testembed

after having un-commented the ADDRESS_IN_RANGE suppressions from Misc/valgrind-python.supp?

What compiler version are you using?
Could you try compiling with -O0, then with -fstack-protector-all?

[rpointel]

Hello,

I just want to inform you that valgrind does not work on OpenBSD (only Linux and Darwin).

Thanks for your help,

Remi.

[neologix]

and Darwin).

It would have been useful...

"""
interp 0x0, thread state 0x81855380:
Program received signal SIGSEGV, Segmentation fault.
"""

Here, the interpreter state is NULL, and this shouldn't happen.
It could be a bug linked to thread-local storage (like issue bpo-10517):
it's just a wild guess, but if for some reason the TLS key isn't reset
before the second pass (by _PyGILState_Init()), the current tstate
after PyGILState_Ensure() could be the first pass' - stale - tstate,
with a potentially NULL interp.
Could you try building Python with --with-pydebug and re-test?

But given the stack trace, it looks more like a stack
corruption/compiler problem.

If you're motivated, you could try setting a watchpoint on the current
tstate's interpreter:

"""
cf@neobox:~/cpython$ gdb Modules/_testembed
[...]
(gdb) watch ((PyThreadState*)_PyThreadState_Current)->interp
Watchpoint 1: ((PyThreadState*)_PyThreadState_Current)->interp
"""

and see when it gets assigned NULL.

[BreamoreBoy]

Is this test failure still relevant three years on?

[landry]

Seems something in between fixed it, because it works with 3.4.1. Can test 3.3 if needed.

mikey:/usr/obj/tmpfs/ports/Python-3.4.1/Python-3.4.1/ $LD_LIBRARY_PATH=. ./Modules/_testembed
--- Pass 0 ---
interp 0xac003e00, thread state 0xb4380f80: id(modules) = 2813069072
interp 0xa84b8480, thread state 0xa897cd00: id(modules) = 2840181576
interp 0xa84b8dc0, thread state 0xb1a0b700: id(modules) = 2840181696
interp 0xb1efb540, thread state 0xaec09880: id(modules) = 2929755640
interp 0xac003e00, thread state 0xb4380f80: id(modules) = 2813069072
--- Pass 1 ---
interp 0xab4f1780, thread state 0xaec09000: id(modules) = 2813071072
interp 0xac0035c0, thread state 0xaae20080: id(modules) = 2981253808
interp 0xb2bd6e00, thread state 0xaac10a00: id(modules) = 2981254728
interp 0xab4f1880, thread state 0xb0f31d00: id(modules) = 2862169464
interp 0xab4f1780, thread state 0xaec09000: id(modules) = 2813071072
--- Pass 2 ---
interp 0xb3731e00, thread state 0xb3a6be00: id(modules) = 2812938720
interp 0xa8842e40, thread state 0xaac10c80: id(modules) = 2981325640
interp 0xab4f1500, thread state 0xaec09a80: id(modules) = 2981325160
interp 0xb3731b00, thread state 0xaec06580: id(modules) = 3039539968
interp 0xb3731e00, thread state 0xb3a6be00: id(modules) = 2812938720