[2.7] Seg fault on archlinux 32 when run tests with xvfb-run #78276

deep-42-thought · 2018-07-11T13:04:13Z

BPO	34095
Nosy	@terryjreedy, @benjaminp, @eli-schwartz, @deep-42-thought
Files	log.all: log of the last command log

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-01-28.23:34:33.618>
created_at = <Date 2018-07-11.13:04:12.926>
labels = ['build', 'type-crash']
title = '[2.7] Seg fault on archlinux 32 when run tests with xvfb-run'
updated_at = <Date 2020-01-28.23:34:33.614>
user = 'https://github.com/deep-42-thought'

bugs.python.org fields:

activity = <Date 2020-01-28.23:34:33.614>
actor = 'benjamin.peterson'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-28.23:34:33.618>
closer = 'benjamin.peterson'
components = ['Build']
creation = <Date 2018-07-11.13:04:12.926>
creator = 'deep42thought'
dependencies = []
files = ['47682', '47869']
hgrepos = []
issue_num = 34095
keywords = []
message_count = 16.0
messages = ['321447', '321453', '321470', '321478', '321631', '321736', '321737', '327520', '327521', '327536', '327625', '327735', '327738', '327766', '327768', '360911']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'benjamin.peterson', 'eschwartz', 'deep42thought']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue34095'
versions = ['Python 2.7']

deep-42-thought · 2018-07-11T13:03:43Z

building python 2.7.15 on archlinux32 from source succeeds, but running the testsuite segfaults with:

test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... /usr/bin/xvfb-run: line 181: 3617 Segmentation fault (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1

the executed commands are:

cd Python-${pkgver}

# Temporary workaround for FS#22322
# See http://bugs.python.org/issue10835 for upstream report
sed -i "/progname =/s/python/python${_pybasever}/" Python/pythonrun.c

# Enable built-in SQLite module to load extensions (fix FS#22122)
sed -i "/SQLITE_OMIT_LOAD_EXTENSION/d" setup.py

# FS#23997
sed -i -e "s|^#.* /usr/local/bin/python|#!/usr/bin/python2|" Lib/cgi.py

sed -i "s/python2.3/python2/g" Lib/distutils/tests/test_build_scripts.py \
Lib/distutils/tests/test_install_scripts.py

# Ensure that we are using the system copy of various libraries (expat, zlib and libffi),
# rather than copies shipped in the tarball
rm -r Modules/expat
rm -r Modules/zlib
rm -r Modules/_ctypes/{darwin,libffi}*

# clean up #!s
find . -name '*.py' | \
xargs sed -i "s|#[ ]![ ]/usr/bin/env python$|#!/usr/bin/env python2|"

# Workaround asdl_c.py/makeopcodetargets.py errors after we touched the shebangs
touch Include/Python-ast.h Python/Python-ast.c Python/opcode_targets.h

# FS#48761
# http://bugs.python.org/issue25750
patch -Np1 -i ../descr_ref.patch

export OPT="${CFLAGS}"
./configure --prefix=/usr \
--enable-shared \
--with-threads \
--enable-optimizations \
--with-lto \
--enable-ipv6 \
--enable-unicode=ucs4 \
--with-system-expat \
--with-system-ffi \
--with-dbmliborder=gdbm:ndbm \
--without-ensurepip

make

LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} \
xvfb-run "${srcdir}/Python-${pkgver}/python" -m test.regrtest -v -uall 2>&1 | tee log.all

vstinner · 2018-07-11T14:12:26Z

I don't see how test_idle runs the /usr/bin/xvfb-run command. First, it would be interesting to see what is the argument of the xvfb-run command and check if it does crash alone.

Can you please try to trace system calls using the following command and then attached the created trace file to this issue?

strace -f -o trace ./python -m test test_idle

test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... /usr/bin/xvfb-run: line 181: 3617 Segmentation fault (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1

What are values of DISPLAY, SERVERNUM and AUTHFILE environment variables?

eli-schwartz · 2018-07-11T14:53:20Z

Note the last line, which shows that the testsuite itself is executed using xvfb-run.

The archlinux32 build script is based off the original PKGBUILD for archlinux: https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python2

xvfb-run is only involved, because the testsuite depends on being run in a GUI environment in order to test GUI components...

vstinner · 2018-07-11T15:10:56Z

Ah, I see. Can you try to get the Python traceback and/or C traceback using gdb?

terryjreedy · 2018-07-13T19:42:36Z

I think we should first determine whether xvfb is required to get the segfault, and second, where it fails.

Does
<python2> -m test.regrtest -v -m test_io test_idle
fail and fail in the same place, the 11th and last test run in the module? If not, what about

xvfb-run <python2> -m ...?

In either case, inserting prints in 'test_writelines', which has multiple asserts, should determine where it fails.

Test.test_idle checks that tkinter can imported, which in turn imports _tkinter, which tries to connect to a windowing system. Running with xvfb satisfies this. Test_idle finds and run the test modules in idlelib.idle_test. Individual modules and classes somewhat redundantly import test.support and check "requires('gui')". So IDLE gui tests, and, I am fairly sure, the tkinter gui tests, also require -ugui or -uall to run.

Test_io is not a gui test, so does not need or run "requires('gui')", and hence does not need -ugui. On the other hand, it imports PyShell, which imports tkinter and _tkinter, and hence would need xvfb-run on a headless server, as opposed to a workstation with screen. Erich, I assume you are running on the latter.

deep-42-thought · 2018-07-16T12:39:09Z

Can you try to get the Python traceback and/or C traceback using gdb?
Sry, I can't - I don't know how to use gdb :-(
Is there some one-line command which makes gdb start xvfb-run and attach to it? Or do I need to start xvfb-run and separately attach gdb to it? (how do I do this?)

... and hence would need xvfb-run on a headless server, as opposed to a workstation with screen. Erich, I assume you are running on the latter.

I'm running this on a headless box (packaging for archlinux32). Setting up a desktop on the test box would be some effort, but should be possible in principle.

In the original PKGBUILD which Eli posted, there is a comment about when the tests started failing - which suggests, the problem is connected to xvfb-run.

deep-42-thought · 2018-07-16T12:57:24Z

ok, some update:

1st: running the complete test suite with xvfb fails (as already reported):

LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} \
xvfb-run "${srcdir}/Python-${pkgver}/python" -m test.regrtest -v -uall
...
0:02:14 load avg: 0.90 [174/403/1] test_idle
test_close (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_read (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readline (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readlines (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_close (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_write (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
/usr/bin/xvfb-run: line 181: 13491 Segmentation fault (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1

running w/o xvfb-run succeeds:

LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} ${srcdir}/Python-${pkgver}/python -m test.regrtest -v -m test_io test_idle
== CPython 2.7.15 (default, Jul 16 2018, 12:22:58) [GCC 8.1.1 20180531]
== Linux-4.17.4-1.0-ARCH-i686-with-glibc2.1.3 little-endian
== /home/vagrant/python2/src/Python-2.7.15/build/test_python_14208
== CPU count: 2
Run tests sequentially
0:00:00 load avg: 0.26 [1/1] test_idle
test_close (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_read (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readline (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readlines (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_close (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_write (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok

----------------------------------------------------------------------
Ran 11 tests in 0.003s

OK
1 test OK.

Total duration: 150 ms
Tests result: SUCCESS

running only these tests with xvfb-run _succeeds_:

LD_LIBRARY_PATH="${srcdir}/Python-${pkgver}":${LD_LIBRARY_PATH} xvfb-run ${srcdir}/Python-${pkgver}/python -m test.regrtest -v -m test_io test_idle
== CPython 2.7.15 (default, Jul 16 2018, 12:22:58) [GCC 8.1.1 20180531]
== Linux-4.17.4-1.0-ARCH-i686-with-glibc2.1.3 little-endian
== /home/vagrant/python2/src/Python-2.7.15/build/test_python_14226
== CPU count: 2
Run tests sequentially
0:00:00 load avg: 0.15 [1/1] test_idle
test_close (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_read (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readline (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_readlines (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeInputFilesTest) ... ok
test_close (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_misc (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_unsupported (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_write (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok
test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... ok

----------------------------------------------------------------------
Ran 11 tests in 0.004s

OK
1 test OK.

Total duration: 144 ms
Tests result: SUCCESS

yan12125 · 2018-10-11T06:50:00Z

I got a similar error and managed to fix it by installing the noto-fonts package [1]. ttf-dejavu [2] works, too. I guess at least one font should be installed to allow those tests to run.

[1] https://www.archlinux.org/packages/extra/any/noto-fonts/
[2] https://www.archlinux.org/packages/extra/any/ttf-dejavu/

yan12125 · 2018-10-11T07:04:36Z

Here is partial backtrace from commit 1d26c72:

#0 0x00007ffff6763197 in GetTkFontAttributes (faPtr=faPtr@entry=0x555555cd91e8, ftFont=<optimized out>)
at /usr/src/debug/tk8.6.8/unix/../unix/tkUnixRFont.c:167
#1 0x00007ffff67639fa in InitFont (tkwin=tkwin@entry=0x555555ee94a0, pattern=pattern@entry=0x555555cfc690, fontPtr=0x555555cd91b0,
fontPtr@entry=0x0) at /usr/src/debug/tk8.6.8/unix/../unix/tkUnixRFont.c:313
#2 0x00007ffff6763c4e in TkpGetFontFromAttributes (tkFontPtr=tkFontPtr@entry=0x0, tkwin=tkwin@entry=0x555555ee94a0,
faPtr=0x555555d1e8b8) at /usr/src/debug/tk8.6.8/unix/../unix/tkUnixRFont.c:482
#3 0x00007ffff66b5ee2 in Tk_AllocFontFromObj (interp=0x555555c4b900, tkwin=tkwin@entry=0x555555ee94a0, objPtr=0x555555e39000)
at /usr/src/debug/tk8.6.8/unix/../generic/tkFont.c:1158
#4 0x00007ffff66ac053 in DoObjConfig (interp=interp@entry=0x555555c4b900, recordPtr=recordPtr@entry=0x555555e53390 "0r\354UUU",
optionPtr=optionPtr@entry=0x555555f4e010, valuePtr=<optimized out>, valuePtr@entry=0x555555e39000,
tkwin=tkwin@entry=0x555555ee94a0, savedOptionPtr=savedOptionPtr@entry=0x0) at /usr/src/debug/tk8.6.8/unix/../generic/tkConfig.c:717
#5 0x00007ffff66ace92 in Tk_InitOptions (interp=interp@entry=0x555555c4b900, recordPtr=recordPtr@entry=0x555555e53390 "0r\354UUU",
optionTable=optionTable@entry=0x555555f4de10, tkwin=0x555555ee94a0) at /usr/src/debug/tk8.6.8/unix/../generic/tkConfig.c:496
#6 0x00007ffff67250b2 in CreateWidget (sharedPtr=0x555555ec7230, tkwin=<optimized out>, interp=0x555555c4b900,
parent=<optimized out>, objc=<optimized out>, objv=<optimized out>) at /usr/src/debug/tk8.6.8/unix/../generic/tkText.c:664
#7 0x00007ffff62efe16 in TclNRRunCallbacks () from /usr/lib/libtcl8.6.so
#8 0x00007ffff67eba0c in Tkapp_Call (selfptr=0x7ffff40242b0, args=<optimized out>)
at /home/yen/Projects/cpython/Modules/_tkinter.c:1508

Looks more like a Tk issue than a CPython one.

For Arch Linux packagers: adding checkdepends=('ttf-font') to PKGBUILD can fix testing errors. Here's an example: archlinuxcn/repo@e4de459

terryjreedy · 2018-10-11T15:06:29Z

Additional notes:

I am puzzled that test_idle output starts with test_io, as that is alphabetically in the middle of the files in idle_test/.

There are 2 independent 'test_io' modules: test/test_io, which tests the io module, and idlelib/test_idle/test_io, which tests idlelib.PyShell.PseudoFile and subclasses (which are all derived from io.TextIOBase). Only the latter is an issue here. The former is an irrelevant distraction and should not have been included with test_idle in msg321737.

Since log.all ends with "test_writelines (idlelib.idle_test.test_io.PseudeOutputFilesTest) ... /usr/bin/xvfb-run: line 181: ...", I initially thought that test_writelines, in particular, stimulated the failure. But in the msg321737 output, test_writelines ran 'OK'.

The fact that xvfv ... test_idle runs by itself but not when alphabetically preceding tests are run says that some preceding test alters something. The failure of test_audioop seems an unlikely cause. I suspect test__all__, by importing _tkinter either directly or via Tkinter or some IDLE module. This could be tested with xvfb ... test___all__ test_idle.

Since the tests are mostly run alphabetically and the tkinter tests, test_tcl, test_tk, test_ttk_guionly, and test_ttk_textonly alphabetically follow test_idle, they should not be the cause of the crash during test_idle. On the other hand, if the test above crashes, and so does
xvbf ... -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_textonly
then test_idle would be eliminated as the direct culprit.

yan12125 · 2018-10-13T05:33:12Z

Sorry, after reading the original issue carefully, I realize that my issue is slightly different. In my case running test_idle only crashes, and it crashes before the first test in test_idle, not in test_writelines.

I copied the Arch Linux packaging script mentioned by Eli [2], modified it so that test_idle, test_tk and test_ttk_guionly are not skipped, and added a dependency ttf-font. With the modified PKGBUILD, the only failing test is test_ssl. I guess the issue has been fixed.

Erich Eckner, could you still reproduce the original crash?

[1] https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python2

deep-42-thought · 2018-10-15T07:58:48Z

Yes, I get much further in the test suite, currently (40k lines logged vs. 23k). Now, it succeeds until:

0:08:26 load avg: 0.72 [358/403/3] test_tuple
test_addmul (test.test_tuple.TupleTest) ... ok
test_bigrepeat (test.test_tuple.TupleTest) ... /usr/bin/xvfb-run: line 181: 231
8 Segmentation fault (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE
"$@" 2>&1

yan12125 · 2018-10-15T08:49:09Z

The new crashing test seems the same as https://bugs.python.org/issue33153. I can't reproduce it on 64-bit Arch Linux, so that might be a 32-bit only issue.

terryjreedy · 2018-10-15T17:48:14Z

It is not clear what you did differently to get a different result. I changed the title since it seems that idlelib code is not the direct problem.

deep-42-thought · 2018-10-15T18:08:17Z

yes, I'm also not sure what's different now - I executed the very same commands on the very same source tree :-/

benjaminp · 2020-01-28T23:34:34Z

2.7 is dead.

deep-42-thought mannequin added the build The build process and cross-build label Jul 11, 2018

vstinner changed the title ~~segfault in testsuite~~ [2.7] test_idle fails with: /usr/bin/xvfb-run: line 181: 3617 Segmentation fault Jul 11, 2018

terryjreedy changed the title ~~[2.7] test_idle fails with: /usr/bin/xvfb-run: line 181: 3617 Segmentation fault~~ [2.7] Seg fault on archlinux 32 when run tests with xvfb-run Oct 15, 2018

terryjreedy added the type-crash A hard crash of the interpreter, possibly with a core dump label Oct 15, 2018

benjaminp closed this as completed Jan 28, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[2.7] Seg fault on archlinux 32 when run tests with xvfb-run #78276

[2.7] Seg fault on archlinux 32 when run tests with xvfb-run #78276

deep-42-thought mannequin commented Jul 11, 2018

deep-42-thought mannequin commented Jul 11, 2018

vstinner commented Jul 11, 2018

eli-schwartz mannequin commented Jul 11, 2018

vstinner commented Jul 11, 2018

terryjreedy commented Jul 13, 2018

deep-42-thought mannequin commented Jul 16, 2018

deep-42-thought mannequin commented Jul 16, 2018

yan12125 mannequin commented Oct 11, 2018

yan12125 mannequin commented Oct 11, 2018

terryjreedy commented Oct 11, 2018

yan12125 mannequin commented Oct 13, 2018

deep-42-thought mannequin commented Oct 15, 2018

yan12125 mannequin commented Oct 15, 2018

terryjreedy commented Oct 15, 2018

deep-42-thought mannequin commented Oct 15, 2018

benjaminp commented Jan 28, 2020

[2.7] Seg fault on archlinux 32 when run tests with xvfb-run #78276

[2.7] Seg fault on archlinux 32 when run tests with xvfb-run #78276

Comments

deep-42-thought mannequin commented Jul 11, 2018

deep-42-thought mannequin commented Jul 11, 2018

vstinner commented Jul 11, 2018

eli-schwartz mannequin commented Jul 11, 2018

vstinner commented Jul 11, 2018

terryjreedy commented Jul 13, 2018

deep-42-thought mannequin commented Jul 16, 2018

deep-42-thought mannequin commented Jul 16, 2018

yan12125 mannequin commented Oct 11, 2018

yan12125 mannequin commented Oct 11, 2018

terryjreedy commented Oct 11, 2018

yan12125 mannequin commented Oct 13, 2018

deep-42-thought mannequin commented Oct 15, 2018

yan12125 mannequin commented Oct 15, 2018

terryjreedy commented Oct 15, 2018

deep-42-thought mannequin commented Oct 15, 2018

benjaminp commented Jan 28, 2020