Crash while compiling Python 3000 in OpenBSD 4.4

[henryprecheur]

BPO	3685
Nosy	@loewis, @warsaw, @hyeshik
Files	fix_wcschr_openbsd.diff fix_wcschr_generic.patch wcslen.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-10-17.01:15:53.536>
created_at = <Date 2008-08-26.06:52:33.665>
labels = ['interpreter-core', 'type-crash', 'release-blocker']
title = 'Crash while compiling Python 3000 in OpenBSD 4.4'
updated_at = <Date 2008-10-17.01:15:53.506>
user = 'https://bugs.python.org/henryprecheur'

bugs.python.org fields:

activity = <Date 2008-10-17.01:15:53.506>
actor = 'barry'
assignee = 'none'
closed = True
closed_date = <Date 2008-10-17.01:15:53.536>
closer = 'barry'
components = ['Interpreter Core']
creation = <Date 2008-08-26.06:52:33.665>
creator = 'henry.precheur'
dependencies = []
files = ['11265', '11369', '11537']
hgrepos = []
issue_num = 3685
keywords = ['patch', 'needs review']
message_count = 9.0
messages = ['71968', '71969', '71977', '72008', '72009', '72472', '73498', '73941', '74886']
nosy_count = 4.0
nosy_names = ['loewis', 'barry', 'hyeshik.chang', 'henry.precheur']
pr_nums = []
priority = 'release blocker'
resolution = 'accepted'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue3685'
versions = ['Python 3.0']

[henryprecheur]

I tried to compile Python 3000 under OpenBSD and the compilation fails
because of a 'MemoryError':

Fatal Python error: can't create sys.path
object : MemoryError()
type : MemoryError
refcount: 4
address : 0x20abfbd08
lost sys.stderr
Abort trap (core dumped)
*** Error code 134

Stop in /home/henry/compile/py3k (line 410 of Makefile).

The command which fail is:
CC='gcc -pthread' LDSHARED='gcc -pthread -shared -fPIC ' OPT='-DNDEBUG
-g -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py build

Here is the backtrace:

(gdb) r -E ./setup.py build
Starting program: /home/henry/compile/py3k/python -E ./setup.py build
Fatal Python error: can't create sys.path
object : MemoryError()
type : MemoryError
refcount: 4
address : 0x2042d3d08
lost sys.stderr

Program received signal SIGABRT, Aborted.
[Switching to process 20134, thread 0x2015d4800]
0x000000020dc4432a in kill () from /usr/lib/libc.so.48.0
(gdb) bt full
#0 0x000000020dc4432a in kill () from /usr/lib/libc.so.48.0
No symbol table info available.
#1 0x000000020dc8b105 in abort () at /usr/src/lib/libc/stdlib/abort.c:68
p = (struct atexit *) 0x2064fd000
cleanup_called = 1
mask = 4294967263
#2 0x0000000000468a59 in Py_FatalError (msg=0x4ea6 <Address 0x4ea6 out
of bounds>)
at Python/pythonrun.c:1880
No locals.
#3 0x000000000046e06c in PySys_SetPath (path=0x4ea6) at
Python/sysmodule.c:1390
v = (PyObject *) 0x0
#4 0x0000000000466b8c in Py_InitializeEx (install_sigs=1) at
Python/pythonrun.c:213
interp = (PyInterpreterState *) 0x20f8af900
tstate = (PyThreadState *) 0x20adeda00
bimod = (PyObject *) 0x2042dc128
sysmod = (PyObject *) 0x2042dc128
pstderr = (PyObject *) 0x2042dc128
p = 0x0
codeset = 0x2042dc128 "\034"
#5 0x0000000000474136 in Py_Main (argc=4, argv=0x20f0331a0) at
Modules/main.c:497
r1 = 0
r2 = 0
c = 0
sts = 4
command = 0x0
filename = (wchar_t *) 0x0
module = 0x0
fp = (FILE *) 0x964e70
p = 0x6c05 <Address 0x6c05 out of bounds>
unbuffered = 0
skipfirstline = 0
stdin_is_interactive = 1
help = 0
version = 0
saw_unbuffered_flag = 0
cf = {cf_flags = 0}
#6 0x0000000000412866 in main (argc=4, argv=0x7f7ffffc7920) at
Modules/python.c:57
argsize = 140187732310304
argv_copy = (wchar_t **) 0x20f0331a0
argv_copy2 = (wchar_t **) 0x20f033140
i = 0
res = -231136
oldloc = 0x20e0c1b00 "C"

I also have core file. If you are interested mail me.

[henryprecheur]

I forgot to mention, I made to following modification to configure.in so
I could compile Python 3000 on OpenBSD 4.4

--- configure.in        (revision 66037)
+++ configure.in        (working copy)
@@ -250,7 +250,7 @@
   # On OpenBSD, select(2) is not available if _XOPEN_SOURCE is defined,
   # even though select is a POSIX function. Reported by J. Ribbens.
   # Reconfirmed for OpenBSD 3.3 by Zachary Hamm, for 3.4 by Jason Ish.
-  OpenBSD/2.* | OpenBSD/3.@<:@0123456789@:>@ | OpenBSD/4.@<:@0123@:>@) 
+  OpenBSD*)
     define_xopen_source=no
     # OpenBSD undoes our definition of __BSD_VISIBLE if _XOPEN_SOURCE is
     # also defined. This can be overridden by defining _BSD_SOURCE

[hyeshik]

This problem is due to OpenBSD's libc bug.
It's fixed 3 days ago. (http://www.openbsd.org/cgi-
bin/cvsweb/src/lib/libc/string/wcschr.c#rev1.4)
We can workaround by replacing use of wcschr(ws, L'\0') to ws +
wcslen(ws).

[henryprecheur]

Indeed it looks like it is the source of the problem.

I created a patch to fix it.

But it looks like there is another problem, instead of crashing the
Python interpreter goes into interactive mode instead of executing the
'setup.py' script ...

I don't think it is related, I have checked the result of ws = ws +
wcslen(ws) and it seems to be correct.

I will investigate the problem and create another entry if it is unrelated.

[henryprecheur]

Looks like my other issue is unrelated. It is caused by a buggy mbstowcs.

[henryprecheur]

Here is a better patch which use the workaround only if wcschr is broken.

I was able to build the python interpreter and to run regrtest.py with
it (some tests fail but it is very likely to be bugs in the modules)

[loewis]

Is there any problem with always computing the end of the string as "s +
wcslen(s)"? I feel that this is actually more readable than wcschr.

Here is a patch that does that. It doesn't change PC/getpathp.c, since
it's only used on Windows, where wcschr works correctly (AFAIK).

Henry, can you confirm that this patch still fixes the problem?

[henryprecheur]

I just tested the patch and it fixes the problem.

[warsaw]

r66948