locale.getpreferredencoding() gives bus error on Mac OS X 10.4.11 PPC

[cfr]

BPO	3362
Nosy	@loewis, @ronaldoussoren, @pitrou, @ned-deily
Superseder	bpo-6202: Obsolete default file encoding "mac-roman" on OS X, not influenced by locale env variables

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ronaldoussoren'
closed_at = <Date 2010-10-26.18:16:58.555>
created_at = <Date 2008-07-15.14:22:29.925>
labels = ['OS-mac', 'type-crash']
title = 'locale.getpreferredencoding() gives bus error on Mac OS X 10.4.11 PPC'
updated_at = <Date 2010-10-26.18:16:58.553>
user = 'https://bugs.python.org/cfr'

bugs.python.org fields:

activity = <Date 2010-10-26.18:16:58.553>
actor = 'ned.deily'
assignee = 'ronaldoussoren'
closed = True
closed_date = <Date 2010-10-26.18:16:58.555>
closer = 'ned.deily'
components = ['macOS']
creation = <Date 2008-07-15.14:22:29.925>
creator = 'cfr'
dependencies = []
files = []
hgrepos = []
issue_num = 3362
keywords = []
message_count = 24.0
messages = ['69683', '69685', '69687', '69717', '69744', '69746', '69752', '69775', '69860', '69950', '70164', '70937', '70941', '71038', '71046', '71074', '71075', '71079', '71081', '71102', '71104', '71108', '119599', '119627']
nosy_count = 6.0
nosy_names = ['loewis', 'ronaldoussoren', 'janssen', 'pitrou', 'ned.deily', 'cfr']
pr_nums = []
priority = 'critical'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '6202'
type = 'crash'
url = 'https://bugs.python.org/issue3362'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2']

[cfr]

Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007;
root:xnu-792.24.17~1/RELEASE_PPC

Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, sys, locale
>>> locale.getpreferredencoding()
Bus error

Sample crash report excerpt follows (plenty more available on request!).
Note that the version of python given in the crash report is *not* the
same as the version of python actually in use. I have never had an alpha
version of python installed. The current version is the standard version
of 2.5.2 available as a dmg download from python.org i.e. the universal
framework build for 10.4.

OS Version: 10.4.11 (Build 8S165)
Report Version: 4

Command: Python
Path:
/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python
Parent: bash [27154]

Version: 2.5a0 (2.5alpha0)

PID: 4692
Thread: 0

Exception: EXC_BAD_ACCESS (0x0001)
Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x00000000

Thread 0 Crashed:
0 com.apple.CoreFoundation 0x907beac0 CFStringGetCStringPtr + 408
1 _locale.so 0x000f1cd8 PyLocale_getdefaultlocale + 328
(_localemodule.c:435)
2 org.python.python 0x002b393c PyEval_EvalFrameEx + 17036
(ceval.c:3557)
3 org.python.python 0x002b5e50 PyEval_EvalCodeEx + 2096
(ceval.c:2836)
4 org.python.python 0x002b3f48 PyEval_EvalFrameEx + 18584
(ceval.c:3669)
5 org.python.python 0x002b5e50 PyEval_EvalCodeEx + 2096
(ceval.c:2836)
6 org.python.python 0x002b5ff0 PyEval_EvalCode + 48 (ceval.c:500)
7 org.python.python 0x002dbb24 PyRun_InteractiveOneFlags + 772
(pythonrun.c:1274)
8 org.python.python 0x002dbd30 PyRun_InteractiveLoopFlags +
288 (pythonrun.c:725)
9 org.python.python 0x002dc3f0 PyRun_AnyFileExFlags + 176
(pythonrun.c:693)
10 org.python.python 0x002eba9c Py_Main + 3052 (main.c:523)
11 org.python.python 0x000019bc 0x1000 + 2492
12 org.python.python 0x000016c0 0x1000 + 1728

Thread 0 crashed with PPC Thread State 64:
srr0: 0x00000000907beac0 srr1: 0x000000000000d030
vrsave: 0x0000000000000000
cr: 0x84244224 xer: 0x0000000020000004 lr:
0x00000000907be930 ctr: 0x00000000907be928
r0: 0x00000000a07bb678 r1: 0x00000000bfffd3a0 r2:
0x00000000a07bb278 r3: 0x0000000000000000
r4: 0x0000000000000000 r5: 0x00000000bfffd2e0 r6:
0x0000000000000005 r7: 0x0000000000000007
r8: 0x0000000000702333 r9: 0x000000000000001c r10:
0x0000000090bb4bb8 r11: 0x00000000000f33d8
r12: 0x00000000907be928 r13: 0x0000000000058b24 r14:
0x0000000000071e40 r15: 0x000000000006aa20
r16: 0x0000000000000000 r17: 0x0000000000000001 r18:
0x00000000000732a8 r19: 0x0000000000619410
r20: 0x0000000000000000 r21: 0x000000000006a9b0 r22:
0x0000000000000000 r23: 0x0000000000058b39
r24: 0x00000000000f1b90 r25: 0x00000000006001d0 r26:
0x000000000007e300 r27: 0x0000000000000000
r28: 0x0000000000000000 r29: 0x00000000a07bbb6c r30:
0x0000000000000000 r31: 0x00000000907be930

Binary Images Description:
0x1000 - 0x1fff org.python.python 2.5a0 (2.5alpha0)
/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python
0xa2000 - 0xd9fff readline.so
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/readline.so
0xf0000 - 0xf2fff _locale.so
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/_locale.so
0x205000 - 0x323fff org.python.python 2.5a0 (2.5)
/Library/Frameworks/Python.framework/Versions/2.5/Python
0x705000 - 0x74afff libncurses.5.dylib
/Library/Frameworks/Python.framework/Versions/2.5/lib/libncurses.5.dylib
0x7de000 - 0x7e1fff operator.so
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/operator.so
0x8fe00000 - 0x8fe52fff dyld 46.16 /usr/lib/dyld
0x90000000 - 0x901bcfff libSystem.B.dylib /usr/lib/libSystem.B.dylib
0x90214000 - 0x90219fff libmathCommon.A.dylib
/usr/lib/system/libmathCommon.A.dylib
0x907bb000 - 0x90895fff com.apple.CoreFoundation 6.4.11 (368.35)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
0x908e0000 - 0x909e2fff libicucore.A.dylib /usr/lib/libicucore.A.dylib
0x90a3c000 - 0x90ac0fff libobjc.A.dylib /usr/lib/libobjc.A.dylib
0x90aea000 - 0x90b5afff com.apple.framework.IOKit 1.4 (???)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
0x90b70000 - 0x90b82fff libauto.dylib /usr/lib/libauto.dylib
0x90b89000 - 0x90e60fff com.apple.CoreServices.CarbonCore 681.17
/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore
0x91111000 - 0x9111ffff libz.1.dylib /usr/lib/libz.1.dylib
0x91122000 - 0x912ddfff com.apple.security 4.6 (29770)
/System/Library/Frameworks/Security.framework/Versions/A/Security
0x913dc000 - 0x913e5fff com.apple.DiskArbitration 2.1.2
/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
0x913ec000 - 0x913f4fff libbsm.dylib /usr/lib/libbsm.dylib
0x913f8000 - 0x91420fff com.apple.SystemConfiguration 1.8.3
/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
0x91433000 - 0x9143efff libgcc_s.1.dylib /usr/lib/libgcc_s.1.dylib
0x945e4000 - 0x94604fff libmx.A.dylib /usr/lib/libmx.A.dylib

I have got as far as I can tracking this issue down but would be happy
to provide further information if somebody would give me (a pointer to)
instructions or a hint.

[cfr]

Although the active version of python on my machine is 2.5.2 and I have
never had an alpha version installed, crash reports for python report
the version as "2.5a0 (2.5alpha0)".

Version details: active version of python is from the current python.org
dmg download for Mac OS X 10.4 i.e. the universal framework build. When
starting python, I get:

Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

but in crash reports, I get:

Command: Python
Path:
/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python
Parent: bash [27154]

Version: 2.5a0 (2.5alpha0)

and python is given as version 2.5a0 in the binary image listing which
follows.

Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007;
root:xnu-792.24.17~1/RELEASE_PPC

I think I did have 2.5.1 installed prior to installing 2.5.2 and I also
have two older versions of python installed - 2.4 (also the python.org
build) and 2.3 (as pre-installed by Apple) - but I never installed 2.5.0
or any version/candidate in the 2.5 line prior to 2.5.1.

I'm not sure what further information might be helpful but would be
happy to provide it on request.

[cfr]

Please ignore the second message. I thought I was creating a second bug
report and cannot figure out anyway to edit it now I realise my error.
I've just copied that to a second report with an appropriate header as I
am assuming the two issues I'm seeing are distinct.

This bug report is intended to cover the bus error I see triggered by
locale.getpreferredencoding().

[loewis]

It would be good to find out what the code inside
PyLocale_getdefaultlocale precisely is doing. I.e. what is the value of
name at that point, and is that perhaps an illegal parameter for
CFStringGetCStringPtr somehow? If the debugger can't help (not even
after compiling Python with --with-pydebug), augment the source code
with a printf, and rebuild Python.

[cfr]

I downloaded the current source (2.5.2) and confirmed that (1) python
will build as a framework (for me) and (2) that the problem occurs for
my build, too. I did not build it as a universal binary just in case
that helped but mostly to speed things up.

I then tried to add the --with-pydebug flag to my configure script and
build that way. I used separate build directories for the two builds to
keep the source clean. Unfortunately, make fails with the following
error in that case:

if test ""; then \
gcc -o Python.framework/Versions/2.5/Python -arch i386 -arch ppc
-dynamiclib \
-isysroot "" \
-all_load libpython2.5.a -Wl,-single_module \
-install_name
/Library/Frameworks/Python.framework/Versions/2.5/Python \
-compatibility_version 2.5 \
-current_version 2.5; \
else \
/usr/bin/libtool -o Python.framework/Versions/2.5/Python
-dynamic libpython2.5.a \
-lSystem -lSystemStubs -arch_only ppc -install_name
/Library/Frameworks/Python.framework/Versions/2.5/Python
-compatibility_version 2.5 -current_version 2.5 ;\
fi
ld: Undefined symbols:
___eprintf
/usr/bin/libtool: internal link edit command failed
make: *** [Python.framework/Versions/2.5/Python] Error 1

I am guessing that eprintf has something to do with the debug option
because the symbol occurs in the debug version of libpython2.5.a but not
the plain version as far as I can tell. But I'm not sure how to fix it.

I am not even sure I am running the debugger correctly with the existing
version of python. I tried passing "-v -v" and "-d" to python after
reading the man page and that didn't get me any extra information.
Nothing useful-looking, at least. ("-v -v" produced a lot of output
beforehand but not around the point the error occurs.) Is that what you
meant or should I be looking at something else?

I am sorry but I don't know how to augment the source code with printf
and that is such a common term I'm not sure what to google to find
instructions for doing it.

[cfr]

I figured out how to do this:

Python 2.5.2 (r252:60911, Jul 16 2008, 01:44:22) 
[GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pdb
>>> import sys, os, locale
>>> pdb.run('locale.getpreferredencoding()')                      
> <string>(1)<module>()
(Pdb) continue
Bus error

though I'm not sure if that is what I was meant to do either. (But it
strikes me as another plausible possibility.)

[loewis]

The Python debugger (pdb) won't help here; you'ld have to use the system
debugger (gdb).

Please add the line

  printf("The value of name is %p\n", name);
  printf("It points to '%s'\n", name);

right before the call to CFStringGetCStringPtr in Modules/_localemodule.c

[cfr]

Thanks.

I couldn't get anything from gdb which wasn't already in the crash log -
likely because I don't know how to elicit the information correctly.

Output from a build with the augmented _localemodule.c:

./python.exe 
Python 2.5.2 (r252:60911, Jul 16 2008, 01:44:22) 
[GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin
iType "help", "copyright", "credits" or "license" for more information.
>>> import os, sys, locale
>>> locale.getpreferredencoding()
The value of name is 0x0
It points to '(null)'
Bus error