PyString_AsString() segfaults when passed a unicode string

[zopyx]

BPO	1082085
Nosy	@tim-one, @zopyx

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2004-12-09.15:05:51.000>
created_at = <Date 2004-12-09.13:08:37.000>
labels = ['interpreter-core', 'invalid']
title = 'PyString_AsString() segfaults when passed a unicode string'
updated_at = <Date 2004-12-09.15:05:51.000>
user = 'https://github.com/zopyx'

bugs.python.org fields:

activity = <Date 2004-12-09.15:05:51.000>
actor = 'tim.peters'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2004-12-09.13:08:37.000>
creator = 'ajung'
dependencies = []
files = []
hgrepos = []
issue_num = 1082085
keywords = []
message_count = 5.0
messages = ['23618', '23619', '23620', '23621', '23622']
nosy_count = 4.0
nosy_names = ['tim.peters', 'anthonybaxter', 'dcjim', 'ajung']
pr_nums = []
priority = 'high'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1082085'
versions = []

[zopyx]

If you pass a PyObject representation the unicode
string u'\xc4'
to PyString_AsString() then Python (2.3.4, 2.4.0) segfault.

Famous last words of Python 2.4:

Exception exceptions.UnicodeEncodeError:
<exceptions.UnicodeEncodeError instance at 0xf6f75e8c>
in 'garbage collection' ignored
Fatal Python error: unexpected exception during garbage
collection

Famous last words of Python 2.4 (debug build):

XXX undetected error
Traceback (most recent call last):
  File "test.py", line 4, in ?
    print S.split(u'\xc4')
UnicodeEncodeError: 'ascii' codec can't encode
character u'\xe4' in position 0: ordinal not in range(128)
[6545 refs]

This bug has been reported first on the zope-dev list.
I confirmed
that this error does not only occur in Zope but also in
a Python-only
environment.

[zopyx]

Logged In: YES
user_id=11084

This error not only happens with u'\xc4', it happens with
*any* string
containing a character >0x7f.

[anthonybaxter]

Logged In: YES
user_id=29957

Please attach a test case that shows the failure. I can't
reproduce it here with the information you've given.

[dcjim]

Logged In: YES
user_id=73023

I can't reproduce this:

>>> ick = u'\xc4'
>>> import struct
>>> struct.pack(ick)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character
u'\xc4' in position 0: ordinal not in range(128)
>>> struct.pack(u'i', 1)
'\x01\x00\x00\x00'

The code that provked this in Zope is buggy.,

[tim-one]

Logged In: YES
user_id=31435

For the record, this appeared to be due to an extension
module doing

PyString_AsString(name)[0]

when name was a Unicode string containing a "high-bit"
character. PyString_AsString(name) legitimately returned
NULL, and bad stuff followed.