tokenizer.c does not handle new_string() failure

[vstinner]

BPO	18470
Nosy	@vstinner, @benjaminp

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-07-16.03:47:54.240>
created_at = <Date 2013-07-16.01:32:23.554>
labels = []
title = 'tokenizer.c does not handle new_string() failure'
updated_at = <Date 2013-07-16.03:47:54.239>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2013-07-16.03:47:54.239>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2013-07-16.03:47:54.240>
closer = 'python-dev'
components = []
creation = <Date 2013-07-16.01:32:23.554>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 18470
keywords = []
message_count = 4.0
messages = ['193143', '193149', '193150', '193152']
nosy_count = 3.0
nosy_names = ['vstinner', 'benjamin.peterson', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue18470'
versions = ['Python 3.4']

[vstinner]

Callers of the new_string() function do not check if the function succeed or not. Python does crash if the function failed, for example in get_coding_spec():

                char* r = new_string(begin, t - begin);
                char* q = get_normal_name(r);

Using pytracemalloc tool, it's easy to tricker this issue (ex: using test_future or test_parser): see issue bpo-18408.

[python-dev]

New changeset c3a510b22218 by Benjamin Peterson in branch '3.3':
check the return value of new_string() (closes bpo-18470)
http://hg.python.org/cpython/rev/c3a510b22218

New changeset 8889c9b5dd3a by Benjamin Peterson in branch '3.3':
merge 3.3 (bpo-18470)
http://hg.python.org/cpython/rev/8889c9b5dd3a

[vstinner]

The fix has not been merged into default.

[python-dev]

New changeset 2650127ce034 by Benjamin Peterson in branch 'default':
merge 3.3 (closes bpo-18470)
http://hg.python.org/cpython/rev/2650127ce034