tokenizer.c memory leak in case of realloc failure

[stratakis]

BPO	36367
Nosy	@vstinner, @stratakis, @pablogsal
PRs	bpo-36367: Free buffer if realloc fails in tokenize.c #12442 [2.7] bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442) #12470 [3.7] bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442) #12471

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-03-19.17:18:20.933>
created_at = <Date 2019-03-19.16:12:41.176>
labels = ['3.7', '3.8']
title = 'tokenizer.c memory leak in case of realloc failure'
updated_at = <Date 2019-03-20.12:03:56.804>
user = 'https://github.com/stratakis'

bugs.python.org fields:

activity = <Date 2019-03-20.12:03:56.804>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-03-19.17:18:20.933>
closer = 'pablogsal'
components = []
creation = <Date 2019-03-19.16:12:41.176>
creator = 'cstratak'
dependencies = []
files = []
hgrepos = []
issue_num = 36367
keywords = ['patch']
message_count = 4.0
messages = ['338375', '338391', '338459', '338460']
nosy_count = 3.0
nosy_names = ['vstinner', 'cstratak', 'pablogsal']
pr_nums = ['12442', '12470', '12471']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36367'
versions = ['Python 2.7', 'Python 3.7', 'Python 3.8']

[stratakis]

In tokenizer.c we have those lines of code [0]:

if (final_length < needed_length && final_length)
/* should never fail */
buf = PyMem_REALLOC(buf, final_length);
return buf;

If however that realloc fails, the memory allocated initially for buf, will not be freed.

[0] https://github.com/python/cpython/blob/master/Parser/tokenizer.c#L652

[pablogsal]

New changeset cb90c89 by Pablo Galindo in branch 'master':
bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442)
cb90c89

[vstinner]

New changeset 65b9849 by Victor Stinner in branch '3.7':
bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442) (GH-12471)
65b9849

[vstinner]

New changeset 469b0a5 by Victor Stinner in branch '2.7':
bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442) (GH-12470)
469b0a5