Buffer overflow in bad_single_statement (Parser/pegen.c)

[pablogsal]

BPO	42150
Nosy	@lysnikolaou, @pablogsal, @miss-islington
PRs	bpo-42150: Avoid buffer overflow in the new parser #22978 [3.9] bpo-42150: Avoid buffer overflow in the new parser (GH-22978) #22979

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-25.23:03:53.522>
created_at = <Date 2020-10-25.21:11:23.396>
labels = ['interpreter-core', '3.10']
title = 'Buffer overflow in bad_single_statement (Parser/pegen.c)'
updated_at = <Date 2020-10-25.23:25:04.619>
user = 'https://github.com/pablogsal'

bugs.python.org fields:

activity = <Date 2020-10-25.23:25:04.619>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-25.23:03:53.522>
closer = 'pablogsal'
components = ['Interpreter Core']
creation = <Date 2020-10-25.21:11:23.396>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 42150
keywords = ['patch']
message_count = 5.0
messages = ['379607', '379608', '379609', '379617', '379619']
nosy_count = 3.0
nosy_names = ['lys.nikolaou', 'pablogsal', 'miss-islington']
pr_nums = ['22978', '22979']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue42150'
versions = ['Python 3.10']

[pablogsal]

Building Python with the address sanitizer and executing ./python -m test test_repl -v shows this:

===========================================================
==26216==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200003990f at pc 0x5625406527b2 bp 0x7ffe8d621910 sp 0x7ffe8d621900
READ of size 1 at 0x60200003990f thread T0
#0 0x5625406527b1 in bad_single_statement Parser/pegen.c:993
#1 0x5625406527b1 in _PyPegen_run_parser Parser/pegen.c:1136
#2 0x5625406529b8 in _PyPegen_run_parser_from_file_pointer Parser/pegen.c:1179
#3 0x5625403f1945 in PyRun_InteractiveOneObjectEx Python/pythonrun.c:208
#4 0x5625403f3cd9 in PyRun_InteractiveLoopFlags Python/pythonrun.c:115
#5 0x5625403f4e3f in PyRun_AnyFileExFlags Python/pythonrun.c:74
#6 0x562540181792 in pymain_run_stdin Modules/main.c:512
#7 0x562540181792 in pymain_run_python Modules/main.c:601
#8 0x562540182904 in Py_RunMain Modules/main.c:677
#9 0x562540182904 in pymain_main Modules/main.c:707
#10 0x562540182904 in Py_BytesMain Modules/main.c:731
#11 0x7f25297aa151 in __libc_start_main (/usr/lib/libc.so.6+0x28151)
#12 0x56254017f0bd in _start (/home/pablogsal/github/python/master/python+0x1640bd)

0x60200003990f is located 1 bytes to the left of 2-byte region [0x602000039910,0x602000039912)
allocated by thread T0 here:
#0 0x7f2529b70459 in __interceptor_malloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0x562540553416 in tok_nextc Parser/tokenizer.c:872
#2 0x562540554eb7 in tok_get Parser/tokenizer.c:1190
#3 0x562540559590 in PyTokenizer_Get Parser/tokenizer.c:1839
#4 0x56254064f6dd in _PyPegen_fill_token Parser/pegen.c:584
#5 0x5625406934a7 in statement_newline_rule Parser/parser.c:1296
#6 0x5625406934a7 in interactive_rule Parser/parser.c:769
#7 0x5625406934a7 in _PyPegen_parse Parser/parser.c:24823
#8 0x562540652382 in _PyPegen_run_parser Parser/pegen.c:1111
#9 0x5625406529b8 in _PyPegen_run_parser_from_file_pointer Parser/pegen.c:1179
#10 0x5625403f1945 in PyRun_InteractiveOneObjectEx Python/pythonrun.c:208
#11 0x5625403f3cd9 in PyRun_InteractiveLoopFlags Python/pythonrun.c:115
#12 0x5625403f4e3f in PyRun_AnyFileExFlags Python/pythonrun.c:74
#13 0x562540181792 in pymain_run_stdin Modules/main.c:512
#14 0x562540181792 in pymain_run_python Modules/main.c:601
#15 0x562540182904 in Py_RunMain Modules/main.c:677
#16 0x562540182904 in pymain_main Modules/main.c:707
#17 0x562540182904 in Py_BytesMain Modules/main.c:731
#18 0x7f25297aa151 in __libc_start_main (/usr/lib/libc.so.6+0x28151)

SUMMARY: AddressSanitizer: heap-buffer-overflow Parser/pegen.c:993 in bad_single_statement
Shadow bytes around the buggy address:
0x0c047ffff2d0: fa fa 00 04 fa fa 00 01 fa fa 00 04 fa fa 00 04
0x0c047ffff2e0: fa fa 00 04 fa fa 00 04 fa fa 00 04 fa fa 00 04
0x0c047ffff2f0: fa fa 00 04 fa fa 00 04 fa fa 00 04 fa fa 00 07
0x0c047ffff300: fa fa 00 04 fa fa 00 04 fa fa 06 fa fa fa 00 fa
0x0c047ffff310: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
=>0x0c047ffff320: fa[fa]02 fa fa fa fd fa fa fa fd fa fa fa fd fa
0x0c047ffff330: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047ffff340: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047ffff350: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047ffff360: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047ffff370: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==26216==ABORTING

[pablogsal]

The problem seems to be that when we do

const char *cur = strchr(p->tok->buf, '\n');
...

*(cur - 1) == '\\'

we don't check if "cur" is before at p->tok->buf (and therefore we cannot check the previous char)

[pablogsal]

Check the errors in this new builder I have prepared:

https://buildbot.python.org/all/#/builders/582/builds/200/steps/5/logs/stdio

[pablogsal]

New changeset e68c678 by Pablo Galindo in branch 'master':
bpo-42150: Avoid buffer overflow in the new parser (GH-22978)
e68c678

[miss-islington]

New changeset 0b290dd by Miss Skeleton (bot) in branch '3.9':
bpo-42150: Avoid buffer overflow in the new parser (GH-22978)
0b290dd