ASan heap-buffer-overflow in re2c::Scanner::scan

[fgeek]

Tested commit: 22b73ed
Credit: Henri Salo from Nixu Corporation
Tools: american fuzzy lop 2.52b, afl-utils

Reproducer re2c-2018-10-28-crash-001.txt.zip (SHA1 bb5bd8951a063f76d8dbc3525c6e127e9094407b)

00000000  25 7b 7b 22                                       |%{{"|
00000004

ASan output:

==21929==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62500000e900 at pc 0x5624ce069fae bp 0x7ffc7c718110 sp 0x7ffc7c718108
READ of size 1 at 0x62500000e900 thread T0
    #0 0x5624ce069fad in re2c::Scanner::scan(re2c::conopt_t const*) src/ast/lex.cc:1818
    #1 0x5624ce07169e in yylex src/ast/parser.ypp:260
    #2 0x5624ce07169e in yyparse(re2c::context_t&) src/ast/parser.cc:1219
    #3 0x5624ce076eed in re2c::parse(re2c::Scanner&, std::vector<re2c::spec_t, std::allocator<re2c::spec_t> >&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, re2c::AST const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, re2c::AST const*> > >&, re2c::Opt&) src/ast/parser.ypp:271
    #4 0x5624cdf9b642 in re2c::compile(re2c::Scanner&, re2c::Output&, re2c::Opt&) src/compile.cc:155
    #5 0x5624cdd96795 in main src/main.cc:31
    #6 0x7f085714c2e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
    #7 0x5624cdd97899 in _start (/home/hsalo/builds/re2c/22b73eddedd41b2ebb10d17a3e669121dd6418d2/bin/re2c+0x1e899)

0x62500000e900 is located 0 bytes to the right of 8192-byte region [0x62500000c900,0x62500000e900)
allocated by thread T0 here:
    #0 0x7f0857e2ad70 in operator new[](unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc2d70)
    #1 0x5624ce036ad3 in re2c::Scanner::fill(unsigned int) src/ast/scanner.cc:63
    #2 0x5624ce050053 in re2c::Scanner::echo(re2c::OutputFile&) src/ast/lex.cc:94
    #3 0x5624cdf9b50a in re2c::compile(re2c::Scanner&, re2c::Output&, re2c::Opt&) src/compile.cc:140
    #4 0x5624cdd96795 in main src/main.cc:31
    #5 0x7f085714c2e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)

SUMMARY: AddressSanitizer: heap-buffer-overflow src/ast/lex.cc:1818 in re2c::Scanner::scan(re2c::conopt_t const*)
Shadow bytes around the buggy address:
  0x0c4a7fff9cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fff9ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fff9cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fff9d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c4a7fff9d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c4a7fff9d20:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fff9d30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fff9d40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fff9d50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fff9d60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c4a7fff9d70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==21929==ABORTING

(gdb) run
Starting program: ./bin/re2c re2c-2018-10-28-crash-001.txt

Program received signal SIGSEGV, Segmentation fault.
re2c::Scanner::scan (this=0x7fffffffd0b0, globopts=<optimized out>) at src/ast/lex.cc:1818
1818            yych = (YYCTYPE)*YYCURSOR;
(gdb) bt
#0  re2c::Scanner::scan (this=0x7fffffffd0b0, globopts=<optimized out>) at src/ast/lex.cc:1818
#1  0x00005555555a201d in yylex (context=...) at ./src/ast/parser.ypp:260
#2  yyparse (context=...) at src/ast/parser.cc:1219
#3  0x00005555555a301f in re2c::parse (input=..., specs=std::vector of length 0, capacity 0, symtab=std::map with 0 elements, opts=...) at ./src/ast/parser.ypp:271
#4  0x0000555555588f98 in re2c::compile (input=..., output=..., opts=...) at src/compile.cc:155
#5  0x0000555555557077 in main (argv=<optimized out>) at src/main.cc:31

[skvadrik]

Confirmed, the error is caused by assuming that strings in the source code cannot contain a sequence of zero bytes (they can in C/C++). RE2C has to "parse" C/C++ code in order to find the closing curly brace in semantic actions.

There is a number of possible fixes and I'm trying to find the best one.Thanks for reporting!

[fgeek]

@skvadrik Thank you for quick reply. I have some other cases from fuzzing. Do you prefer issue report per unique case or zip file with all samples e.g. via email? I can also give you short introduction how to use afl, but I'm more than happy to continue fuzzing after fixes.

[skvadrik]

Pushed a fix: 44737d3

@fgeek I'll be greedy, let's have the whole zip at once. :) I think you can attach it right here, but email is also fine (re2c-devel@lists.sourceforge.net is the official mailing list). I shall learn to use afl myself (probably not that hard), but let's fix what you've found first.

[fgeek]

I can confirm that commit fixes original issue. Two more cases re2c-2018-10-29-crashes.zip

SUMMARY: AddressSanitizer: heap-buffer-overflow src/ast/lex.cc:2084 in re2c::Scanner::lex_cls(bool)
SUMMARY: AddressSanitizer: heap-buffer-overflow src/ast/lex.cc:2426 in re2c::Scanner::lex_str_chr(char, bool&)

I'll bet using afl is very easy for you. See documentation in http://lcamtuf.coredump.cx/afl/README.txt and feel free to contact me if you need help via henri@nerv.fi email. Remember to build with ASan and use /dev/shm or other memory partition for your fuzzing to avoid wasting HDD/SSD.

[skvadrik]

Pushed a fix for the other two crashes: f062a8b. Thanks for afl links!

[fgeek]

re2c-2018-10-31.zip

SUMMARY: AddressSanitizer: heap-buffer-overflow src/ast/lex.cc:2113 in re2c::Scanner::lex_cls(bool)
SUMMARY: AddressSanitizer: heap-buffer-overflow src/ast/lex.cc:2464 in re2c::Scanner::lex_str_chr(char, bool&)

[skvadrik]

Fixed! 0f7b449

[fgeek]

Thank you. I'll open new issue report if I find something else. Currently running 731 executions per second with nine CPU cores.