Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overflow-1.re test fails on system with small stack #219

Closed
trofi opened this issue Sep 5, 2018 · 5 comments
Closed

overflow-1.re test fails on system with small stack #219

trofi opened this issue Sep 5, 2018 · 5 comments

Comments

@trofi
Copy link
Collaborator

trofi commented Sep 5, 2018

Factoring out the issue of #215 (comment)

On small stack systems (ulimit -s 256) overflow-1.re test fails:

$ ./autogen.sh
$ ulimit -s 256
$ make
$ LANG=C make check VERBOSE=1
...
./run_tests.sh: line 111: 17604 Segmentation fault      (core dumped) $valgrind $wine ../../$re2c $switches "$outx" 2> "$outc.stderr" 1>&2
FAIL       overflow-1.re
...
Error: 1 out 1409 tests failed.
@trofi
Copy link
Collaborator Author

trofi commented Sep 5, 2018

$ gdb --args ../re2c -o overflow-1.c --no-version --no-generation-date -W overflow-1.re
...
Program received signal SIGSEGV, Segmentation fault.
0x00005555555893b2 in re2c::find_fixed_tags (re=0x555555690e50, tags=std::vector of length 0, capacity 0, dist=@0x7fffffffbf20: 3119, base=@0x7fffffffbf18: 18446744073709551615, 
    toplevel=toplevel@entry=true) at src/re/fixed_tags.cc:33
33      {
(gdb) bt
#0  0x00005555555893b2 in re2c::find_fixed_tags (re=0x555555690e50, tags=std::vector of length 0, capacity 0, dist=@0x7fffffffbf20: 3119, base=@0x7fffffffbf18: 18446744073709551615, 
    toplevel=toplevel@entry=true) at src/re/fixed_tags.cc:33
#1  0x0000555555589543 in re2c::find_fixed_tags (re=0x555555690e70, tags=std::vector of length 0, capacity 0, dist=@0x7fffffffbf20: 3119, base=@0x7fffffffbf18: 18446744073709551615, 
    toplevel=toplevel@entry=true) at src/re/fixed_tags.cc:47
#2  0x0000555555589558 in re2c::find_fixed_tags (re=0x555555690eb0, tags=std::vector of length 0, capacity 0, dist=@0x7fffffffbf20: 3119, base=@0x7fffffffbf18: 18446744073709551615, 
    toplevel=toplevel@entry=true) at src/re/fixed_tags.cc:48
...
#3118 0x0000555555589558 in re2c::find_fixed_tags (re=0x5555556ff0c0, tags=std::vector of length 0, capacity 0, dist=@0x7fffffffbf20: 3119, base=@0x7fffffffbf18: 18446744073709551615, 
    toplevel=toplevel@entry=true) at src/re/fixed_tags.cc:48
#3119 0x0000555555589558 in re2c::find_fixed_tags (re=0x5555556ff100, tags=std::vector of length 0, capacity 0, dist=@0x7fffffffbf20: 3119, base=@0x7fffffffbf18: 18446744073709551615, 
    toplevel=toplevel@entry=true) at src/re/fixed_tags.cc:48

#3120 0x0000555555589558 in re2c::find_fixed_tags (re=0x5555556ff140, tags=std::vector of length 0, capacity 0, dist=@0x7fffffffbf20: 3119, base=@0x7fffffffbf18: 18446744073709551615, 
    toplevel=toplevel@entry=true) at src/re/fixed_tags.cc:48
#3121 0x000055555558961e in re2c::find_fixed_tags (spec=...) at src/re/fixed_tags.cc:80
#3122 0x000055555558af2f in re2c::ast_to_dfa (output=..., spec=...) at src/compile.cc:62
#3123 re2c::compile(re2c::Scanner&, re2c::Output&, re2c::Opt&) () at src/compile.cc:172
#3124 0x000055555555aa4c in main () at src/main.cc:31
#3125 0x00007ffff74a579b in __libc_start_main (main=0x55555555a8f0 <main>, argc=7, argv=0x7fffffffd678, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffd668) at ../csu/libc-start.c:308
#3126 0x000055555555acba in _start () at /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/include/g++-v8/limits:1002

skvadrik added a commit that referenced this issue Apr 21, 2020
…tates).

This is to avoid stack overflow on large RE (especially on instrumented
builds that have larger stack frames, like AddressSanitizer).

Stack overflow reported by Agostino Sarubbo.
Related to #219 "overflow-1.re test fails on system with small stack".
skvadrik added a commit that referenced this issue Apr 22, 2020
This is to avoid stack overflow on large RE (especially on instrumented
builds that have larger stack frames, like AddressSanitizer).

Partial fix for #219 "overflow-1.re test fails on system with small stack".
skvadrik added a commit that referenced this issue Apr 23, 2020
This is to avoid stack overflow on large RE (especially on instrumented
builds that have larger stack frames, like AddressSanitizer).

Partial fix for #219 "overflow-1.re test fails on system with small stack".
skvadrik added a commit that referenced this issue Apr 23, 2020
This is to avoid stack overflow on large RE (especially on instrumented
builds that have larger stack frames, like AddressSanitizer).

Partial fix for #219 "overflow-1.re test fails on system with small stack".
@skvadrik
Copy link
Owner

A quick way to reproduce without polluting the shell:

bash -c "ulimit -s 256; ./re2c overflow-1.re >/dev/null"

Run the same in GDB to see the faulty recursive function:

bash -c "ulimit -s 256; echo run | gdb --args ./re2c overflow-1.re"

@kirotawa
Copy link

kirotawa commented May 4, 2020

This issue was assigned to CVE-2018-21232

omasanori added a commit to omasanori/nixpkgs that referenced this issue Feb 26, 2021
It backports patches from 2.0 series to fix CVE-2018-21232.

References:
NixOS#88391
https://www.openwall.com/lists/oss-security/2020/04/27/2
skvadrik/re2c#219

Signed-off-by: Masanori Ogino <167209+omasanori@users.noreply.github.com>
omasanori added a commit to omasanori/nixpkgs that referenced this issue Mar 6, 2021
It backports patches from 2.0 series to fix CVE-2018-21232.

References:
NixOS#88391
https://www.openwall.com/lists/oss-security/2020/04/27/2
skvadrik/re2c#219

Signed-off-by: Masanori Ogino <167209+omasanori@users.noreply.github.com>
(cherry picked from commit 5854bf0)
skvadrik referenced this issue Mar 12, 2021
The crash happened in a rare case of a very long lexeme that doen't fit
into the buffer, forcing buffer reallocation.

The crash was caused by an incorrect calculation of the shift offset
(it was smaller than necessary). As a consequence, the data from buffer
start and up to the beginning of the current lexeme was not discarded
(as it should have been), resulting in less free space for new data than
expected.
@yifan19
Copy link

yifan19 commented Apr 20, 2021

@trofi I was testing against 2.0.3 and test still failed.

also fails on 2.1.1

running oe-core(master) on qemux86-64 where i build from source inside of the emulator.

root@qemux86-64:~/re2c-2.1.1# bash -c "ulimit -s 256; echo run | gdb --args ./re2c ./test/overflow-1.re"
GNU gdb (GDB) 10.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-oe-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./re2c...
(gdb) Starting program: /home/root/re2c-2.1.1/re2c ./test/overflow-1.re

Program received signal SIGSEGV, Segmentation fault.
0x0000000000425d4b in re2c::stats (n=0x55c620, topord=@0x7fffffffcde0: 0, ncores=@0x7fffffffcfe8: 7919)
    at src/nfa/re_to_nfa.cc:170
170                 stats(n->ran.out, topord, ncores);
(gdb) quit
A debugging session is active.

        Inferior 1 [process 7509] will be killed.

Quit anyway? (y or n) [answered Y; input not from terminal]
root@qemux86-64:~/re2c-2.1.1# bash -c "ulimit -s 256; echo run | gdb --args ./re2c ./test/ove

@skvadrik
Copy link
Owner

@yifan19 Correct, this hasn't been fixed yet. I eliminated some of the recursive functions, but a few still remain (such as re2c::stats which you see in GDB).

halstead pushed a commit to openembedded/openembedded-core that referenced this issue Mar 23, 2022
Backport commits from the following issue:
skvadrik/re2c#219

CVE: CVE-2018-21232

Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
splitice pushed a commit to HalleyAssist/poky that referenced this issue Mar 23, 2022
Backport commits from the following issue:
skvadrik/re2c#219

CVE: CVE-2018-21232

(From OE-Core rev: 8c5ee47d446b36d6832acc8452687f50101f3e65)

Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
jpuhlman pushed a commit to MontaVista-OpenSourceTechnology/poky that referenced this issue Jun 3, 2022
Source: poky
MR: 118243
Type: Integration
Disposition: Merged from poky
ChangeID: 9d155cbf956024e6ade0f10486ed8fe427652ad0
Description:

Backport commits from the following issue:
skvadrik/re2c#219

CVE: CVE-2018-21232

(From OE-Core rev: 8c5ee47d446b36d6832acc8452687f50101f3e65)

Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com>
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
skvadrik added a commit that referenced this issue Jul 24, 2022
This fixes bug #219. Tested as follows (all tests passed):

    bash -c "ulimit -s 256; time ./run_tests.py"

Original test that revealed the problematic recursive functions:

    bash -c "ulimit -s 256; echo run | gdb --args ./re2c overflow-1.re"
skvadrik added a commit that referenced this issue Jul 25, 2022
This fixes bug #219. Tested as follows (all tests passed):

    bash -c "ulimit -s 256; time ./run_tests.py"

Original test that revealed the problematic recursive functions:

    bash -c "ulimit -s 256; echo run | gdb --args ./re2c overflow-1.re"
skvadrik added a commit that referenced this issue Jul 25, 2022
This fixes bug #219. Tested as follows (all tests passed):

    bash -c "ulimit -s 256; time ./run_tests.py"

Original test that revealed the problematic recursive functions:

    bash -c "ulimit -s 256; echo run | gdb --args ./re2c overflow-1.re"
skvadrik added a commit that referenced this issue Jul 25, 2022
Run tests as:
    bash -c "ulimit -s 256; make check -j$(nproc)"

I also verified that the tests fail on CI without commit
4604129 (the last change that was
needed to fix #219).
skvadrik added a commit that referenced this issue Jul 25, 2022
This fixes bug #219. Tested as follows (all tests passed):

    bash -c "ulimit -s 256; time ./run_tests.py"

Original test that revealed the problematic recursive functions:

    bash -c "ulimit -s 256; echo run | gdb --args ./re2c overflow-1.re"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants