Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spacegrep stack overflow on file with 20K lines #1944

Closed
mjambon opened this issue Nov 1, 2020 · 1 comment · Fixed by #1986
Closed

spacegrep stack overflow on file with 20K lines #1944

mjambon opened this issue Nov 1, 2020 · 1 comment · Fixed by #1986
Assignees
@mjambon
Labels
alpha Relates to an experimental feature bug Something isn't working lang:generic generic mode issues (spacegrep, aliengrep)

Comments

@mjambon
Copy link
Member

mjambon commented Nov 1, 2020

Running spacegrep x -d . on the semgrep repo fails, with a stack overflow and possibly a segfault (depending on command line). The following input file makes it crash:

$ wc -l ./semgrep-core/_build/default/tests/OTHER/PERF/my_first_calculator.py
20822 ./semgrep-core/_build/default/tests/OTHER/PERF/my_first_calculator.py

$ ./spacegrep/bin/spacegrep 'x' -d ./semgrep-core/_build/default/tests/OTHER/PERF/my_first_calculator.py
read document: ./semgrep-core/_build/default/tests/OTHER/PERF/my_first_calculator.py
Fatal error: exception Stack overflow
@mjambon mjambon added bug Something isn't working lang:generic generic mode issues (spacegrep, aliengrep) labels Nov 1, 2020
@mjambon mjambon self-assigned this Nov 1, 2020
@mjambon
Copy link
Member Author

mjambon commented Nov 1, 2020

Increasing the stack size on linux with ulimit -s 64000 is a temporary workaround which allows me to scan all the files in the semgrep repo (including build products).

@nbrahms nbrahms added the alpha Relates to an experimental feature label Nov 2, 2020
mjambon pushed a commit that referenced this issue Nov 7, 2020
Fix one source of stack overflow in spacegrep parsing on large input.
b7713a6 
Allows a semgrep scan on all the files of the semgrep repo without
crashing, on Ubuntu/Linux with default 'ulimit -s' setting (8 MB).
Fixes #1944
@mjambon mjambon mentioned this issue Nov 7, 2020
Merged
mjambon added a commit that referenced this issue Nov 7, 2020
@mjambon
Fix stack overflows (#1986)
07ec23a 
* Fix one source of stack overflow in spacegrep parsing on large input.
Allows a semgrep scan on all the files of the semgrep repo without
crashing, on Ubuntu/Linux with default 'ulimit -s' setting (8 MB).
Fixes #1944

* Eliminate another source of stack overflow.

* Eliminate two more possible stack overflows, in the lexer.

* Add big test input to spacegrep examples and run it as part of the tests.

* Update changelog

Co-authored-by: Martin Jambon <Martin Jambon>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mjambon mjambon

Labels
alpha Relates to an experimental feature bug Something isn't working lang:generic generic mode issues (spacegrep, aliengrep)
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix stack overflows semgrep/semgrep
2 participants
@mjambon @nbrahms