-
Notifications
You must be signed in to change notification settings - Fork 589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
match on type of immediate #49
Comments
This will help semgrep/semgrep#49 Test plan: make test
This fixes issue #49. Test plan: test file included make test
* Typed metavar (part 2, sgrep part) for literals. This fixes issue #49. Test plan: test file included make test * * lib/typechecking_generic.ml: small diff, just to retrigger CI with latest pfff
@aryx I was unable to get this to work using the patterns in https://github.com/returntocorp/sgrep-rules/blob/develop/python/exceptions/exceptions.yaml#L7-L10; ie:
|
There are some ambiguities in the python grammar if you use $X : int anywhere in an expression, it's because K: V is also a key/value in a dictionary. Right now I've extended the python gramamr to allow X:T to appear only in arguments (the examples you had above). I can extend that to anywhere, but then you will have to use parenthesis around, as in foo(($X:int)). What do you prefer? |
* rename and add black (#62) * rename and add black * blacken * exclude * Update README.md (#63) * add --error to exit 1 if there are findings (#64) * adapt to latest pfff, after AST generic modifications with the Golang integration (#69) test plan: make test also I've disabled the python regression testing because of hardcoded path to isaac dir * support for nested booleans (#67) Updated docs. Re-use patterns key to mean (group of ANDed expressions). Cleaned up some of the traversal logic Add support for nested boolean expression, e.g.: - pattern-either: - pattern: $X == $X - pattern: $X != $X - patterns: - pattern-inside: | def __init__(...): ... - pattern: self.$X == self.$X * * bin/main_sgrep.ml: use better Lang.files_of_dirs... when can. (#71) Should fix issue #66 Test plan: $ ./_build/default/bin/main_sgrep.exe -lang javascript -e foo tests/js/ * Support matching over decorators/attributes/annotations (#72) This solves issue #29 Test plan: file test included make test * Update python.yml (#75) * Adapt to latest pfff, and allow ... to match optional expression in return (#78) This should fix issue #73 and #52 Test plan: tests included $ make test * conform to r2c output with relative paths (#77) * make paths relative to cwd * cleanup rule id prefixes a bit * update expected test, re-enable (#79) * multilingual (#80) * group by languages for sgrep * add a test for multilingual lint * Update README.md (#82) close #81 * Ie/dockerfile paths (#83) * cleanup paths and re-written check IDs when running in docker * black * prefix stripping * revert Dockerfile * safer relative paths * black * update action names (#84) * Change readme to correct code equivalence (#86) * Better class matching (#85) * * lib/generic_vs_generic.ml: comments * * tests/python/dots_fields.py: support for dots in fields. This is related to issue #50 Test plan: test included. * * bin/main_sgrep.ml: add dumper actions to cli, -dump_pattern and -dump_ast * * lib/generic_vs_generic.ml: misc renaming, more consistent use of m_list__xxx * Dots for inheritance, and less-is-ok for inheritance Test plan: tests included make test * Typed metavar (part 2, sgrep part) for literals. (#88) * Typed metavar (part 2, sgrep part) for literals. This fixes issue #49. Test plan: test file included make test * * lib/typechecking_generic.ml: small diff, just to retrigger CI with latest pfff * Function pattern now match methods (and nested functions) (#89) This fixes issue #65 Test plan: tests included. * New --validate-pattern-stdin command for sgrep, to be used in CI (#97) This fixes issue #6 Test plan: echo '$X == $X' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 0 pad@yrax:~/github/sgrep$ echo '$X == ' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 1 * * lib/generic_vs_generic.ml: CEs on recent changes on ast_generic (#98) removal of Nop and additions of a few more expr option test plan: make test still works * tests/go/: adding more tests for sgrep for Go! (#99) * tests/go/: adding more tests for sgrep for Go! This should fix issue #96 Test plan: test files included * * tests/go/: more tests * * tests/go/TODO: a few TODOs * * lib/generic_vs_generic.ml: CEs on CaseEqualExpr * * tests/go/: remove optional ';' Test plan: make test also pad@yrax:~/github/sgrep$ ./_build/default/bin/main_sgrep.exe -lang go -e 'foobar()' -r2c ~/work/lang-go/ now works without any exceptions or noice on stdout * Add meetup invitation link (#100) Co-authored-by: Drew Dennison <dennison@mit.edu> Co-authored-by: Yoann Padioleau <yoann.padioleau@gmail.com> Co-authored-by: Pablo Estrada <pabloest@yahoo.com>
I think the approach you took is good! Thanks for the clarification |
* rename and add black (#62) * rename and add black * blacken * exclude * Update README.md (#63) * add --error to exit 1 if there are findings (#64) * adapt to latest pfff, after AST generic modifications with the Golang integration (#69) test plan: make test also I've disabled the python regression testing because of hardcoded path to isaac dir * support for nested booleans (#67) Updated docs. Re-use patterns key to mean (group of ANDed expressions). Cleaned up some of the traversal logic Add support for nested boolean expression, e.g.: - pattern-either: - pattern: $X == $X - pattern: $X != $X - patterns: - pattern-inside: | def __init__(...): ... - pattern: self.$X == self.$X * * bin/main_sgrep.ml: use better Lang.files_of_dirs... when can. (#71) Should fix issue #66 Test plan: $ ./_build/default/bin/main_sgrep.exe -lang javascript -e foo tests/js/ * Support matching over decorators/attributes/annotations (#72) This solves issue #29 Test plan: file test included make test * Update python.yml (#75) * Adapt to latest pfff, and allow ... to match optional expression in return (#78) This should fix issue #73 and #52 Test plan: tests included $ make test * conform to r2c output with relative paths (#77) * make paths relative to cwd * cleanup rule id prefixes a bit * update expected test, re-enable (#79) * multilingual (#80) * group by languages for sgrep * add a test for multilingual lint * Update README.md (#82) close #81 * Ie/dockerfile paths (#83) * cleanup paths and re-written check IDs when running in docker * black * prefix stripping * revert Dockerfile * safer relative paths * black * update action names (#84) * Change readme to correct code equivalence (#86) * Better class matching (#85) * * lib/generic_vs_generic.ml: comments * * tests/python/dots_fields.py: support for dots in fields. This is related to issue #50 Test plan: test included. * * bin/main_sgrep.ml: add dumper actions to cli, -dump_pattern and -dump_ast * * lib/generic_vs_generic.ml: misc renaming, more consistent use of m_list__xxx * Dots for inheritance, and less-is-ok for inheritance Test plan: tests included make test * Typed metavar (part 2, sgrep part) for literals. (#88) * Typed metavar (part 2, sgrep part) for literals. This fixes issue #49. Test plan: test file included make test * * lib/typechecking_generic.ml: small diff, just to retrigger CI with latest pfff * Function pattern now match methods (and nested functions) (#89) This fixes issue #65 Test plan: tests included. * New --validate-pattern-stdin command for sgrep, to be used in CI (#97) This fixes issue #6 Test plan: echo '$X == $X' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 0 pad@yrax:~/github/sgrep$ echo '$X == ' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 1 * * lib/generic_vs_generic.ml: CEs on recent changes on ast_generic (#98) removal of Nop and additions of a few more expr option test plan: make test still works * tests/go/: adding more tests for sgrep for Go! (#99) * tests/go/: adding more tests for sgrep for Go! This should fix issue #96 Test plan: test files included * * tests/go/: more tests * * tests/go/TODO: a few TODOs * * lib/generic_vs_generic.ml: CEs on CaseEqualExpr * * tests/go/: remove optional ';' Test plan: make test also pad@yrax:~/github/sgrep$ ./_build/default/bin/main_sgrep.exe -lang go -e 'foobar()' -r2c ~/work/lang-go/ now works without any exceptions or noice on stdout * Add meetup invitation link (#100) * Update README.md (#93) * better handling of invalid keys (#94) * better handling of invalid keys * use validate patterns flag in sgrep (#102) * use validate patterns flag in sgrep * Update README.md (#107) * clean up paths, skip hidden dirs, and use r2c as the default config (#103) * clean up paths, skip hidden dirs, and use r2c as the default config * remove debugging try * add generate config arg * only replace if base path is given * blacken * change to kwarg * ignore when folder doesn't exist (#108) ebf * fix bad merge * Update README.md (#112) * Create sgrep-lint.yml (#111) * Create sgrep-lint.yml * Update sgrep-lint.yml * normal output (#113) * actually write and create file given with -o (#114) * use -o instead of | tee (#117) * use -o instead of | tee * remove newline * better runs with -e (#118) * Update sgrep-lint.yml (#119) * Regression tests showing the absence of FakeTok exn. (#121) This shows that issue #104 and #106 currently work with the develop branch. Test plan: file tests included, make test * Newline has a meaning in Go which automatically inserts semicolons with (#122) certain newlines. In sgrep we do not want those extra semicolons hence this diff. Test plan: make test * ask user to report sgrep core errors (#125) * ask user to report sgrep core errors * add pre-commit * trying out pre-commit * * tests/java/misc_assert.java: regression test (#127) This fixes issue #123 * make sgrep a pre-commit hook (#129) * add integrations.md (#130) * Update .pre-commit-hooks.yaml (#131) * Update .pre-commit-hooks.yaml * fmt * remove no-commit-to-branch breaks CI (#132) * Update sgrep-lint.yml (#133) * Update sgrep-lint.yml * add sgrep to our pre-commit hook (#134) * add sgrep to our pre-commit hook * This should fix issue #106 (#137) Test plan: in bento-core/ repo git checkout lomalley/sgrep-test ~/sgrep/.../main_sgrep.exe -verbose -rules_file .sgrep.yml bento/ no exn anymore * Bugfix/finding config (#135) * fix bug where config not found if path contains .. * --exclude-tests option, friendlier error messages * black Co-authored-by: Yoann Padioleau <yoann.padioleau@gmail.com> Co-authored-by: Isaac Evans <409041+ievans@users.noreply.github.com> Co-authored-by: Pablo Estrada <pabloest@yahoo.com>
* rename and add black (#62) * rename and add black * blacken * exclude * Update README.md (#63) * add --error to exit 1 if there are findings (#64) * adapt to latest pfff, after AST generic modifications with the Golang integration (#69) test plan: make test also I've disabled the python regression testing because of hardcoded path to isaac dir * support for nested booleans (#67) Updated docs. Re-use patterns key to mean (group of ANDed expressions). Cleaned up some of the traversal logic Add support for nested boolean expression, e.g.: - pattern-either: - pattern: $X == $X - pattern: $X != $X - patterns: - pattern-inside: | def __init__(...): ... - pattern: self.$X == self.$X * * bin/main_sgrep.ml: use better Lang.files_of_dirs... when can. (#71) Should fix issue #66 Test plan: $ ./_build/default/bin/main_sgrep.exe -lang javascript -e foo tests/js/ * Support matching over decorators/attributes/annotations (#72) This solves issue #29 Test plan: file test included make test * Update python.yml (#75) * Adapt to latest pfff, and allow ... to match optional expression in return (#78) This should fix issue #73 and #52 Test plan: tests included $ make test * conform to r2c output with relative paths (#77) * make paths relative to cwd * cleanup rule id prefixes a bit * update expected test, re-enable (#79) * multilingual (#80) * group by languages for sgrep * add a test for multilingual lint * Update README.md (#82) close #81 * Ie/dockerfile paths (#83) * cleanup paths and re-written check IDs when running in docker * black * prefix stripping * revert Dockerfile * safer relative paths * black * update action names (#84) * Change readme to correct code equivalence (#86) * Better class matching (#85) * * lib/generic_vs_generic.ml: comments * * tests/python/dots_fields.py: support for dots in fields. This is related to issue #50 Test plan: test included. * * bin/main_sgrep.ml: add dumper actions to cli, -dump_pattern and -dump_ast * * lib/generic_vs_generic.ml: misc renaming, more consistent use of m_list__xxx * Dots for inheritance, and less-is-ok for inheritance Test plan: tests included make test * Typed metavar (part 2, sgrep part) for literals. (#88) * Typed metavar (part 2, sgrep part) for literals. This fixes issue #49. Test plan: test file included make test * * lib/typechecking_generic.ml: small diff, just to retrigger CI with latest pfff * Function pattern now match methods (and nested functions) (#89) This fixes issue #65 Test plan: tests included. * New --validate-pattern-stdin command for sgrep, to be used in CI (#97) This fixes issue #6 Test plan: echo '$X == $X' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 0 pad@yrax:~/github/sgrep$ echo '$X == ' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 1 * * lib/generic_vs_generic.ml: CEs on recent changes on ast_generic (#98) removal of Nop and additions of a few more expr option test plan: make test still works * tests/go/: adding more tests for sgrep for Go! (#99) * tests/go/: adding more tests for sgrep for Go! This should fix issue #96 Test plan: test files included * * tests/go/: more tests * * tests/go/TODO: a few TODOs * * lib/generic_vs_generic.ml: CEs on CaseEqualExpr * * tests/go/: remove optional ';' Test plan: make test also pad@yrax:~/github/sgrep$ ./_build/default/bin/main_sgrep.exe -lang go -e 'foobar()' -r2c ~/work/lang-go/ now works without any exceptions or noice on stdout * Add meetup invitation link (#100) * Update README.md (#93) * better handling of invalid keys (#94) * better handling of invalid keys * use validate patterns flag in sgrep (#102) * use validate patterns flag in sgrep * Update README.md (#107) * clean up paths, skip hidden dirs, and use r2c as the default config (#103) * clean up paths, skip hidden dirs, and use r2c as the default config * remove debugging try * add generate config arg * only replace if base path is given * blacken * change to kwarg * ignore when folder doesn't exist (#108) ebf * fix bad merge * Update README.md (#112) * Create sgrep-lint.yml (#111) * Create sgrep-lint.yml * Update sgrep-lint.yml * normal output (#113) * actually write and create file given with -o (#114) * use -o instead of | tee (#117) * use -o instead of | tee * remove newline * better runs with -e (#118) * Update sgrep-lint.yml (#119) * Regression tests showing the absence of FakeTok exn. (#121) This shows that issue #104 and #106 currently work with the develop branch. Test plan: file tests included, make test * Newline has a meaning in Go which automatically inserts semicolons with (#122) certain newlines. In sgrep we do not want those extra semicolons hence this diff. Test plan: make test * ask user to report sgrep core errors (#125) * ask user to report sgrep core errors * add pre-commit * trying out pre-commit * * tests/java/misc_assert.java: regression test (#127) This fixes issue #123 * make sgrep a pre-commit hook (#129) * add integrations.md (#130) * Update .pre-commit-hooks.yaml (#131) * Update .pre-commit-hooks.yaml * fmt * remove no-commit-to-branch breaks CI (#132) * Update sgrep-lint.yml (#133) * Update sgrep-lint.yml * add sgrep to our pre-commit hook (#134) * add sgrep to our pre-commit hook * This should fix issue #106 (#137) Test plan: in bento-core/ repo git checkout lomalley/sgrep-test ~/sgrep/.../main_sgrep.exe -verbose -rules_file .sgrep.yml bento/ no exn anymore * Bugfix/finding config (#135) * fix bug where config not found if path contains .. * --exclude-tests option, friendlier error messages * black * Fix consecutive statements matching (#141) * * lib/sgrep_generic.ml: do not stop at first match, behave like sgrep_lint_generic and try to find all matches. This avoid disrespancies between the behavior of sgrep -e and sgrep -rules_file. * * lib/generic_vs_generic.ml: add debugging information * * tests/python/misc_stmts1.py: support for multiple statements matching! This show that issue #92 is now fixed (require latest of pfff). * * bin/main_sgrep.ml: small diff just to retrigger CI * update multiline syntax to use the yaml | operator (#140) Co-authored-by: Drew Dennison <dennison@mit.edu> Co-authored-by: Yoann Padioleau <yoann.padioleau@gmail.com> Co-authored-by: Pablo Estrada <pabloest@yahoo.com>
* rename and add black (#62) * rename and add black * blacken * exclude * Update README.md (#63) * add --error to exit 1 if there are findings (#64) * adapt to latest pfff, after AST generic modifications with the Golang integration (#69) test plan: make test also I've disabled the python regression testing because of hardcoded path to isaac dir * support for nested booleans (#67) Updated docs. Re-use patterns key to mean (group of ANDed expressions). Cleaned up some of the traversal logic Add support for nested boolean expression, e.g.: - pattern-either: - pattern: $X == $X - pattern: $X != $X - patterns: - pattern-inside: | def __init__(...): ... - pattern: self.$X == self.$X * * bin/main_sgrep.ml: use better Lang.files_of_dirs... when can. (#71) Should fix issue #66 Test plan: $ ./_build/default/bin/main_sgrep.exe -lang javascript -e foo tests/js/ * Support matching over decorators/attributes/annotations (#72) This solves issue #29 Test plan: file test included make test * Update python.yml (#75) * Adapt to latest pfff, and allow ... to match optional expression in return (#78) This should fix issue #73 and #52 Test plan: tests included $ make test * conform to r2c output with relative paths (#77) * make paths relative to cwd * cleanup rule id prefixes a bit * update expected test, re-enable (#79) * multilingual (#80) * group by languages for sgrep * add a test for multilingual lint * Update README.md (#82) close #81 * Ie/dockerfile paths (#83) * cleanup paths and re-written check IDs when running in docker * black * prefix stripping * revert Dockerfile * safer relative paths * black * update action names (#84) * Change readme to correct code equivalence (#86) * Better class matching (#85) * * lib/generic_vs_generic.ml: comments * * tests/python/dots_fields.py: support for dots in fields. This is related to issue #50 Test plan: test included. * * bin/main_sgrep.ml: add dumper actions to cli, -dump_pattern and -dump_ast * * lib/generic_vs_generic.ml: misc renaming, more consistent use of m_list__xxx * Dots for inheritance, and less-is-ok for inheritance Test plan: tests included make test * Typed metavar (part 2, sgrep part) for literals. (#88) * Typed metavar (part 2, sgrep part) for literals. This fixes issue #49. Test plan: test file included make test * * lib/typechecking_generic.ml: small diff, just to retrigger CI with latest pfff * Function pattern now match methods (and nested functions) (#89) This fixes issue #65 Test plan: tests included. * New --validate-pattern-stdin command for sgrep, to be used in CI (#97) This fixes issue #6 Test plan: echo '$X == $X' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 0 pad@yrax:~/github/sgrep$ echo '$X == ' | ./_build/default/bin/main_sgrep.exe -lang python --validate-pattern-stdin pad@yrax:~/github/sgrep$ echo $? 1 * * lib/generic_vs_generic.ml: CEs on recent changes on ast_generic (#98) removal of Nop and additions of a few more expr option test plan: make test still works * tests/go/: adding more tests for sgrep for Go! (#99) * tests/go/: adding more tests for sgrep for Go! This should fix issue #96 Test plan: test files included * * tests/go/: more tests * * tests/go/TODO: a few TODOs * * lib/generic_vs_generic.ml: CEs on CaseEqualExpr * * tests/go/: remove optional ';' Test plan: make test also pad@yrax:~/github/sgrep$ ./_build/default/bin/main_sgrep.exe -lang go -e 'foobar()' -r2c ~/work/lang-go/ now works without any exceptions or noice on stdout * Add meetup invitation link (#100) * Update README.md (#93) * better handling of invalid keys (#94) * better handling of invalid keys * use validate patterns flag in sgrep (#102) * use validate patterns flag in sgrep * Update README.md (#107) * clean up paths, skip hidden dirs, and use r2c as the default config (#103) * clean up paths, skip hidden dirs, and use r2c as the default config * remove debugging try * add generate config arg * only replace if base path is given * blacken * change to kwarg * ignore when folder doesn't exist (#108) ebf * fix bad merge * Update README.md (#112) * Create sgrep-lint.yml (#111) * Create sgrep-lint.yml * Update sgrep-lint.yml * normal output (#113) * actually write and create file given with -o (#114) * use -o instead of | tee (#117) * use -o instead of | tee * remove newline * better runs with -e (#118) * Update sgrep-lint.yml (#119) * Regression tests showing the absence of FakeTok exn. (#121) This shows that issue #104 and #106 currently work with the develop branch. Test plan: file tests included, make test * Newline has a meaning in Go which automatically inserts semicolons with (#122) certain newlines. In sgrep we do not want those extra semicolons hence this diff. Test plan: make test * ask user to report sgrep core errors (#125) * ask user to report sgrep core errors * add pre-commit * trying out pre-commit * * tests/java/misc_assert.java: regression test (#127) This fixes issue #123 * make sgrep a pre-commit hook (#129) * add integrations.md (#130) * Update .pre-commit-hooks.yaml (#131) * Update .pre-commit-hooks.yaml * fmt * remove no-commit-to-branch breaks CI (#132) * Update sgrep-lint.yml (#133) * Update sgrep-lint.yml * add sgrep to our pre-commit hook (#134) * add sgrep to our pre-commit hook * This should fix issue #106 (#137) Test plan: in bento-core/ repo git checkout lomalley/sgrep-test ~/sgrep/.../main_sgrep.exe -verbose -rules_file .sgrep.yml bento/ no exn anymore * Bugfix/finding config (#135) * fix bug where config not found if path contains .. * --exclude-tests option, friendlier error messages * black * Fix consecutive statements matching (#141) * * lib/sgrep_generic.ml: do not stop at first match, behave like sgrep_lint_generic and try to find all matches. This avoid disrespancies between the behavior of sgrep -e and sgrep -rules_file. * * lib/generic_vs_generic.ml: add debugging information * * tests/python/misc_stmts1.py: support for multiple statements matching! This show that issue #92 is now fixed (require latest of pfff). * * bin/main_sgrep.ml: small diff just to retrigger CI * update multiline syntax to use the yaml | operator (#140) * Update sgrep.py * report parse errors and fail on them with --strict (closes #142) (#151) * report parse errors and fail on them with --strict (closes #142) * another YAML parse clarification, closes #143 and adds a test (#152) * another YAML parse clarification, closes #143 and adds a test * Update README.md (#150) * Update README.md Co-Authored-By: Pablo Estrada <pablo@r2c.dev> * * lib/generic_vs_generic.ml: adapt to changes in ast_generic.ml after (#158) php_to_generic.ml introduced a few new constructs in ast_generic.ml * Normalize some formatting / capitalization in docs/patterns.md * add a flag to skip pattern validation, which takes 0.05 seconds per rule (#156) * add a flag to skip pattern validation, which takes 0.05 seconds per rule * clean up builds and try to make work for forks * remove on push * use consistant on block * fix #155 (#167) * update to latest pfff * * lib/match_result.ml: better failure message when NoTokenLocation * Test files for issue #153 (#168) * test file for issue #157 (#169) * Add many new tests for ... in more places for Go, Js, and Java. (#170) This show issue #160 is fixed. * More prominent link to sgrep-rules repo (#165) Co-authored-by: Yoann Padioleau <yoann.padioleau@gmail.com> Co-authored-by: Isaac Evans <409041+ievans@users.noreply.github.com> Co-authored-by: Pablo Estrada <pabloest@yahoo.com>
should match
os.remove($X: float)
->os.remove(1.0)
os.remove($X: str)
->os.remove("1.0")
Etc., for the python primitive types seen as immediates
The text was updated successfully, but these errors were encountered: