Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codegen select fuzzer v1 #30012

Merged
merged 9 commits into from
Oct 16, 2021

Conversation

PaulCher
Copy link
Contributor

@PaulCher PaulCher commented Oct 12, 2021

Changelog category (leave one):

  • Testing Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Implemented structure-aware fuzzing approach in ClickHouse for select statement parser.

Detailed description / Documentation draft:
Structure-aware fuzzing has been previously been used in many projects, where it has demonstrated its effectiveness. This approach was observed in V8 fuzzing by Google and implemented for ClickHouse parser fuzzing. The idea behind codegen_fuzzer is very simple: with the help of libfuzzer and libprotobuf-mutator we can fuzz language parsers with coverage. Technically, it is a grammar based fuzzer, where grammar is simply TOKEN*, but it is done with feedback, which would be very useful for finding interesting paths without much manual work.

If this approach would show any results, I would like to add similar fuzzers for other parts of the project.

gen.py -- script to generate C++ string generation routines and .proto messages
clickhouse-template.g -- human-generated list of tokens
clickhouse.g -- full list of tokens generated by call to script update.sh.

By adding documentation, you'll allow users to try your new feature immediately, not when someone else will have time to document it later. Documentation is necessary for all features that affect user experience in any way. You can add brief documentation draft above, or add documentation right into your patch as Markdown files in docs folder.

If you are doing this for the first time, it's recommended to read the lightweight Contributing to ClickHouse Documentation guide first.

Information about CI checks: https://clickhouse.tech/docs/en/development/continuous-integration/

@CLAassistant
Copy link

CLAassistant commented Oct 12, 2021

CLA assistant check
All committers have signed the CLA.

@robot-ch-test-poll3 robot-ch-test-poll3 added the submodule changed At least one submodule changed in this PR. label Oct 12, 2021
@nikitamikhaylov nikitamikhaylov self-assigned this Oct 12, 2021
@nikitamikhaylov nikitamikhaylov merged commit 6b22fdc into ClickHouse:master Oct 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
submodule changed At least one submodule changed in this PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants