Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lexers for syzkaller DSLs #1699

Merged
merged 2 commits into from Feb 3, 2022
Merged

Add lexers for syzkaller DSLs #1699

merged 2 commits into from Feb 3, 2022

Conversation

xairy
Copy link
Contributor

@xairy xairy commented Apr 6, 2021

This changes adds lexers for two domain-specific languages used by syzkaller, a production-grade kernel fuzzer: syzlang, which allows describing system call interfaces, and syzprog, which allows describing sequences of system calls.

Both lexers are less restrictive than syzkaller's parsers to allow more whitespaces and comments. This is done deliberately to allow adding line-by-line annotations to snippets.

CC @dvyukov

syzlang
syzprog

@xairy
Copy link
Contributor Author

xairy commented Apr 7, 2021

As CI is currently broken, here's the result of running rake manually:

============
Run `rackup` and visit localhost:9292/:lexer_name to visually test a lexer.
============

Run options: --seed 5317

# Running:

................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 2.546288s, 427.2887 runs/s, 921.7340 assertions/s.

1088 runs, 2347 assertions, 0 failures, 0 errors, 0 skips
Running RuboCop...
Inspecting 485 files
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

485 files inspected, no offenses detected

@dblessing PTAL, when you get a chance. Thanks!

@xairy
Copy link
Contributor Author

xairy commented Apr 28, 2021

@dblessing, a reminder ping as you have asked.

I'm also getting a word from Travis that Rouge has been allotted the requested OSS credits.

@xairy
Copy link
Contributor Author

xairy commented Dec 12, 2021

@tancnle @jneen Could you please take a look at this PR? Thanks!

@xairy
Copy link
Contributor Author

xairy commented Feb 1, 2022

Hi @tancnle,

How could I help with getting this PR merged?

Being one of syzkaller's developers, I have a lot of experience working with the mentioned languages, so I'm confident in the semantical correctness of these lexers. I've also included extensive tests for both lexers, which I manually written and checked.

It would be awesome to have these lexers merged into Rouge, and thus allow highlighting syzkaller-related snippets on GitHub Pages.

Thank you!

@tancnle tancnle added the needs-review The PR needs to be reviewed label Feb 1, 2022
@tancnle
Copy link
Collaborator

tancnle commented Feb 1, 2022

@xairy Thank you for your hard work on this PR. I will spend some time reviewing it later today 👍🏼 .

Copy link
Collaborator

@tancnle tancnle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution @xairy 👍🏼 Really appreciated the level of care you have put into this PR. Great write up on the issue and implementation (https://xairy.io/articles/2021/syzkaller-syntax-highlight) ❤️

I have left some suggestions for your consideration, nothing major, mostly around indentation and replacing tests with visual samples. Please let me know what you think 🙏🏼

lib/rouge/lexers/syzlang.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/syzlang.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/syzprog.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/syzprog.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/syzlang.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/syzprog.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/syzprog.rb Outdated Show resolved Hide resolved
lib/rouge/lexers/syzprog.rb Outdated Show resolved Hide resolved
spec/lexers/syzprog_spec.rb Show resolved Hide resolved
spec/lexers/syzlang_spec.rb Show resolved Hide resolved
@xairy
Copy link
Contributor Author

xairy commented Feb 3, 2022

Sorry for the indentation issues, all should be fixed now. Thanks a lot for looking at the PR!

This change adds a lexer for syzlang, a DSL used by syzkaller [1] to describe
system call interfaces.

The lexer intentionally recognizes code with less strict whitespacing rules
than what syzkaller demands. This is useful for line-splitting and annotating
syzlang snippets.

[1] https://github.com/google/syzkaller

Signed-off-by: Andrey Konovalov <andreyknvl@gmail.com>
This change adds a lexer for syzprog, a DSL used by syzkaller [1] to describe
sequences of system calls.

The lexer intentionally recognizes code with less strict whitespacing rules
than what syzkaller demands. This is useful for line-splitting and annotating
syzprog snippets.

[1] https://github.com/google/syzkaller

Signed-off-by: Andrey Konovalov <andreyknvl@gmail.com>
@xairy
Copy link
Contributor Author

xairy commented Feb 3, 2022

Oh, and thank you for checking out the write-up! =)

@tancnle
Copy link
Collaborator

tancnle commented Feb 3, 2022

Great work on this @xairy 👍🏼 Merging 🚀

@tancnle tancnle merged commit 9b5532d into rouge-ruby:master Feb 3, 2022
@xairy
Copy link
Contributor Author

xairy commented Feb 3, 2022

Awesome! Thank you, @tancnle! Looking forward to the next release! =)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-review The PR needs to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants