Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(lexer): Use SIMD to accelerate whitespace skipping #13

Closed
Boshen opened this issue Feb 12, 2023 · 1 comment
Closed

perf(lexer): Use SIMD to accelerate whitespace skipping #13

Boshen opened this issue Feb 12, 2023 · 1 comment
Labels
good first issue Experience Level - Good for newcomers

Comments

@Boshen
Copy link
Member

Boshen commented Feb 12, 2023

Sometime ago I drafted a version to use SIMD for skipping whitespaces:

image

In this good first issue, you should be able to replicate the code from above, and add it to the lexer.

You should refactor the code above into a function and call the function just above the while loop:

https://github.com/Boshen/oxc/blob/93993978de969589f1fb91a689594a5e5c4b8f02/crates/oxc_parser/src/lexer/mod.rs#L320-L330

You will also need to add #![feature(portable_simd)] to the top of crates/oxc_parser/src/lib.rs

To the run parser, you can run the example binary:

https://github.com/Boshen/oxc/blob/3daf5bcea95f6fe2c88a7956948f4cb22711f99e/crates/oxc_parser/examples/simple.rs#L7-L10

Background reading:

@Boshen Boshen added the good first issue Experience Level - Good for newcomers label Feb 12, 2023
@Boshen Boshen changed the title Perf(lexer): Use SIMD to accelerate whitespace skipping perf(lexer): Use SIMD to accelerate whitespace skipping Feb 12, 2023
@Boshen Boshen added this to the Lexer milestone Feb 12, 2023
@ematipico
Copy link

Adding also a beginner introduction, taken from the rust repository: https://github.com/rust-lang/portable-simd/blob/master/beginners-guide.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Experience Level - Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants