Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster scanner #4576

Merged
merged 4 commits into from
Aug 21, 2023
Merged

Faster scanner #4576

merged 4 commits into from
Aug 21, 2023

Conversation

tenderlove
Copy link
Contributor

This commit speeds up the scanner by using instance variables for storing line / column information (rather than a hash).

IVs have faster read / write speeds than hash keys

I couldn't get the c ext to build, so I commented it out from the benchmarks.

On master:

$ bundle exec ruby -S rake bench:scan
Warming up --------------------------------------
scan - introspection   315.000  i/100ms
    scan - fragments   463.000  i/100ms
    scan - big query    52.000  i/100ms
Calculating -------------------------------------
scan - introspection      3.153k (± 0.7%) i/s -     16.065k in   5.096124s
    scan - fragments      4.638k (± 0.7%) i/s -     23.613k in   5.091799s
    scan - big query    524.253  (± 0.6%) i/s -      2.652k in   5.058781s

This branch:

$ bundle exec ruby -S rake bench:scan
Warming up --------------------------------------
scan - introspection   358.000  i/100ms
    scan - fragments   523.000  i/100ms
    scan - big query    60.000  i/100ms
Calculating -------------------------------------
scan - introspection      3.587k (± 0.4%) i/s -     18.258k in   5.090098s
    scan - fragments      5.305k (± 0.5%) i/s -     26.673k in   5.028207s
    scan - big query    600.081  (± 0.3%) i/s -      3.060k in   5.099354s

Seems about 14% faster on big queries.

Again with YJIT:

master:

$ bundle exec ruby --jit -S rake bench:scan
Warming up --------------------------------------            
scan - introspection   486.000  i/100ms                      
    scan - fragments   714.000  i/100ms                      
    scan - big query    80.000  i/100ms                      
Calculating -------------------------------------            
scan - introspection      4.855k (± 0.3%) i/s -     24.300k in   5.004814s
    scan - fragments      7.124k (± 0.4%) i/s -     35.700k in   5.011340s
    scan - big query    805.246  (± 0.5%) i/s -      4.080k in   5.066896s

This branch:

$ bundle exec ruby --jit -S rake bench:scan
Warming up --------------------------------------
scan - introspection   557.000  i/100ms
    scan - fragments   825.000  i/100ms
    scan - big query    94.000  i/100ms
Calculating -------------------------------------
scan - introspection      5.599k (± 0.3%) i/s -     28.407k in   5.074005s
    scan - fragments      8.313k (± 0.3%) i/s -     42.075k in   5.061280s
    scan - big query    943.246  (± 0.3%) i/s -      4.794k in   5.082520s

This commit speeds up the scanner by using instance variables for
storing line / column information (rather than a hash).

IVs have faster read / write speeds than hash keys
@rmosolgo rmosolgo added this to the 2.0.25 milestone Aug 4, 2023
@rmosolgo rmosolgo modified the milestones: 2.0.25, 2.0.27 Aug 21, 2023
@rmosolgo
Copy link
Owner

Thanks for this improvement! I moved replace_escaped_characters_in_place back to being a class method because the C parser actually calls out to it 🙈 (I didn't dare try to re-implement it without Ruby....).

@rmosolgo rmosolgo merged commit 0cc82b9 into rmosolgo:master Aug 21, 2023
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants