Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve sanitize_sql_like performance #44769

Conversation

jonathanhefner
Copy link
Member

This improves performance and reduces memory allocations for sanitize_sql_like across multiple use cases.

Benchmark script
require "benchmark/memory"
require "benchmark/ips"

def old_sanitize_sql_like(string, escape_character = "\\")
  pattern = Regexp.union(escape_character, "%", "_")
  string.gsub(pattern) { |x| [escape_character, x].join }
end

def new_sanitize_sql_like(string, escape_character = "\\")
  if string.include?(escape_character) && escape_character != "%" && escape_character != "_"
    string = string.gsub(escape_character, '\0\0')
  end

  string.gsub(/(?=[%_])/, escape_character)
end

[
  "no-special-characters",
  "one_wildcard",
  "one\\escape",
  "two_wildcards%and\\two\\escapes",
].each do |string|
  puts "\n= #{string.inspect} ".ljust(72, "=")

  Benchmark.memory do |x|
    # Warmup:
    old_sanitize_sql_like(string)
    new_sanitize_sql_like(string)

    x.report("old") { old_sanitize_sql_like(string) }
    x.report("new") { new_sanitize_sql_like(string) }
  end

  puts

  Benchmark.ips do |x|
    x.report("old") { old_sanitize_sql_like(string) }
    x.report("new") { new_sanitize_sql_like(string) }
    x.compare!
  end
end

Benchmark results:

= "no-special-characters" =============================================
Calculating -------------------------------------
                 old   875.000  memsize (     0.000  retained)
                        10.000  objects (     0.000  retained)
                         5.000  strings (     0.000  retained)
                 new    40.000  memsize (     0.000  retained)
                         1.000  objects (     0.000  retained)
                         1.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     9.483k i/100ms
                 new   119.791k i/100ms
Calculating -------------------------------------
                 old     95.694k (± 0.3%) i/s -    483.633k in   5.053985s
                 new      1.228M (± 0.5%) i/s -      6.229M in   5.072317s

Comparison:
                 new:  1228094.1 i/s
                 old:    95694.3 i/s - 12.83x  (± 0.00) slower

= "one_wildcard" ======================================================
Calculating -------------------------------------
                 old     1.395k memsize (     0.000  retained)
                        15.000  objects (     0.000  retained)
                         6.000  strings (     0.000  retained)
                 new   440.000  memsize (     0.000  retained)
                         3.000  objects (     0.000  retained)
                         1.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     6.527k i/100ms
                 new    30.205k i/100ms
Calculating -------------------------------------
                 old     65.255k (± 0.6%) i/s -    326.350k in   5.001318s
                 new    300.058k (± 0.5%) i/s -      1.510M in   5.033333s

Comparison:
                 new:   300057.6 i/s
                 old:    65255.2 i/s - 4.60x  (± 0.00) slower

= "one\\escape" =======================================================
Calculating -------------------------------------
                 old     1.395k memsize (     0.000  retained)
                        15.000  objects (     0.000  retained)
                         6.000  strings (     0.000  retained)
                 new   560.000  memsize (     0.000  retained)
                         6.000  objects (     0.000  retained)
                         2.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     6.506k i/100ms
                 new    24.312k i/100ms
Calculating -------------------------------------
                 old     65.467k (± 0.4%) i/s -    331.806k in   5.068392s
                 new    245.704k (± 0.5%) i/s -      1.240M in   5.046485s

Comparison:
                 new:   245703.8 i/s
                 old:    65466.9 i/s - 3.75x  (± 0.00) slower

= "two_wildcards%and\\two\\escapes" ===================================
Calculating -------------------------------------
                 old     1.755k memsize (     0.000  retained)
                        24.000  objects (     0.000  retained)
                         8.000  strings (     0.000  retained)
                 new   832.000  memsize (     0.000  retained)
                         8.000  objects (     0.000  retained)
                         3.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     5.265k i/100ms
                 new    12.360k i/100ms
Calculating -------------------------------------
                 old     52.698k (± 0.3%) i/s -    268.515k in   5.095425s
                 new    124.006k (± 0.3%) i/s -    630.360k in   5.083351s

Comparison:
                 new:   124006.2 i/s
                 old:    52697.9 i/s - 2.35x  (± 0.00) slower

This improves performance and reduces memory allocations for
`sanitize_sql_like` across multiple use cases.

Benchmark script:

```ruby
require "benchmark/memory"
require "benchmark/ips"

def old_sanitize_sql_like(string, escape_character = "\\")
  pattern = Regexp.union(escape_character, "%", "_")
  string.gsub(pattern) { |x| [escape_character, x].join }
end

def new_sanitize_sql_like(string, escape_character = "\\")
  if string.include?(escape_character) && escape_character != "%" && escape_character != "_"
    string = string.gsub(escape_character, '\0\0')
  end

  string.gsub(/(?=[%_])/, escape_character)
end

[
  "no-special-characters",
  "one_wildcard",
  "one\\escape",
  "two_wildcards%and\\two\\escapes",
].each do |string|
  puts "\n= #{string.inspect} ".ljust(72, "=")

  Benchmark.memory do |x|
    # Warmup:
    old_sanitize_sql_like(string)
    new_sanitize_sql_like(string)

    x.report("old") { old_sanitize_sql_like(string) }
    x.report("new") { new_sanitize_sql_like(string) }
  end

  puts

  Benchmark.ips do |x|
    x.report("old") { old_sanitize_sql_like(string) }
    x.report("new") { new_sanitize_sql_like(string) }
    x.compare!
  end
end
```

Benchmark results:

```
= "no-special-characters" =============================================
Calculating -------------------------------------
                 old   875.000  memsize (     0.000  retained)
                        10.000  objects (     0.000  retained)
                         5.000  strings (     0.000  retained)
                 new    40.000  memsize (     0.000  retained)
                         1.000  objects (     0.000  retained)
                         1.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     9.483k i/100ms
                 new   119.791k i/100ms
Calculating -------------------------------------
                 old     95.694k (± 0.3%) i/s -    483.633k in   5.053985s
                 new      1.228M (± 0.5%) i/s -      6.229M in   5.072317s

Comparison:
                 new:  1228094.1 i/s
                 old:    95694.3 i/s - 12.83x  (± 0.00) slower

= "one_wildcard" ======================================================
Calculating -------------------------------------
                 old     1.395k memsize (     0.000  retained)
                        15.000  objects (     0.000  retained)
                         6.000  strings (     0.000  retained)
                 new   440.000  memsize (     0.000  retained)
                         3.000  objects (     0.000  retained)
                         1.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     6.527k i/100ms
                 new    30.205k i/100ms
Calculating -------------------------------------
                 old     65.255k (± 0.6%) i/s -    326.350k in   5.001318s
                 new    300.058k (± 0.5%) i/s -      1.510M in   5.033333s

Comparison:
                 new:   300057.6 i/s
                 old:    65255.2 i/s - 4.60x  (± 0.00) slower

= "one\\escape" =======================================================
Calculating -------------------------------------
                 old     1.395k memsize (     0.000  retained)
                        15.000  objects (     0.000  retained)
                         6.000  strings (     0.000  retained)
                 new   560.000  memsize (     0.000  retained)
                         6.000  objects (     0.000  retained)
                         2.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     6.506k i/100ms
                 new    24.312k i/100ms
Calculating -------------------------------------
                 old     65.467k (± 0.4%) i/s -    331.806k in   5.068392s
                 new    245.704k (± 0.5%) i/s -      1.240M in   5.046485s

Comparison:
                 new:   245703.8 i/s
                 old:    65466.9 i/s - 3.75x  (± 0.00) slower

= "two_wildcards%and\\two\\escapes" ===================================
Calculating -------------------------------------
                 old     1.755k memsize (     0.000  retained)
                        24.000  objects (     0.000  retained)
                         8.000  strings (     0.000  retained)
                 new   832.000  memsize (     0.000  retained)
                         8.000  objects (     0.000  retained)
                         3.000  strings (     0.000  retained)

Warming up --------------------------------------
                 old     5.265k i/100ms
                 new    12.360k i/100ms
Calculating -------------------------------------
                 old     52.698k (± 0.3%) i/s -    268.515k in   5.095425s
                 new    124.006k (± 0.3%) i/s -    630.360k in   5.083351s

Comparison:
                 new:   124006.2 i/s
                 old:    52697.9 i/s - 2.35x  (± 0.00) slower
```
@rafaelfranca rafaelfranca merged commit eeb2cfb into rails:main Mar 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants