-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve performance of regexp matching in the sqlite adapter #2108
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm OK with supporting this, but it must be conditional and off by default for backwards compatibility. This introduces a potential memory leak for code that does not expect the regexps to be cached.
I've changed the implementation. It now only caches Regexp objects if |
Thinking about it a little longer, maybe the option |
This is how I plan to use it now: regexp_cache = Hash.new { |h, k| h[k] = Regexp.new(k, timeout: 0.005) }
opts = {
setup_regexp_function: proc { |re_str, str| regexp_cache[re_str].match? str }
}
Sequel.connect('sqlite:///path/db.sqlite3', opts) Let me know what you think. I left the |
Squash merged at c824824 |
Thanks! |
Nothing much, just what the title says. Theoretically, if an infinite amount of different regexp are used in SQL queries, it could of course fill up RAM, but I think that's uncommon.
In my case, this brought the runtime from ~300ms down to ~70ms for a query that uses one single regexp in a table with around ~5700 rows.