Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regex string allocation #1943

Merged
merged 4 commits into from
Dec 16, 2019
Merged

Conversation

ericproulx
Copy link
Contributor

While memory profilling our app, I found these strange memory allocations :

1442  "(?<_"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
468  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

983  ")"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 9  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/mustermann-grape-1.0.0/lib/mustermann/grape.rb:20

974  ">"
974  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41

468  ">)"
468  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53
12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:a|%61)(?:c|%63)(?:c|%63)(?:o|%6f|%6F)(?:u|%75)(?:n|%6e|%6E)(?:t|%74)(?:s|%73)\\/(?<account_id>[^\\/\\?#.]+)\\/(?:m|%6d|%6D)(?:e|%65)(?:s|%73)(?"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:a|%61)(?:p|%70)(?:p|%70)(?:o|%6f|%6F)(?:i|%69)(?:n|%6e|%6E)(?:t|%74)(?:m|%6d|%6D)(?:e|%65)(?:n|%6e|%6E)(?:t|%74)(?:s|%73)\\/(?:a|%61)(?:v|%7"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:c|%63)(?:h|%68)(?:a|%61)(?:n|%6e|%6E)(?:g|%67)(?:e|%65)(?:_|%5f|%5F)(?:r|%72)(?:e|%65)(?:q|%71)(?:u|%75)(?:e|%65)(?:s|%73)(?:t|%74)(?:s|%73"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:c|%63)(?:o|%6f|%6F)(?:n|%6e|%6E)(?:v|%76)(?:e|%65)(?:r|%72)(?:s|%73)(?:a|%61)(?:t|%74)(?:i|%69)(?:o|%6f|%6F)(?:n|%6e|%6E)(?:s|%73)\\/(?<conv"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:a|%61)(?:p|%70)(?:p|%70)(?:o|%6f|%6F)(?:i|%69)(?:n|%6e"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:f|%66)(?:o|%6f|%6F)(?:r|%72)(?:m|%6d|%6D)(?:s|%73)\\/(?"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:m|%6d|%6D)(?:a|%61)(?:n|%6e|%6E)(?:a|%61)(?:g|%67)(?:e"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<group_id>[^\\/\\?#.]+)\\/(?:t|%74)(?:e|%65)(?:a|%61)(?:m|%6d|%6D)(?:s|%73)(?:(?:\\."
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:h|%68)(?:o|%6f|%6F)(?:s|%73)(?:p|%70)(?:i|%69)(?:t|%74)(?:a|%61)(?:l|%6c|%6C)(?:s|%73)\\/(?<hospital_id>[^\\/\\?#.]+)\\/(?:c|%63)(?:o|%6f|%6F)("
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

12  "(?-mix:\\A(?-mix:\\/(?<version>(?-mix:(?:v|%76)(?:2|%32)))\\/(?:s|%73)(?:c|%63)(?:h|%68)(?:e|%65)(?:d|%64)(?:u|%75)(?:l|%6c|%6C)(?:e|%65)\\/(?:g|%67)(?:r|%72)(?:o|%6f|%6F)(?:u|%75)(?:p|%70)(?:s|%73)\\/(?<i"
10  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:41
 2  /Users/eproulx/.rbenv/versions/2.4.9/lib/ruby/gems/2.4.0/gems/grape-1.2.6/lib/grape/router.rb:53

Turns out, its 2 regex defined in router.rb with interpolation. I replaced the 2 lliteral regexs by the Regexp constructor. Therefore, the strings above (memory profiling) are just allocated once and benefits from the frozen_string_literal

@grape-bot
Copy link

grape-bot commented Dec 14, 2019

1 Warning
⚠️ There’re library changes, but not tests. That’s OK as long as you’re refactoring existing code.

Generated by 🚫 danger

CHANGELOG.md Outdated
@@ -3,6 +3,7 @@
#### Features

* Your contribution here.
* [#1943](https://github.com/ruby-grape/grape/pull/1943): Regex string allocation - [@ericproulx](https://github.com/ericproulx).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick, this should describe what it does, eg. "Reduce number of regex string allocations"

@dblock
Copy link
Member

dblock commented Dec 14, 2019

Great work. Rebase, I'll merge.

CHANGELOG.md Show resolved Hide resolved
CHANGELOG.md Outdated
@@ -3,7 +3,8 @@
#### Features

* Your contribution here.
* [#1942](https://github.com/ruby-grape/grape/pull/1942): Optimized retained memory methods - [@ericproulx](https://github.com/ericproulx).
* [#1943](https://github.com/ruby-grape/grape/pull/1943): Reduce number of regex string allocations - [@ericproulx](https://github.com/ericproulx).
[#1942](https://github.com/ruby-grape/grape/pull/1942): Optimized retained memory methods - [@ericproulx](https://github.com/ericproulx).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing * so danger complains.

@dblock dblock merged commit d30b780 into ruby-grape:master Dec 16, 2019
@dblock
Copy link
Member

dblock commented Dec 16, 2019

Merged thank you

basjanssen pushed a commit to basjanssen/grape that referenced this pull request Feb 28, 2020
Use Regexp instead of // for regex in router.rb for a more efficient string allocation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants