Switch to mini_mime for memory savings #1059
Conversation
…_mime Also - Explicit support for Ruby 1.8 and 1.9 is removed - Appraisal gem no longer needed as it was testing for mime types gem
|
my hero |
|
FYI did a similar PR to rest-client |
|
This is great! Thank you @SamSaffron. Is there any particular reason why you're removing the older versions of Ruby from the travis config? Seems out of scope for this PR. |
|
@radar it was a bit fiddly to remove the giant matrix of mime types tests and this seemed like the simplest way to resolve, when I removed stuff and left the rake hack in the Gemfile, it had issues bundling. supporting 1.8 or 1.9 seems very very bad these days ... 1.9 has no support or security patches for almost 2 years. ultimately it is your call if to support 1.8 but would appreciate some help figuring out an appropriate Gemfile hack here. |
jeremy
changed the title
|
Thanks a lot. Ruby and Rails need this kind of optimisation ! |
|
Sam you are the |
|
Am I missing something or it is not released yet? https://rubygems.org/gems/mail/ |
`mini_mime` is a minimal mime type library that's more performant and less memory hungry. https://github.com/discourse/mini_mime It has replaced `mime-types` in the `mail` gem: (which is a dependency of `actionmailer`, and by extension, `rails`) mikel/mail#1059 As well as `capybara`: teamcapybara/capybara#1884 Which also means using it as a dependency of `httparty` would be able to reuse the same dependency that should be already available in most Rails apps, instead of pulling in an extra `mime-types` dependency. The change in code is pretty straightforward, the same one made by the capybara PR linked above.
The mime types gem (even with the magical columnar store by @jeremyevans) is a memory hog.
In particular on boot, mime/types/columnar will allocate 109k objects and end up with a retained count of 31K objects.
This bloat leaves objects that need to be marked and swept every major GC and bloats ruby processes.
To circumvent mini_mime was created.
It uses the exact same database as the full fledged mime types and is capable of only loading stuff on demand with a practical, safe and bound in-memory cache.
mini_mime will allocate 398 objects on boot and only retain 62 (due to rubygems inefficiency)
In table form:
The performance of mini_mime is pretty good, cached lookups are faster than the mime types gem and uncached lookups are 10x slower. (which is really not at huge issue considering the memory savings)