Switch to mini_mime for memory savings #1059

wants to merge 2 commits into


None yet

6 participants

SamSaffron commented Dec 14, 2016 edited

The mime types gem (even with the magical columnar store by @jeremyevans) is a memory hog.

In particular on boot, mime/types/columnar will allocate 109k objects and end up with a retained count of 31K objects.

This bloat leaves objects that need to be marked and swept every major GC and bloats ruby processes.

To circumvent mini_mime was created.

It uses the exact same database as the full fledged mime types and is capable of only loading stuff on demand with a practical, safe and bound in-memory cache.

mini_mime will allocate 398 objects on boot and only retain 62 (due to rubygems inefficiency)

In table form:

boot allocations boot retained lookup lookup uncached
mini_mime 398 62 641K/s 33K/s
mime-types 109796 31165 361K/s 361K/s

The performance of mini_mime is pretty good, cached lookups are faster than the mime types gem and uncached lookups are 10x slower. (which is really not at huge issue considering the memory savings)

SamSaffron added some commits Dec 14, 2016
@SamSaffron SamSaffron FEATURE: remove the dependency on the mime types gem in place of mini…


- Explicit support for Ruby 1.8 and 1.9 is removed
- Appraisal gem no longer needed as it was testing for mime types gem
@SamSaffron SamSaffron try to fix travis 2986672
mperham commented Dec 14, 2016

my hero


FYI did a similar PR to rest-client


radar commented Dec 14, 2016

This is great! Thank you @samsaffron.

Is there any particular reason why you're removing the older versions of Ruby from the travis config? Seems out of scope for this PR.

SamSaffron commented Dec 14, 2016 edited

@radar it was a bit fiddly to remove the giant matrix of mime types tests and this seemed like the simplest way to resolve, when I removed stuff and left the rake hack in the Gemfile, it had issues bundling.

supporting 1.8 or 1.9 seems very very bad these days ... 1.9 has no support or security patches for almost 2 years.

ultimately it is your call if to support 1.8 but would appreciate some help figuring out an appropriate Gemfile hack here.

@jeremy jeremy added this to the 2.7.0 milestone Dec 14, 2016
jeremy commented Dec 14, 2016

Squashed with #1060 and merged @ 634d911 🖖

@jeremy jeremy closed this Dec 14, 2016
@jeremy jeremy changed the title from FEATURE: remove the dependency on the mime types use mini_mime to Switch to mini_mime for memory savings Dec 14, 2016
czj commented Dec 14, 2016

Thanks a lot. Ruby and Rails need this kind of optimisation !


Sam you are the mini_* master 😎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment