Idea: generate cache at 'gem install' / 'bundle install' time

[FooBarWidget]

Bootsnap is a great idea! I think it can be further improved as follows: instead of generating a cache at application boot time, generate it at 'gem install' / 'bundle install' time. After all, for stable paths, gem and bundle are the only apps that could modify the contents. This avoids a big scan during application boot time.

The cache file index filenames to the directories in which they can be found, e.g.:

nokogiri/xpath.rb => ["/gems/nokogiri-1.5.2/lib", "/gems/nokogir-1.2.0/lib"]
active_support/all.rb => ["/gems/activesupport-5.0.0/lib"]

And suppose we make the cache file in an mmap()-able format. Then we can even have multiple processes share the memory occupied by this cache, which reduces memory usage.

[jules2689]

That's an interesting idea. Do you have any thoughts on how we could accomplish hooking into the install process? I don't remember there being hooks at all.

[burke]

It could be bootsnap rebuild or something like that. Bootsnap could then, at runtime, preferentially load from this cache for directories it knows about.

It would get a bit weirder for compile caching. You could, though, just recurse through GEM_HOME and whatever else, building something like .rbc files beside the rb files.

I'm not really against any of this, but the implementation we have right now was easier to optimize in order to make subsequent boots faster (We haven't really cared about the speed of the first boot past not introducing any surprise 100% penalties or anything)

[burke]

I'm not seriously against this idea, but realistically I have no plans to implement it soon and I'm sorting through the open issues. Feel free to re-open if anyone has something new to contribute here.