Add bootsnap to default Gemfile

[burke]

This adds bootsnap to newly-generated projects, which generally reduces application boot times by over 50%.

---

I heard through the grapevine that people would be open to this.

Early releases of bootsnap only supported macOS, but recent versions now additionally support linux and (at least in theory) most *BSDs. Windows support is still not implemented.

This change is small, but not trivial, and it's been a long time since I've been in this codebase. I'm looking for guidance on how best to get this change documented, tested, and merged.

cc @rafaelfranca

[rails-bot]

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

[matthewd]

This is way too much / too specific configuration.

[kaspth]

Maybe we should follow bundlers lead:

require "bundler/setup"    # Set up gems listed in the Gemfile
require "bootsnap/setup" # Caches and compiles gems for faster boot.

Which then just includes the above conventionalized config.

[kaspth]

Could also be require "bootsnap/railtie" or Rails could just set up bootsnap automatically after executing boot.rb if the gem is present, e.g.:

# Somewhere in the Rails bowels:
begin
  require "bootsnap"

  Bootsnap.setup(…)
rescue LoadError
end

[matthewd]

We should always require bootsnap and let it decide whether it supports the current environment.

[matthewd]

Can we loosen this version constraint a bit?

Can bootsnap be installed on all platforms, even if it won't Do Cool Stuff?

[y-yagi]

bootsnap only supports Ruby 2.3.0 and higher, but Rails still supports Ruby 2.2.
https://github.com/Shopify/bootsnap/blob/master/bootsnap.gemspec#L26

Should we check the Ruby version?

[matthewd]

@burke could bootsnap allow installation on 2.2, and just operate in a degraded mode? AIUI path caching would still work -- it's the iseq cache that is strictly 2.3+, right?

[yivo]

This is not necessary thing for most Rails apps. Things must be kept as simple as possible. In future this change may bring some bugs. If people need this it is better to create separate official gem bootsnap-rails which is officially tested and ready for Rails. Merging everything is a hell.

[burke]

Ok -- I don't have any time this week to revise this, but here's what I'm going to do:

1. Get bootsnap to install correctly even on platforms where it can't use the native extension;
2. Have it understand which features can be enabled based on the current platform;
3. Add a bootsnap/setup requirable that sets up the maximal set of available features based on the environment, and assuming ActiveSupport will be in use.

The method used here will still be available, but bootsnap/setup will wrap it.

[rafaelfranca]

@yivo we are not merging anything. It is not even being added as a dependency of a framework. It is being added to the generated Gemfile and if people don't want they can just remove. Rails give people sane defaults so they don't need to spend time thinking if they want to use something or not.

@burke seems like a good plan 👍

[burke]

I've updated this PR to use bootsnap-1.1.0.pre. I'll update again to 1.1.0 after some more testing/feedback.

Changes:

• Added a jruby version that doesn't include the native extension and doesn't support compile caching;
• config/boot.rb now just requires bootsnap/setup;
• require is no longer conditional on platform;
• bootsnap/setup.rb enables whichever features are available given the current environment;
• bootsnap/setup.rb attempts to derive the cache directory from caller.

See also bootsnap/setup.rb.

Thoughts?

[jeremy]

Do we need a conservative ~> dep? How about >= 1.1.0.pre

[burke]

Sure, updated.

[grosser]

compile_cache_iseq broke code-coverage single_cov ... so had to disable it in test zendesk/samson#2038 ... so would be nice to see an easy escape hatch for this like an ENV var or defined?(Coverage) ... not urgent, would still love to see this merged, but FYI

[burke]

Yeah -- it's unfortunately hard to detect if coverage is currently enabled. Ruby, perplexingly, doesn't provide an API for this. defined?(Coverage) is about as close as we can get. I'll change that.

[burke]

Bootsnap bumped to 1.1.0. This fixes @grosser's issue, and a few other things.

IMO this is ready to go.

[guilleiguaran]

Not sure what can I be doing wrong but adding bootsnap to my app doubles the loading time:

~/src/app[master] % time bin/rails runner "puts Rails.env"
development
bin/rails runner "puts Rails.env"  10.34s user 3.74s system 56% cpu 24.864 total

~/src/app[master] % time bin/rails runner "puts Rails.env"
development
bin/rails runner "puts Rails.env"  7.22s user 1.69s system 17% cpu 51.722 total

I've tested this a dozen times getting similar numbers every time.

[burke]

Are you running it in a VM? Is the application source in a shared folder? If so, set bootsnap's cache directory outside of your app, or symlink tmp/cache to some other location on disk.

[guilleiguaran]

Are you running it in a VM? Is the application source in a shared folder?

No and no. I tried removing the tmp/cache folder anyway and creating a symlink under /tmp but I don't have luck yet.

During the first run (when cache isn't created yet) the time is very similar to the normal boot time of the app:

~/src/cookpad[master] % time bin/rails runner "puts Rails.env"
development
bin/rails runner "puts Rails.env"  8.67s user 2.57s system 44% cpu 25.136 total

During subsequent boots the time goes up to 2x the normal time:

~/src/cookpad[master] % time bin/rails runner "puts Rails.env"
development
bin/rails runner "puts Rails.env"  6.42s user 1.42s system 15% cpu 49.072 total

[burke]

Very strange, so it sounds like RubyVM::InstructionSequence.load_from_binary is running slower than loading the file directly from source.

What do you get back from:

ruby -v
uname -msr

[burke]

It might make sense to move this to an issue on bootsnap.

[guilleiguaran]

@burke sure!! rails/bootsnap#68

[kaspth]

Just one more thing… plus squash your commits and add a changelog entry. Then I think we're good to go! 🤘

[kaspth]

I don't like us needing to bundle Bootsnap in the main gemfile for our Railties tests to work. Can't we find out what Gemfile template / gems the Railties generated Rails app uses? @rafaelfranca are you familiar with gems and railties tests?

Or maybe we should just use require "bootsnap/setup" rescue LoadError in config/boot.rb.

[rafaelfranca]

The application inside the railties test suite uses the Rails main Gemfile. I don't see a problem on adding bootsnap in the main gemfile, we already do to gems like jbuilder, erubis, etc.

[burke]

Rebased, squashed, and added changelog entry.

I'm not against searching for an alternative solution if the addition to the root Gemfile is unacceptable on balance, but I'm strongly against rescuing LoadError in every downstream project to support not adding a gem to the Rails development Gemfile.

[kaspth]

I was just going to merge this, but @rafaelfranca's too fast for me! 😄

[kirs]

🎉 🎉 🎉