Let's cache our gems in vendor/cache!

[jdoss]

What type of PR is this? (check all applicable)

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Description

This PR adds in bundle package caching which will help speed up deployments by caching all of the gems that we use in our git repo. This way if we are doing a deployment we do not have to depend on the Internet to fetch gems. Everything we need is tied to our git repo.

This was done with the following commands on a fresh repo clone:

bundle config set cache_all true
bundle install
bundle package

The first line sets BUNDLE_CACHE_ALL: "true" in our bundler config so we cache git and path resources along with gems. This will be the default in bundler 3.0

The second line bundle install installs the gems to your BUNDLE_PATH.

The last command bundle package cached all of our gems in vendor/cache, so when you do a bundle install when you are updating or adding gems from the Gemfile, it will update our gem cache too.

You can read more about bundle package here:

https://bundler.io/man/bundle-package.1.html

Added tests?

• yes
• no, because they aren't needed
• no, because I need help

Added to documentation?

• docs.dev.to
• readme
• no documentation needed

[optional] Are there any post deployment tasks we need to perform?

Since Heroku uses bundler under the hood in their ruby buildpack this should just work. ™️

If you are doing work on our gems you should set bundle config set cache_all true and when you do bundle clean to remove outdated gems, it should remove the old gems from vendor/cache as well.

[optional] What gif best describes this PR or how it makes you feel?

Step 1: Cut a hole in that box!

Step 2: Cache our Gems in that box!

Step 3: Deploy that box!

[mstruve]

We should add bundle config set cache_all true to our bin/setup file so when people fork the repo and set it up, it is automatically done for them and they don't have to think about it.

[jdoss]

@mstruve Good idea. Added that.

[rhymes]

@mstruve @jdoss @citizen428 will this work with dependabot which automatically updates our gems? I'd say around 10 gems are updated each week.

Also, isn't this going to increase a lot the size of our repository since we start bundling binaries? I downloaded the branch and the vendor folder sits at 51 mega bytes (plus old versions of binaries will stay in the repo forever).

This is our situation right now:

$ git count-objects -vH
count: 1052
size: 4.25 MiB
in-pack: 81123
packs: 1
size-pack: 97.22 MiB
prune-packable: 0
garbage: 0
size-garbage: 0 bytes

I have the feeling these discarded binaries plus the 50+% increase in repo size (which contrary from regular text changes is going to increase linearly for binaries) are going to add up in a year's time. For contrast, we have less than 4 mega bytes of binary images and they don't change that often.

Also, does this work with different architectures? For example, nokogiri needs to be compiled.

Last but not least: is the cache tied to the version of Ruby? Will we have to re-fresh it completely when we change versions?

[jdoss]

@mstruve @jdoss @citizen428 will this work with dependabot which automatically updates our gems? I'd say around 10 gems are updated each week.

Yes, it should work as is but since Dependabot just updates the Gemfile with the new versions it won't add the gems to vendor/cache. It seems that Dependabot is missing this functionality to add them into the repo. It's reported here: https://github.com/dependabot/feedback/issues/474 and they have a workaround with a GitHub action that we can try if we find it to be an issue.

Also, isn't this going to increase a lot the size of our repository since we start bundling binaries? I downloaded the branch and the vendor folder sits at 51 mega bytes (plus old versions of binaries will stay in the repo forever).

This vendoring .gem files doesn't build any binaries. It caches the .gem files only.

I have the feeling these discarded binaries plus the 50+% increase in repo size (which contrary from regular text changes is going to increase linearly for binaries) are going to add up in a year's time. For contrast, we have less than 4 mega bytes of binary images and they don't change that often.

This is why after a gem update we need to run bundle clean which will clean up the old .gem files in vendor/cache as well. Support for cleaning vendor/cache was added to bundle clean back in 2011. rubygems/bundler@06c7f88

Also, does this work with different architectures? For example, nokogiri needs to be compiled.

This PR doesn't replace the bundle install step. We still have to install (and compile gems like nokogiri) the gems in GEM_HOME with bundle install

If we ever have gems that are platform specific https://bundler.io/man/bundle-package.1.html#SUPPORT-FOR-MULTIPLE-PLATFORMS fixes that issue. AFAIK we do not have this issue yet.

Last but not least: is the cache tied to the version of Ruby? Will we have to re-fresh it completely when we change versions?

No, isn't tied to any ruby version as it is just caching the .gem files.

This cuts out the need to have to rely on fetching gems from the Internet N number of times every time we run through our CI/CD. It's pretty standard practice for any Rails project I have worked on and it ensures that deploys do not break due to Rubygems taking a dive.

TL; DR It's simply keeping a .gem file cache nothing else.

[mstruve]

🔥

[rhymes]

@jdoss thanks for the explanation and clarification!

[maestromac]

Thank you for the explanation @jdoss ! Can't wait for the speed improvement!