Add switch for "bundle clean" on "store build cache"

[apepper]

Background

I'm using multiple Gemfiles in one travis run, e.g. ./Gemfile, ./folder_a/Gemfile and ./folder_b/Gemfile. To improve the speed of one run, caching for bundle install is advised. Using directory caching (see http://docs.travis-ci.com/user/caching/#With-a-custom-install-step ) this is already possible.

Limits to this approach

Sadly right before uploading the cache (store build cache), a bundle clean is executed. This means, that all gems, which are not listed in ./Gemfile are removed from the bundle cache. So all gems from ./folder_a/Gemfile and ./folder_b/Gemfile which are not in ./Gemfile are removed. This also implies, that all Gems from folder_a and folder_b need to be installed on every travis run, thus defeating the purpose of a cache.

Workaround

It's possible to use two different bundle cache folders. One for ./Gemfile and the other for all the other Gemfiles. This implies, that whenever something is called from folder_a or folder_b an additional ENV variable needs to be set.

This is cumbersome and error prune, since forgetting to set the correct ENV:

• installing gems to the wrong cache and therefore being removed from the cache, once bundle clean will run
• or bundle exec can not be executed, because the gems where installed into another cache folder.

Wish

Please provide a way to disable bundle clean during store build cache. This would allow me to use multiple Gemfiles, while using one cache folder.

I understand, that bundle clean is there for a reason, to ensure, that the cache will not contain unneeded or obsolete files. But it's in our (the customers) interest to keep the cache folder small to improve the speed of a travis run. So a manual travis cache --delete once in awhile would be enough.

[rkh]

So, just to verify, using our gemfile directive is not possible in your case? Because then you'd have a separate cache per Gemfile: https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/script/ruby.rb#L15-L18

[apepper]

No, it's not an option, because at least in our case in one travis job multiple Gemfile's are needed. E.g. one Gemfile for packaging a gem (with asset precompile and other stuff) and the other Gemfile to use the packaged gem without all the packaging dependencies for an integration test.

[roidrage]

@rkh is there something we can/should do here to improve this?

[apepper]

Any updates on this? We still have to use workarounds with different bundle paths, which is annoying.

[kikonen-fiksu]

I would be curious also to hear about progress on this.

My scenario:

• main server repo with tests
  • Thus has its' own Gemfile
• launches server from another repo as embedded server for testing
  • Thus has its' own Gemfile

=> Enforced "bundle clean" caused that direct built in bundle caching logic doesn't work, and I ended up simply using

cache:
  directories:
  - vendor/bundle

which works partially (but I presume that it will grow in size when gems change?!?)

[kostia]

Any progress on this? Using different directories is annoying, since you have sometimes to manually clear the caches, because the suite is broken for no reason.

[joshk]

@rkh what are your thoughts on this?

[BanzaiMan]

I think it is reasonable.

cache:
  skip_bundle_clean: true

for example.

[rkh]

The

cache:
  directories:
  - vendor/bundle

version should work just fine.

but I presume that it will grow in size when gems change?!?

That is true. But it would also be true if we'd implement skip_bundle_clean: true, since bundle clean is used to keep the cache from growing indefinitely.

[apepper]

The

cache:
  directories:
  - vendor/bundle

version should work just fine.

It does. Thanks for the pointer.

[BanzaiMan]

The documentation is now live. https://docs.travis-ci.com/user/caching/#Cleaning-up-bundle