Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add switch for "bundle clean" on "store build cache" #2518

Closed
apepper opened this issue Jul 11, 2014 · 11 comments
Closed

Add switch for "bundle clean" on "store build cache" #2518

apepper opened this issue Jul 11, 2014 · 11 comments
Labels

Comments

@apepper
Copy link

@apepper apepper commented Jul 11, 2014

Background

I'm using multiple Gemfiles in one travis run, e.g. ./Gemfile, ./folder_a/Gemfile and ./folder_b/Gemfile. To improve the speed of one run, caching for bundle install is advised. Using directory caching (see http://docs.travis-ci.com/user/caching/#With-a-custom-install-step ) this is already possible.

Limits to this approach

Sadly right before uploading the cache (store build cache), a bundle clean is executed. This means, that all gems, which are not listed in ./Gemfile are removed from the bundle cache. So all gems from ./folder_a/Gemfile and ./folder_b/Gemfile which are not in ./Gemfile are removed. This also implies, that all Gems from folder_a and folder_b need to be installed on every travis run, thus defeating the purpose of a cache.

Workaround

It's possible to use two different bundle cache folders. One for ./Gemfile and the other for all the other Gemfiles. This implies, that whenever something is called from folder_a or folder_b an additional ENV variable needs to be set.

This is cumbersome and error prune, since forgetting to set the correct ENV:

  • installing gems to the wrong cache and therefore being removed from the cache, once bundle clean will run
  • or bundle exec can not be executed, because the gems where installed into another cache folder.

Wish

Please provide a way to disable bundle clean during store build cache. This would allow me to use multiple Gemfiles, while using one cache folder.

I understand, that bundle clean is there for a reason, to ensure, that the cache will not contain unneeded or obsolete files. But it's in our (the customers) interest to keep the cache folder small to improve the speed of a travis run. So a manual travis cache --delete once in awhile would be enough.

@rkh
Copy link
Contributor

@rkh rkh commented Jul 14, 2014

So, just to verify, using our gemfile directive is not possible in your case? Because then you'd have a separate cache per Gemfile: https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/script/ruby.rb#L15-L18

@apepper
Copy link
Author

@apepper apepper commented Jul 14, 2014

No, it's not an option, because at least in our case in one travis job multiple Gemfile's are needed. E.g. one Gemfile for packaging a gem (with asset precompile and other stuff) and the other Gemfile to use the packaged gem without all the packaging dependencies for an integration test.

@roidrage
Copy link
Contributor

@roidrage roidrage commented Aug 20, 2014

@rkh is there something we can/should do here to improve this?

@apepper
Copy link
Author

@apepper apepper commented Nov 25, 2014

Any updates on this? We still have to use workarounds with different bundle paths, which is annoying.

@kikonen-fiksu
Copy link

@kikonen-fiksu kikonen-fiksu commented Jan 16, 2015

I would be curious also to hear about progress on this.

My scenario:

  • main server repo with tests
    • Thus has its' own Gemfile
  • launches server from another repo as embedded server for testing
    • Thus has its' own Gemfile

=> Enforced "bundle clean" caused that direct built in bundle caching logic doesn't work, and I ended up simply using

cache:
  directories:
  - vendor/bundle

which works partially (but I presume that it will grow in size when gems change?!?)

@kostia
Copy link

@kostia kostia commented Feb 2, 2015

Any progress on this? Using different directories is annoying, since you have sometimes to manually clear the caches, because the suite is broken for no reason.

@joshk
Copy link
Contributor

@joshk joshk commented Feb 23, 2015

@rkh what are your thoughts on this?

@BanzaiMan
Copy link
Member

@BanzaiMan BanzaiMan commented Jul 14, 2015

I think it is reasonable.

cache:
  skip_bundle_clean: true

for example.

@rkh
Copy link
Contributor

@rkh rkh commented Jul 14, 2015

The

cache:
  directories:
  - vendor/bundle

version should work just fine.

but I presume that it will grow in size when gems change?!?

That is true. But it would also be true if we'd implement skip_bundle_clean: true, since bundle clean is used to keep the cache from growing indefinitely.

@apepper
Copy link
Author

@apepper apepper commented Jul 29, 2015

The

cache:
  directories:
  - vendor/bundle

version should work just fine.

It does. Thanks for the pointer.

@BanzaiMan
Copy link
Member

@BanzaiMan BanzaiMan commented Feb 1, 2017

@BanzaiMan BanzaiMan closed this Feb 1, 2017
i2000s added a commit to i2000s/en that referenced this issue May 9, 2017
y-yagi added a commit to y-yagi/rails that referenced this issue Feb 2, 2018
I'm not sure cause, but due to the influence of `gem clean`, the expected
gem seems not to be installed correctly.
In order to avoid a test failure due to this, I fixed that `gem clean` not be executed.
Ref: travis-ci/travis-ci#2518 (comment)

This is a workaround. If Travis fixes something, please revert this.
y-yagi added a commit to rails/rails that referenced this issue Feb 2, 2018
I'm not sure cause, but due to the influence of `gem clean`, the expected
gem seems not to be installed correctly.
In order to avoid a test failure due to this, I fixed that `gem clean` not be executed.
Ref: travis-ci/travis-ci#2518 (comment)

This is a workaround. If Travis fixes something, please revert this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.