bundle caching and bundle install --local does not work with private git based gems

[sajusat]

Describe the problem as clearly as you can

I am making a docker image for an appliation with official ruby image as the base (verion 3.1.0). As part of the multi-stage image build, I cache all the gems with the following commands

...
RUN    gem install bundler:2.3.13 && \
            bundle config set --local cache_path 'vendor/cache' && \
            bundle config set --local cache_all 'true' && \
            bundle config set --local cache_all_platforms 'true' && \
            bundle config set --local allow_offline_install 'true' && \
            bundle cache

In another stage, I copy over the application directory which contains the cached "vendor/cache" folder and run the installation.

...
COPY    --from=app-gem-base /contract-service/ ./
.....

RUN     bundle config set --local path 'vendor/cache' && \
             bundle config set --local allow_offline_install 'true' && \
             bundle install --local --jobs 20 --retry 2

Bundle is trying to access pull the private git gems again from the repository. (it is not possible as I access those with https and gitlab token). It errors out after 3 retries and then tries to get the cached git data from the application folders vendor/cache, but with a non existing one.

Using cached git data because of network errors:
#19 7.729 Git error: command `git clone --bare --no-hardlinks --quiet -- git@gitlab.xxxx.de:/services-base.git /contract-service/vendor/cache/ruby/3.1.0/cache/bundler/git/services-base-27d49680d3a1ae5bdad625579b9139b5174f79ed` in directory /contract-service has failed.

There is a folder in the first phase of the image (/usr/local/bundle/cache/bundler/git). It looks like it is cached here, with contents of the private git gem

#15 0.246 drwxr-xr-x 7 root root 4096 Mar 16 18:44 services-base-f5f916b3f3a9c97713b85aa75a6a5834d198380c

There is also a directory entry for the private gem in the vendor/cache folder with the lastest short commit hash as suffix (given below)

#19 0.199 drwxr-xr-x 8 root root 4096 Mar 16 18:45 services-base-227c7d1e41c1

For some reason the bundler after trying to fetch from the private git repo, it trying to access the folder contract-service/vendor/cache/ruby/3.1.0/cache/bundler/git/ which is not present in the application vendor/cache direcotry. It is trying to access services-base-27d49680d3a1ae5bdad625579b9139b5174f79ed (latest commit hash as prefix). None of these are obviously there.

1. Why should the previously cached gem access the internet when it is specifically says to install offline from the cached direcotry. (It defeats the purpose of caching itself)
2. If the point (1) is justified, then, It is a security issue to provide the token to access the private git repository.

Did you try upgrading rubygems & bundler?

Yes. It is just the same.

Post steps to reproduce the problem

1. A private git repository with a gem project
2. A ruby application that uses the gem
3. gem 'services-base', gemsource: "projs/services-base", branch: "master", require: false. ( e.g. Gemfile entry, with https access to the private git gem repo )
4. Multi-stage dockerfile with commands from above to cache and install from cache directory
5. docker build teh final stage.

Which command did you run?

bundle install --local

What were you expecting to happen?

The copied cached directory should contain all the gems and dependencies to install from the cached directory without hitting the internet.

What actually happened?

Frist it is trying to fetch from the internet. Then looking for wrong cached directories in the cached directory.

If not included with the output of your command, run bundle env and paste the output below

#17 0.416 ## Environment
#17 0.416 
#17 0.416 ```
#17 0.416 Bundler       2.3.3
#17 0.416   Platforms   ruby, x86_64-linux
#17 0.416 Ruby          3.1.0p0 (2021-12-25 revision fb4df44d1670e9d25aef6b235a7281199a177edb) [x86_64-linux]
#17 0.416   Full Path   /usr/local/bin/ruby
#17 0.416   Config Dir  /usr/local/etc
#17 0.416 RubyGems      3.3.3
#17 0.416   Gem Home    /contract-service/vendor/cache/ruby/3.1.0
#17 0.416   Gem Path    /contract-service/vendor/cache/ruby/3.1.0
#17 0.416   User Home   /root
#17 0.416   User Path   /root/.local/share/gem/ruby/3.1.0
#17 0.416   Bin Dir     /contract-service/vendor/cache/ruby/3.1.0/bin
#17 0.416 Tools         
#17 0.416   Git         2.20.1
#17 0.416   RVM         not installed
#17 0.416   rbenv       not installed
#17 0.416   chruby      not installed
#17 0.416 ```
#17 0.416 
#17 0.416 ## Bundler Build Metadata
#17 0.416 
#17 0.416 ```
#17 0.416 Built At          2023-03-16
#17 0.416 Git SHA           unknown
#17 0.416 Released Version  false
#17 0.416 ```
#17 0.416 
#17 0.416 ## Bundler settings
#17 0.416 
#17 0.416 ```
#17 0.416 allow_offline_install
#17 0.416   Set for your local app (/usr/local/bundle/config): true
#17 0.416 app_config
#17 0.416   Set via BUNDLE_APP_CONFIG: "/usr/local/bundle"
#17 0.416 path
#17 0.416   Set for your local app (/usr/local/bundle/config): "vendor/cache"
#17 0.416 silence_root_warning
#17 0.416   Set via BUNDLE_SILENCE_ROOT_WARNING: true
#17 0.416 ```
#17 0.416 
#17 0.416 ## Gemfile
#17 0.416 
#17 0.416 ### Gemfile
#17 0.416 
#17 0.416 ```ruby
#17 0.416 # frozen_string_literal: true
#17 0.416 ruby '3.1.0'
#17 0.416 
#17 0.416 source "https://rubygems.org"
#17 0.416 
#17 0.416 git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }
#17 0.416 git_source(:gitlab) { |repo|
#17 0.416   ENV["CI_ACCESS_TOKEN"] ?
#17 0.416   "https://#{ENV["GITLAB_USERNAME"]}:#{ENV["CI_ACCESS_TOKEN"]}@#{ENV["GITLAB_HOSTNAME"]}:#{ENV["GITLAB_PORT"]}/#{repo}.git" :
#17 0.416   "git@gitlab.xxxxx.de:#{repo}.git"
#17 0.416 }
#17 0.416 
#17 0.416 gem 'dotenv'
#17 0.416 gem 'zeitwerk', require: false
#17 0.416 
#17 0.416 gem 'puma'
#17 0.416 gem 'services-base', gitlab: "projs/services-base", branch: "master", require: false
#17 0.416 
#17 0.416 
#17 0.416 gem "pg"
#17 0.416 gem "sequel"
#17 0.416 
#17 0.416 gem "rake"
#17 0.416 gem 'pry'
#17 0.416 gem 'pry-nav'
#17 0.416 
#17 0.416 gem "nokogiri"
#17 0.416 gem "aws-sdk-s3"
#17 0.416 
#17 0.416 group :test, :development do
#17 0.416   gem 'rubocop'
#17 0.416   gem 'byebug'
#17 0.416 end
#17 0.416 
#17 0.416 group :test do
#17 0.416   gem 'simplecov', "0.21.2", require: false
#17 0.416   gem 'simplecov-cobertura', require: false
#17 0.416 
#17 0.416   gem 'webmock', '~> 3.13.0' # Mock of 3rd party call
#17 0.416 
#17 0.416   gem 'cucumber'
#17 0.416 
#17 0.416   gem 'rspec'
#17 0.416   gem 'rspec-core', '~> 3.10'
#17 0.416   gem 'rspec_junit_formatter'
#17 0.416 
#17 0.416   gem 'rack-test'
#17 0.416 end
#17 0.416 ```
#17 0.416 
#17 0.416 ### Gemfile.lock
#17 0.416 
#17 0.416 ```
#17 0.416 GIT
#17 0.416   remote: https://gitlab-ci-token:xxxxxxxxx@gitlab.xxxx.de:/projs/services-base.git
#17 0.416   revision: 227c7d1e41c128e40ced3db6778a824bd00724a9
#17 0.416   branch: master
#17 0.416   specs:
#17 0.416     services-base (1.0)
#17 0.416       activesupport (~> 6.0.3)
#17 0.416       dry-logic
#17 0.416       dry-schema (~> 1.9.1)
#17 0.416       dry-types (~> 1.5.0)
#17 0.416       dry-validation (~> 1.8.1)
#17 0.416       httparty (~> 0.20)
#17 0.416       i18n (~> 1.10.0)
#17 0.416       mimemagic (~> 0.4.3)
#17 0.416       nats-pure (~> 2.0)
#17 0.416       parse-cron (~> 0.1.4)
#17 0.416       shrine (~> 3.4.0)
#17 0.416       sinatra (~> 3.0.2)
#17 0.416       sinatra-contrib (~> 3.0.2)
#17 0.416       thor (~> 1.2.1)
#17 0.416 
#17 0.416 GEM
#17 0.416   remote: https://rubygems.org/
#17 0.416   specs:
#17 0.416     activesupport (6.0.6.1)
#17 0.416       concurrent-ruby (~> 1.0, >= 1.0.2)
#17 0.416       i18n (>= 0.7, < 2)
#17 0.416       minitest (~> 5.1)
#17 0.416       tzinfo (~> 1.1)
#17 0.416       zeitwerk (~> 2.2, >= 2.2.2)
#17 0.416     addressable (2.8.1)
#17 0.416       public_suffix (>= 2.0.2, < 6.0)
#17 0.416     ast (2.4.2)
#17 0.416     aws-eventstream (1.2.0)
#17 0.416     aws-partitions (1.607.0)
#17 0.416     aws-sdk-core (3.131.2)
#17 0.416       aws-eventstream (~> 1, >= 1.0.2)
#17 0.416       aws-partitions (~> 1, >= 1.525.0)
#17 0.416       aws-sigv4 (~> 1.1)
#17 0.416       jmespath (~> 1, >= 1.6.1)
#17 0.416     aws-sdk-kms (1.57.0)
#17 0.416       aws-sdk-core (~> 3, >= 3.127.0)
#17 0.416       aws-sigv4 (~> 1.1)
#17 0.416     aws-sdk-s3 (1.114.0)
#17 0.416       aws-sdk-core (~> 3, >= 3.127.0)
#17 0.416       aws-sdk-kms (~> 1)
#17 0.416       aws-sigv4 (~> 1.4)
#17 0.416     aws-sigv4 (1.5.0)
#17 0.416       aws-eventstream (~> 1, >= 1.0.2)
#17 0.416     builder (3.2.4)
#17 0.416     byebug (11.1.3)
#17 0.416     coderay (1.1.3)
#17 0.416     concurrent-ruby (1.2.2)
#17 0.416     content_disposition (1.0.0)
#17 0.416     crack (0.4.5)
#17 0.416       rexml
#17 0.416     cucumber (8.0.0)
#17 0.416       builder (~> 3.2, >= 3.2.4)
#17 0.416       cucumber-ci-environment (~> 9.0, >= 9.0.4)
#17 0.416       cucumber-core (~> 11.0, >= 11.0.0)
#17 0.416       cucumber-cucumber-expressions (~> 15.1, >= 15.1.1)
#17 0.416       cucumber-gherkin (~> 23.0, >= 23.0.1)
#17 0.416       cucumber-html-formatter (~> 19.1, >= 19.1.0)
#17 0.416       cucumber-messages (~> 18.0, >= 18.0.0)
#17 0.416       diff-lcs (~> 1.5, >= 1.5.0)
#17 0.416       mime-types (~> 3.4, >= 3.4.1)
#17 0.416       multi_test (~> 1.1, >= 1.1.0)
#17 0.416       sys-uname (~> 1.2, >= 1.2.2)
#17 0.416     cucumber-ci-environment (9.0.4)
#17 0.416     cucumber-core (11.0.0)
#17 0.416       cucumber-gherkin (~> 23.0, >= 23.0.1)
#17 0.416       cucumber-messages (~> 18.0, >= 18.0.0)
#17 0.416       cucumber-tag-expressions (~> 4.1, >= 4.1.0)
#17 0.416     cucumber-cucumber-expressions (15.2.0)
#17 0.416     cucumber-gherkin (23.0.1)
#17 0.416       cucumber-messages (~> 18.0, >= 18.0.0)
#17 0.416     cucumber-html-formatter (19.2.0)
#17 0.416       cucumber-messages (~> 18.0, >= 18.0.0)
#17 0.416     cucumber-messages (18.0.0)
#17 0.416     cucumber-tag-expressions (4.1.0)
#17 0.416     diff-lcs (1.5.0)
#17 0.416     docile (1.4.0)
#17 0.416     dotenv (2.7.6)
#17 0.416     down (5.4.0)
#17 0.416       addressable (~> 2.8)
#17 0.416     dry-configurable (0.16.1)
#17 0.416       dry-core (~> 0.6)
#17 0.416       zeitwerk (~> 2.6)
#17 0.416     dry-container (0.11.0)
#17 0.416       concurrent-ruby (~> 1.0)
#17 0.416     dry-core (0.9.1)
#17 0.416       concurrent-ruby (~> 1.0)
#17 0.416       zeitwerk (~> 2.6)
#17 0.416     dry-inflector (0.3.0)
#17 0.416     dry-initializer (3.1.1)
#17 0.416     dry-logic (1.3.0)
#17 0.416       concurrent-ruby (~> 1.0)
#17 0.416       dry-core (~> 0.9, >= 0.9)
#17 0.416       zeitwerk (~> 2.6)
#17 0.416     dry-schema (1.9.3)
#17 0.416       concurrent-ruby (~> 1.0)
#17 0.416       dry-configurable (~> 0.13, >= 0.13.0)
#17 0.416       dry-core (~> 0.5, >= 0.5)
#17 0.416       dry-initializer (~> 3.0)
#17 0.416       dry-logic (~> 1.0)
#17 0.416       dry-types (~> 1.5)
#17 0.416     dry-types (1.5.1)
#17 0.416       concurrent-ruby (~> 1.0)
#17 0.416       dry-container (~> 0.3)
#17 0.416       dry-core (~> 0.5, >= 0.5)
#17 0.416       dry-inflector (~> 0.1, >= 0.1.2)
#17 0.416       dry-logic (~> 1.0, >= 1.0.2)
#17 0.416     dry-validation (1.8.1)
#17 0.416       concurrent-ruby (~> 1.0)
#17 0.416       dry-container (~> 0.7, >= 0.7.1)
#17 0.416       dry-core (~> 0.5, >= 0.5)
#17 0.416       dry-initializer (~> 3.0)
#17 0.416       dry-schema (~> 1.8, >= 1.8.0)
#17 0.416     ffi (1.15.5)
#17 0.416     hashdiff (1.0.1)
#17 0.416     httparty (0.21.0)
#17 0.416       mini_mime (>= 1.0.0)
#17 0.416       multi_xml (>= 0.5.2)
#17 0.416     i18n (1.10.0)
#17 0.416       concurrent-ruby (~> 1.0)
#17 0.416     jmespath (1.6.1)
#17 0.416     json (2.6.2)
#17 0.416     jwt (2.2.3)
#17 0.416     method_source (1.0.0)
#17 0.416     mime-types (3.4.1)
#17 0.416       mime-types-data (~> 3.2015)
#17 0.416     mime-types-data (3.2022.0105)
#17 0.416     mimemagic (0.4.3)
#17 0.416       nokogiri (~> 1)
#17 0.416       rake
#17 0.416     mini_mime (1.1.2)
#17 0.416     minitest (5.18.0)
#17 0.416     multi_json (1.15.0)
#17 0.416     multi_test (1.1.0)
#17 0.416     multi_xml (0.6.0)
#17 0.416     mustermann (3.0.0)
#17 0.416       ruby2_keywords (~> 0.0.1)
#17 0.416     nats-pure (2.2.1)
#17 0.416     nio4r (2.5.8)
#17 0.416     nokogiri (1.13.10-arm64-darwin)
#17 0.416       racc (~> 1.4)
#17 0.416     nokogiri (1.13.10-x86_64-darwin)
#17 0.416       racc (~> 1.4)
#17 0.416     nokogiri (1.13.10-x86_64-linux)
#17 0.416       racc (~> 1.4)
#17 0.416     parallel (1.22.1)
#17 0.416     parse-cron (0.1.4)
#17 0.416     parser (3.1.2.0)
#17 0.416       ast (~> 2.4.1)
#17 0.416     pg (1.4.1)
#17 0.416     pry (0.14.1)
#17 0.416       coderay (~> 1.1)
#17 0.416       method_source (~> 1.0)
#17 0.416     pry-nav (1.0.0)
#17 0.416       pry (>= 0.9.10, < 0.15)
#17 0.416     public_suffix (5.0.1)
#17 0.416     puma (5.6.4)
#17 0.416       nio4r (~> 2.0)
#17 0.416     racc (1.6.2)
#17 0.416     rack (2.2.5)
#17 0.416     rack-protection (3.0.5)
#17 0.416       rack
#17 0.416     rack-test (2.0.2)
#17 0.416       rack (>= 1.3)
#17 0.416     rainbow (3.1.1)
#17 0.416     rake (13.0.6)
#17 0.416     regexp_parser (2.5.0)
#17 0.416     rexml (3.2.5)
#17 0.416     rspec (3.11.0)
#17 0.416       rspec-core (~> 3.11.0)
#17 0.416       rspec-expectations (~> 3.11.0)
#17 0.416       rspec-mocks (~> 3.11.0)
#17 0.416     rspec-core (3.11.0)
#17 0.416       rspec-support (~> 3.11.0)
#17 0.416     rspec-expectations (3.11.0)
#17 0.416       diff-lcs (>= 1.2.0, < 2.0)
#17 0.416       rspec-support (~> 3.11.0)
#17 0.416     rspec-mocks (3.11.1)
#17 0.416       diff-lcs (>= 1.2.0, < 2.0)
#17 0.416       rspec-support (~> 3.11.0)
#17 0.416     rspec-support (3.11.0)
#17 0.416     rspec_junit_formatter (0.5.1)
#17 0.416       rspec-core (>= 2, < 4, != 2.12.0)
#17 0.416     rubocop (1.31.2)
#17 0.416       json (~> 2.3)
#17 0.416       parallel (~> 1.10)
#17 0.416       parser (>= 3.1.0.0)
#17 0.416       rainbow (>= 2.2.2, < 4.0)
#17 0.416       regexp_parser (>= 1.8, < 3.0)
#17 0.416       rexml (>= 3.2.5, < 4.0)
#17 0.416       rubocop-ast (>= 1.18.0, < 2.0)
#17 0.416       ruby-progressbar (~> 1.7)
#17 0.416       unicode-display_width (>= 1.4.0, < 3.0)
#17 0.416     rubocop-ast (1.19.1)
#17 0.416       parser (>= 3.1.1.0)
#17 0.416     ruby-progressbar (1.11.0)
#17 0.416     ruby2_keywords (0.0.5)
#17 0.416     sequel (5.58.0)
#17 0.416     shrine (3.4.0)
#17 0.416       content_disposition (~> 1.0)
#17 0.416       down (~> 5.1)
#17 0.416     simplecov (0.21.2)
#17 0.416       docile (~> 1.1)
#17 0.416       simplecov-html (~> 0.11)
#17 0.416       simplecov_json_formatter (~> 0.1)
#17 0.416     simplecov-cobertura (2.1.0)
#17 0.416       rexml
#17 0.416       simplecov (~> 0.19)
#17 0.416     simplecov-html (0.12.3)
#17 0.416     simplecov_json_formatter (0.1.4)
#17 0.416     sinatra (3.0.5)
#17 0.416       mustermann (~> 3.0)
#17 0.416       rack (~> 2.2, >= 2.2.4)
#17 0.416       rack-protection (= 3.0.5)
#17 0.416       tilt (~> 2.0)
#17 0.416     sinatra-contrib (3.0.5)
#17 0.416       multi_json
#17 0.416       mustermann (~> 3.0)
#17 0.416       rack-protection (= 3.0.5)
#17 0.416       sinatra (= 3.0.5)
#17 0.416       tilt (~> 2.0)
#17 0.416     sys-uname (1.2.2)
#17 0.416       ffi (~> 1.1)
#17 0.416     thor (1.2.1)
#17 0.416     thread_safe (0.3.6)
#17 0.416     tilt (2.1.0)
#17 0.416     tzinfo (1.2.11)
#17 0.416       thread_safe (~> 0.1)
#17 0.416     unicode-display_width (2.2.0)
#17 0.416     webmock (3.13.0)
#17 0.416       addressable (>= 2.3.6)
#17 0.416       crack (>= 0.3.2)
#17 0.416       hashdiff (>= 0.4.0, < 2.0.0)
#17 0.416     zeitwerk (2.6.6)
#17 0.416 
#17 0.416 PLATFORMS
#17 0.416   arm64-darwin-21
#17 0.416   x86_64-darwin-19
#17 0.416   x86_64-darwin-20
#17 0.416   x86_64-darwin-21
#17 0.416   x86_64-linux-s
#17 0.416 
#17 0.416 DEPENDENCIES
#17 0.416   aws-sdk-s3
#17 0.416   byebug
#17 0.416   cucumber
#17 0.416   dotenv
#17 0.416   services-base!
#17 0.416   nokogiri
#17 0.416   pg
#17 0.416   pry
#17 0.416   pry-nav
#17 0.416   puma
#17 0.416   rack-test
#17 0.416   rake
#17 0.416   rspec
#17 0.416   rspec-core (~> 3.10)
#17 0.416   rspec_junit_formatter
#17 0.416   rubocop
#17 0.416   sequel
#17 0.416   simplecov (= 0.21.2)
#17 0.416   simplecov-cobertura
#17 0.416   webmock (~> 3.13.0)
#17 0.416   zeitwerk
#17 0.416 
#17 0.416 RUBY VERSION
#17 0.416    ruby 3.1.0p0
#17 0.416 
#17 0.416 BUNDLED WITH
#17 0.416    2.3.13
#17 0.416 ```

[deivid-rodriguez]

This seems very similar if not a duplicate of #3204 and #4226, and will get fixed by #4469. Can you check whether any of those issues look the same as yours?

[sajusat]

@deivid-rodriguez Hi, thanks for your reply. I looked at all the issues. I believe my issue is only a partially related to the #4226. The way I see it, the issue is only with the different folder names with different hash suffix.

1. bundle install is not installing project vendor cached git gems, but using the cache as the install location #3204 talks about my minor pain point that, bundler trying to access the internet even thought everything is in cache. ( I have asked this in stack overflow but nobody really answered it. https://stackoverflow.com/questions/75738686/bundle-cache-pulling-the-latest-commit-from-master-branch-for-a-private-hosted-g. It is not a big concern for me but it doesn't make sense to access the internet again if it is already cached everything is need in a local directory and we specifically say bundler to install stuff only from that cached directory. (with the --local switch flag.
2. Even though the bundler trying to access internet and (in my case) there is no credential available, and its switches back to offline mode and is trying to install from the cache but it is not finding the correct folder name in the locally cached directory.
   Please let me know if you need further details or any help to fix the issue. I am glad to help. I am no a ruby developer otherwise I would try to submit a patch myself.
   This is very important fix from a security point of view. Nobody will be able to cleanly create a docker container without having a token or credentials in its layer when using https with token or credentials. It is not a problem for the git based access to the repositories as docker can safely access with --mount=ssh way.

[kimoKimo77]

عليك التقدم والانجاز

[deivid-rodriguez]

I think I saw the issue here, you're setting the path setting to the cache folder. This is incorrect, vendor/cache is the cache folder, not the installation folder.

I've been testing several things and I believe everything is working correctly in the latest version of Bundler, but if you're still having issues, feel free to reopen providing reproduction steps!

Thanks!