Bundler 2.1 has deprecated --deployment, --path, and --without

[jasonivers]

After updating to bundler version 2.1, I see these messages on deployment:

 DEBUG [e0e96756] 	[DEPRECATED] The `--deployment` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set deployment 'true'`, and stop using this flag
 DEBUG [e0e96756] 	[DEPRECATED] The `--path` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set path '/my/app/path/shared/bundle'`, and stop using this flag
 DEBUG [e0e96756] 	[DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'development test'`, and stop using this flag

I don't have any bundler configurations present in my deploy file. The gem should be updated to not use deprecated flags.

[pdobb]

Info on this can be found here: https://github.com/bundler/bundler/blob/d4993be66fa2e76b3ca00ea56a51ecab5478b726/UPGRADING.md#bundler-3

[mattbrictson]

Here's my recommendation:

We add a :bundle_config setting. It expects a hash, and by default it has this value:

{
  deployment: true
}

This will allow Capistrano users to specify arbitrary config, like they used to be able to do by providing arbitrary flags via :bundle_flags.

We change the default :bundle_flags to be just --quiet (i.e. remove --deployment).

Then, before running bundle check or bundle install, we execute the following as many times as necessary:

bundle config --local SETTING VALUE

Where SETTING and VALUE are based on the contents of :bundle_config as well as :bundle_without and :bundle_path.

So in other words, cap production deploy would produce something like this:

bundle config --local deployment true
bundle config --local without "development test"
bundle config --local path /var/www/app/shared/bundle
bundle check
bundle install --quiet --jobs 4

That means we are executing three extra commands (bundle config ...) on every Capistrano deploy. So deploys will be a little slower than before. That's unfortunate but it seems like the most straightforward solution.

Thoughts?

[gurgeous]

looks good to me!

[m0n9oose]

@mattbrictson Bundler maintainers are going to deprecate these settings because they intended to be persistent, app has to keep them between deploys, which makes sense to me. Your suggestion does not solve this. It's just a workaround to silence warnings instead of adopting the changes. I guess it's better to extract this as a separate step like "cap deploy:check:bundler" and run it just one time when the user is setting up deployment.

[mattbrictson]

@m0n9oose thanks for your feedback!

Just to be clear, what you are proposing is that .bundle be included in :linked_dirs and then bundle config --local ... be run once before the first deploy. Thus the config will be shared among each release by virtue of .bundle/config being shared. Is that accurate?

You wrote:

Bundler maintainers are going to deprecate these settings because they intended to be persistent, app has to keep them between deploys, which makes sense to me. Your suggestion does not solve this.

I think we are solving for slightly different use cases.

For me it is not necessarily important to share config across deploys. You may start out deploying your app with without "development test" but later add a qa group and change your config to be without "development test qa". In other words, bundler config may change over time. My recommendation was to run bundle config --local for each Capistrano release. What I assumed and did not explain was that you would not include .bundle in :linked_dirs. Together, this means that each release can have a self-contained bundler config. This seems like it could be desirable.

Your proposal is more efficient if the "separate config per release" use case does not need to be supported. I would be fine with that, but I do worry that it could lead to more user error:

• User could forget to add .bundle to :linked_dirs
• User could forget to run deploy:check:bundler before deploying
• User could change :bundle_config and then forget to run deploy:check:bundler again

Automatically running bundle config --local for each deploy sidesteps these problems, making Capistrano easier to use, albeit slightly less efficient.

Honestly I could go either way on this. I'm looking forward to more feedback from the community.

[m0n9oose]

@mattbrictson

Just to be clear, what you are proposing is that .bundle be included in :linked_dirs and then bundle config --local ... be run once before the first deploy

Not really. Instead of sharing any configuration files, we could extract basic configuration steps (like setting path, gem groups exclusion and others) to the separate cap task and run it just one time on setting up deployment. After that, deployment instance will have its own bundle configuration, and we won't have to repeat the same configuration calls every time.

[synek317]

As a temporary workaround, I use this and it works just great:

set :bundle_flags,      '--quiet' # this unsets --deployment, see details in config_bundler task details
set :bundle_path,       nil
set :bundle_without,    nil

namespace :deploy do
  desc 'Config bundler'
  task :config_bundler do
    on roles(/.*/) do
      execute 'bundle config --local deployment true'
      execute 'bundle config --local without "development test"'
      execute 'bundle config --local path vendor'
    end
  end
end

before 'bundler:install', 'deploy:config_bundler'

[jesperronn]

Suggestion by @synek317 failed for me, so I had to rewrite it:

-execute 'bundle config --local deployment true'
+execute :bundle, :config, '--local deployment true'
-execute 'bundle config --local without "development test"'
+execute :bundle, :config, '--local without "development test"'

original stacktrace:

01:00 deploy:config_bundler
      01 bundle config --local deployment true
      01 bash: bundle: command not found
#<Thread:0x00007f91dfb02548@~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
	9: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
	8: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `run'
	7: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:31:in `instance_exec'
	6: from ~/src/project_x/lib/capistrano/tasks/capistrano_bundler.rake:17:in `block (3 levels) in <top (required)>'
	5: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:80:in `execute'
	4: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `create_command_and_execute'
	3: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `tap'
	2: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/backends/abstract.rb:148:in `block in create_command_and_execute'
	1: from ~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/backends/netssh.rb:170:in `execute_command'
~/.rvm/gems/ruby-2.6.5@project_x/gems/sshkit-1.20.0/lib/sshkit/command.rb:97:in `exit_status=': bundle config --local deployment true exit status: 127 (SSHKit::Command::Failed)
``

[majksner]

@synek317

bundle path should be out of current as it will install gems every time even if there is no changes to the Gemfile.lock.

-execute 'bundle config --local path vendor'
+execute :bundle, :config, "--local path #{shared_path.join('bundle')}"

[mcr]

  02 /usr/local/rvm/bin/rvm 2.6.6 do bundle config --local without "development test"
  02 You are replacing the current local value of without, which is currently "development:test"

Should we use a colon rather than a space?

[doo2-likelion]

As a temporary workaround, I use this and it works just great:

set :bundle_flags,      '--quiet' # this unsets --deployment, see details in config_bundler task details
set :bundle_path,       nil
set :bundle_without,    nil

namespace :deploy do
  desc 'Config bundler'
  task :config_bundler do
    on roles(/.*/) do
      execute 'bundle config --local deployment true'
      execute 'bundle config --local without "development test"'
      execute 'bundle config --local path vendor'
    end
  end
end

before 'bundler:install', 'deploy:config_bundler'

execute 'bundle config --local without "development test"'

should be

execute 'bundle config --local without "development:test"'

search "BUNDLE_WITHOUT" in the page https://bundler.io/v2.0/man/bundle-config.1.html

[dgarwood]

@mattbrictson @m0n9oose

I'm inclined to agree that @mattbrictson 's solution is probably the best. trying to add the config file that bundler would normally use leads to either 1) devs have to remember to run a "special" task when setting up a system, which is undesireable (deploys should not require human interaction to work from as "bare metal" as possible); or 2) the config file interferes with local development, because bundler now does something undesirable in local dev (egL --without "development:test")

It almost seems like we need bundler to support the idea of profiles in the config. Then we could do bundle install --profile deploy vs bundle install --profile development, the file could be committed to the repo, and deploys would just use/read the appropriate config.

the only other way I see to get around a slightly slower deploy is to add env vars directly to the deployment system and remove those settings from capistrano-bundler entirely. Maybe we just need to handle turning settings into env vars for bundler to read?

[mjgiarlo]

I've been using the code @synek317 pasted above in a gem that my team maintains and it works like a charm. Sharing the code here just in case folks are interested: https://github.com/sul-dlss/dlss-capistrano/blob/master/lib/dlss/capistrano/tasks/bundle_config.rake

[patsch]

This thread has been very helpful to me in sorting a new Rails 6 app out for deployment with capistrano. I am using RVM and Gemsets in production and thus couldn't use the dlss-capistrano gem - but the code snippet above from @synek317 worked after I modified it slightly to ensure that the bundle command is properly invoked from the Gemset's bin directory - also requires the capistrano-rvm gem to be installed and your Capfile needs to have this line:
require "capistrano/rvm"

then this snippet in deploy.rb did the job for me:

namespace :deploy do
  desc 'Config bundler'

  task :config_bundler do
    on roles(/.*/) do
      execute :bundle, 'config', '--local deployment', true
      execute :bundle, 'config', '--local', 'without', "development:test"
      execute :bundle , 'config', '--local', 'path', shared_path.join('bundle')
    end
  end
end

before 'bundler:install', 'deploy:config_bundler'

[leesmith]

@patsch That snippet worked great for me too...thanks for sharing! I think it's missing the closing end though.

[mjgiarlo]

@patsch 💬

This thread has been very helpful to me in sorting a new Rails 6 app out for deployment with capistrano. I am using RVM and Gemsets in production and thus couldn't use the dlss-capistrano gem...

Yes, I'm not surprised. I should have been more careful: dlss-capistrano is likely not going to work for (m)any of you since it works how my team needs it to work and was never intended to be used outside of that context. What may be useful in that repository is a pattern to adopt or adapt for your own purposes to kick bundler deprecation warnings to the curb. 🍻

[will-in-wi]

I took a whack at a different approach to this using environment variables: #121

Thoughts?

[tsechingho]

Vote for recommendation of @mattbrictson

Here is my implementation to patch v1.4.0+.
Just put this patch file to lib/capistrano/tasks.

You could move hook to deploy.rb if you just want to run bundler:config callback once manually with linked_dirs setting.

# in deploy.rb
append :linked_dirs, '.bundle'
after 'deploy:check, 'bundler:config'

If you prefer use environment variables for bundler, you could set bundle_env_variables supported after v1.1.2 or set default_env globally.

# in deploy.rb
set :bundle_flags, '--quiet'
set :bundle_path, nil
set :bundle_without, nil
set :default_env, {
  ...
  bundle_path: shared_path.join('bundle').to_s,
  bundle_without: %w[development test].join(':'),
  bundle_deployment: 'true'
}

[salex]

I'm not sure what this patch is doing except for getting rid of the DEPRECATED warnings.

I've been using Capistrano for years, basically out of the box. I've spent the last few months converting my 3 rails apps to rails 6.0.3. These are semi-private apps. All 3 are deployed to a local staging server running debian 10 and to Digital Oceans on an Ubuntu 18.04 server. All running nginx/puma as a systemd service. All Capistrano config files are the same except for :stage, :deploy_to and :user. All have Bundler version 2.1.4. all use same Capfile requires:

require "capistrano/rails/assets"
require "capistrano/rbenv"
require 'capistrano/rails'
require 'capistrano/bundler'
require 'capistrano/puma'
install_plugin Capistrano::Puma  # Default puma tasks
install_plugin Capistrano::Puma::Nginx  # if you want to upload a nginx site template

I added the bundler_config.rake file to my lib/capistrono/tasks for one of the apps and deployed to production. All went fine, no DEPRECATED warning etc. I also deployed to staging, again without problems

My problem, which I don't have the knowledge to explain why, is that if I do a bin/rails c -e production, I'll get the warning:

WARNING: Spring is running in production. To fix this make sure the spring gem is only present in `development` and `test` groups in your Gemfile and make sure you always use `bundle install --without development test` in production

On staging, I don't get the spring warning. Seems to me that Spring or something needs fixen.

The staging and production config/environment files are the same on each app.

EDIT: Think I found out the difference between staging and production. Production has a spring shim in .rbenv/shims but there is not one in staging. Don't know why. It did start out as a DiginalOceans quickstart rails app droplet. I just disabled spring by putting DISABLE_SPRING=1 in my .rbenv-vars file for each app

[jamesw]

I'm not at all clear on what the correct solution to this problem should be but I am experiencing this and would like to know if there is a clear solution?

[jamesw]

Can no one provide a clear, accurate, concise and definitive solution for this?

[jamesw]

mjgiarlo and askrynnikov you have thumbed down my request for clarification. As you clearly have that clarification then please do share so that myself plus the 4 others that have thumbed up my previous request over two weeks ago can also benefit.

[ansonhoyt]

@jamesw many of us, like you, are interested in a solution for this. The issue is still open, shows thoughtful balancing of complexities and has reasonably recent activity. Several workarounds are mentioned to get us by for now. It probably wasn't your intention, but you might have sounded a bit demanding and impatient for someone that is benefiting from a free, open-source library being maintained by volunteers.

Just a guess, but maybe that explains the downvotes?

[mattbrictson]

Hi all, it looks like my original proposal has gotten the most upvotes in terms of comments and like-minded patches.

Full disclosure, my day job does not currently use Capistrano, so I am not in a good position to test a solution in a real-world scenario. I was hoping a member of the community would step in with a PR. The patches are great but there is a lot more work to turn a patch into something that is fully-configurable, update the documentation, and write release notes that explain how existing Capistrano users should upgrade.

Does anyone want to volunteer to take on that work and put up a PR?

If not, I can take a stab at it, but I will need to lean heavily on those that have commented in this thread to test and review my code. Either way it will be probably be a little slow-going before anything is officially released. Thanks for your patience!

[mattbrictson]

Also, does anyone have contacts at Heroku or the Bundler team? It seems that if the Bundler team introduced these deprecations, they may also have strong opinions on how Bundler should be used in a deployment scenario. Likewise Heroku usually sets the direction for the Rails community in how to perform stable and repeatable deployments in the form of their official ruby build pack. I'd be curious if either of these teams have weighed in. I assume users deploying via Heroku are seeing similar deprecation warnings.

[mattbrictson]

It looks like the Heroku team hasn't figured this out either. See: heroku/heroku-buildpack-ruby#943 (comment)

Part of me wonders if Bundler might revisit its decision to introduce these deprecations because it seems to be universally causing confusion for deployment tools; no one seems to know how to proceed.

[ansonhoyt]

I'm not seeing any improved guidance, so Bundler revisiting the decision is an interesting theory.

The heroku issue links to rubygems/bundler#7531 (since closed), where a Bundler maintainer comments (Feb 2) that better documentation and migration guides are needed. In bundler's new repository, I checked history since then and I'm not seeing much for enhanced explanations on the (previously mentioned) upgrading guide or manual. Folks are asking for manual updates on the deprecations too.

[will-in-wi]

I think @mattbrictson's proposal is the right way to move forward, assuming no upstream changes. In reading the various issue threads, my understanding is that the current bundler command is effectively doing two things at once:

• Setting up the bundler config file
• Running bundler command

Upstream is trying to make commands stateless, so we need to make these two operations independent as well, in order to fit with the upstream convention.

I'm happy to help test, I have a couple of sites that still use Capistrano. I might be able to work on this too, but not in the short term.

[mattbrictson]

Thanks @ansonhoyt and @will-in-wi for the additional research and summary. It looks like not much is happening upstream so it is up to this community to find a solution that works. I'll try to put something together over the weekend so that we have a PR to review by this time next week, if not sooner.

[mattbrictson]

Here's a PR to implement the bundle config solution. Please try out the branch and add your feedback to the PR. Thanks!

#122

[mattbrictson]

This should now be fixed as of capistrano-bundler 2.0.0.

Edit: note that capistrano-rails still requires capistrano-bundler ~> 1.1. I'll release a fix for this soon.

[jamesw]

@mattbrictson Thank you and looking forward to the capistrano-rails update

[thbar]

Huge thanks @mattbrictson, thank you for your work on this, appreciated!

[mattbrictson]

A new version of capistrano-rails is out: https://github.com/capistrano/rails/releases/tag/v1.6.0

[iGEL]

Thank you for the discussion. I'm not using capistrano, but this and the linked issues have helped me to understand the problem.

However, I'm not sure whether your solution is the best way. The --deployment flag was deprecated, because it is surprising to many remembered across bundler runs. So if you ran bundle install --deployment, all future runs of bundle install will still use the deployment strategy until you explicitly unset it. You basically now adopted this behavior, possibly rightfully so.

But I chose to not remember the flag by changing bundle install --deployment to BUNDLE_DEPLOYMENT=true bundle install. Added advantage: Still just one command.

[j-manu]

Using environment variables is the path heroku buildpack has taken - heroku/heroku-buildpack-ruby@655e246