1.16.0 is intercepting binstub arguments and throwing 'unknown switches' error

[johnmcdowall]

I've just created a new Rails project with bundler 1.16.0, and when trying to invoke the webpack binstub with command line arguments for the underlying executable, Bundler is throwing the message 'unknown switches':

$ ./bin/webpack --colors --progress
Unknown switches '--colors, --progress'

This did not / does not happen with 1.15.x Bundler.

I would expect that binstub command line arguments should still be proxied through to the underlying executable? I can find no documentation or issues that would suggest this chnge in behaviour was wanted.

[johnmcdowall]

It seems this change: https://github.com/bundler/bundler/blob/01fe509b27b317dd4d640beb9140ab31331ea349/lib/bundler/templates/Executable#L11-L12

Is responsible for the change in behaviour? Is this needed?

[johnmcdowall]

Confirmed, removing those two lines from my binstubs restores the correct behaviour of proxying command line arguments along to the underlying executable.

Removing them also seems to have no detrimental effect?

[colby-swandale]

I was able to reproduce this, thanks for the report!

[segiddins]

This shouldn't be happening, since we only old the "real" bundler exe when that file is invoked directly: https://github.com/bundler/bundler/blob/892a637f2d9bf18b9898ea31d4f5bf8d1e8cb8e4/lib/bundler/templates/Executable.bundler#L103-L105

[mattbrictson]

I just ran into this problem as well.

This happens to me in Rails, because out of the box rails new generates the following bin/bundle binstub:

#!/usr/bin/env ruby
ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)
load Gem.bin_path('bundler', 'bundle')

When I generate a binstub manually, the new binstub doesn't work:

$ bundle binstub unicorn
$ bin/unicorn -E development
Unknown switches '-E'

The fix is to regenerate bundler's binstub:

$ bundle binstub bundler --force
$ bin/unicorn -E development
I, [2017-11-11T14:59:40.867225 #93138]  INFO -- : listening on addr=0.0.0.0:8080 fd=9
I, [2017-11-11T14:59:40.867334 #93138]  INFO -- : worker=0 spawning...
I, [2017-11-11T14:59:40.868225 #93138]  INFO -- : master process ready
I, [2017-11-11T14:59:40.868991 #93151]  INFO -- : worker=0 spawned pid=93151
I, [2017-11-11T14:59:40.869413 #93151]  INFO -- : Refreshing Gem list
I, [2017-11-11T14:59:42.193645 #93151]  INFO -- : worker=0 ready

I suspect that this will commonly be a problem even for non-Rails projects. E.g., if you started the project before bundler 1.16.0 was released, you have the pre-1.16.0 bin/bundle binstub. If you upgrade bundler to 1.16.0 and then generate a new binstub, you will encounter this error.

[guilleiguaran]

Also reported in Rails issue tracker: rails/rails#31193

Looks like many existing Rails users are going to experience this problem.

[indirect]

@mattbrictson thanks for the heads up! Just so things are clear, this is not an upgrade issue for Bundler. Bundler 1.15 refused to create a binstub for itself:

$ bundle _1.15.4_ binstubs bundler
Resolving dependencies...
Sorry, Bundler can only be run via Rubygems.

Bundler 1.16 started to support a bundle binstub as a first step towards letting applications quickly load the correct version of Bundler for that application.

I'm not sure how long Rails has been creating a bin/bundle, but it's definitely not a Bundler binstub. To resolve this, we might need to start throwing an informative exception if bin/bundle is the wrong kind of binstub.

[adamof]

I can confirm that this is a problem with other executables as well. For example sidekiq:

$ gem install bundler
Fetching: bundler-1.16.0.gem (100%)
Successfully installed bundler-1.16.0
1 gem installed
$ bundle version
Bundler version 1.16.0 (2017-10-31 commit 10f20fa33)
$ bundle binstubs sidekiq
$ bin/sidekiq -C config/sidekiq.yml
Unknown switches '-C'

[mattbrictson]

@indirect Understood that the bin/bundle that Rails has been distributing is not a Bundler binstub. The reason I brought this up as an upgrade issue for Bundler is that as @johnmcdowall mentioned, binstubs generated by Bundler 1.16 now contain this code:

bundle_binstub = File.expand_path("../bundle", __FILE__) 
load(bundle_binstub) if File.file?(bundle_binstub)

So before, the binstubs generated by Bundler 1.15 and below would be oblivious to the bin/bundle provided by Rails. But now the binstubs generated by Bundler 1.16 explicitly look for bin/bundle and load it, which leads to the problem.

Perhaps, as you said, Bundler needs to identify this and fail with a helpful error message. For example, it could work like this:

$ bundle binstubs sidekiq
An outdated bin/bundle executable was detected and this command cannot continue.
Run `bundle binstubs bundler --force` to correct this.

[indirect]

@mattbrictson yep, that's close to what I was imagining. I think we'll probably need to do something like this, to ensure that a bad bin/bundle fails in a predictable way:

def bundle_stub?(path)
    return false unless File.file?(path)
    return true if File.read(path, 150) =~ /This file was generated by Bundler/
    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
end

bundle_binstub = File.expand_path("../bundle", __FILE__)
load(bundle_binstub) if bundle_stub?(bundle_binstub)

[mattbrictson]

@indirect that looks like a good solution 👍

[paddor]

I'm having this issue with Bundler version 2.0.1 (2019-01-04 commit d7ad2192f).
The switches --name-only, --without-group, and --only-group are unknown. Only --paths works.

which bundle just prints /home/p/.gem/ruby/2.6.2/bin/bundle, which is where I installed Ruby using ruby-install. The header in the file itself says it was generated by RubyGems.

Any ideas?

[deivid-rodriguez]

You mean with bundle list, right? The problem is that the help that documents those options is for the new list command, that it's yet to be released. That has been fixed in #7072, and will be released with the next version.

[paddor]

Yes, I meant bundle list. Is there another way to just list the gems in a particular group?