Presence of gemfile does not necessarily imply that bundler is being used #454

Closed
p opened this Issue Sep 8, 2011 · 26 comments

Comments

Projects
None yet
8 participants

p commented Sep 8, 2011

Many ruby projects nowadays support or make use of bundler in some fashion. I don't use bundler and don't have it installed. Usually this is does not present a problem: after putting the bundler require in a begin-rescue block, I found that tests can still be run and packages created. Today however I discovered that yajl-ruby cannot be tested without bundler, due to rspec performing a bundle exec: https://gist.github.com/dbbac9db30cce38f5016

Presence of a gemfile does not mean the user running the code is using bundler, or has it at all.

Owner

dchelimsky commented Sep 8, 2011

I'm assuming you're talking about the assumption made in the rake task. You can get around it with the skip_bundler option on the rake task [1]:

RSpec::Core::RakeTask.new(:spec) do |t|
  t.skip_bundler = true
end

[1] http://rdoc.info/gems/rspec-core/RSpec/Core/RakeTask:skip_bundler

I'll leave this open until it's documented at http://relishapp.com/rspec as well.

If you're not talking about the rake task, please tell me how you're invoking rspec.

p commented Sep 9, 2011

I am invoking the rake task to test yajl-ruby. I'm sure skip_bundler would work, however, I am not yajl-ruby maintainer and therefore I am not at liberty to make this change in yajl-ruby.

I believe a possible solution would be to check an environment variable in https://github.com/rspec/rspec-core/blob/master/lib/rspec/core/rake_task.rb#L159. Then I can set this environment variable to tell rspec that I don't have bundler even though I may be testing a project which includes a gemfile.

Owner

dchelimsky commented Sep 9, 2011

Where is the Gemfile? From where are you running rake and where is the yajl-ruby code relative to where you are running rake?

Owner

dchelimsky commented Sep 9, 2011

OK - I see yajl-ruby has a Gemfile but it only references the gemspec, and is not referenced in its rake task. I'm guessing the author likes to use bundler to install gems, but not to manage them at runtime.

When Bundler is invoked, it sets a BUNDLE_GEMFILE env variable. I think it would be reasonable to check if that is set. I tried that locally with the yajl-ruby code and it does work. I'm going to to ahead and make that change, but I need to get some feedback from other users to make sure this doesn't break their environments before releasing it.

dchelimsky added a commit that referenced this issue Sep 9, 2011

Owner

dchelimsky commented Sep 9, 2011

I pushed a fix to a bundler-raketask branch. Going to let it sit for a few days for feedback before merging to master.

raldred commented Sep 9, 2011

Seems like the proposed fix would resolve this.

Owner

dchelimsky commented Sep 9, 2011

@raldred - thx.

For anybody commenting here, I'm interested in both +1s and concerns. My instinct is this should not screw anybody up, but I want to know if you think it would.

We ran into a similar issue a while back where Passenger assumed the presence of a Gemfile meant we were using bundler in production, when we weren't yet. We worked around it by just starting to use bundler :)

I think a environment variable is the right approach but the name BUNDLE_GEMFILE is odd to me. Not sure I'm reading this write, but if we want rspec to not call bundle exec we set BUNDLE_GEMFILE to empty string?

  •    ENV["BUNDLE_GEMFILE"] if ENV["BUNDLE_GEMFILE"] unless ENV["BUNDLE_GEMFILE"] == ""
    

The name sort of implies it is specifying a Gemfile path or something. Is there some additional behavior implied?

Thanks for posting to the rspec forum for feedback!

Owner

dchelimsky commented Sep 11, 2011

Bundler sets BUNDLE_GEMFILE for its own internal use (it's documented on http://gembundler.com/man/bundle-exec.1.html and discussed in various email threads on the ruby-bundler google group). I'd rather piggy-back on that variable than introduce a new one.

The assumption is that you would only set this variable yourself if you were using Bundler, and Bundler could only set it if it were activated, so its absence/presence is a fair target for deciding whether to shell out to Bundler or not.

re: posting for feedback, it's my pleasure! I'd much rather get feedback now than after its released :)

wonderful. Absolutely the right idea to use something that bundler already sets. Thanks for pointing out the doc!

p commented Sep 11, 2011

The fix seems to work for me, thanks.

(Took me a while to realize that:

/usr/bin/ruby1.8: No such file or directory -- rspec (LoadError)

refers to binary PATH, not $:.)

Contributor

coreyhaines commented Sep 12, 2011

This would be nice, as it should negate the need to create a custom spec task just to keep bundler exec from getting added to the command. I use bundler to manage my gem installation, but do not use bundle exec for any commands. So, I have a custom spec task that sets skip_bundler => true

I think I missed something, though.

BUNDLE_GEM is set if bundler is activated (via manually or bundle exec?)
Rspec adds 'bundle exec' if it is set
'bundle exec' would set BUNDLE_GEM

So this assumes that Rake will start up bundler to get BUNDLE_GEM set for people who want bundle exec?

Owner

myronmarston commented Sep 12, 2011

I had the same question as Corey Haines--won't this only work if you run bundle exec rake spec? If you just run rake spec (which I know is not recommended but works fine if you only have one version of rake installed and you setup bundler in you spec_helper or whatever), then I don't think this approach will work.

Also, there's a weird issue with bundler and ruby warnings: when I run my VCR specs with bundle exec and -w, it silences the warnings. I have to use skip_bundler to allow the warnings to be printed (which I want--I'm trying to keep VCR warning-free these days).

So...I think it'd be nice to keep a skip_bundler option that can be used to manually disable bundler.

Owner

dchelimsky commented Sep 12, 2011

@coreyhaines and @myronmarston - with this new scheme, bundler would only be invoked if:

  1. you have BUNDLE_GEMFILE set in your environment
  2. you have Bundler.setup in your code (e.g. in your spec_helper.rb)
  3. you have Bundler.setup in your Rakefile (or other that might be loaded e.g. lib/tasks/whatever.rake in a rails app)
  4. you invoke rake with bundle exec

In any of those cases, I think it's fair to assume you want Bundler, and I think it's fair to assume the opposite as well, no?

re: bundle exec and -w, that sounds like a bug, and should be dealt with separately, no?

Contributor

coreyhaines commented Sep 12, 2011

@dchelimsky

Definitely sounds reasonable. Thanks for the clarification.

Owner

myronmarston commented Sep 12, 2011

  1. you have Bundler.setup in your code (e.g. in your spec_helper.rb)

I'm not understanding how this works. I would think it would only work if spec_helper.rb is required before the rake task runs--otherwise BUNDLE_GEMFILE won't already be set when the RSpec rake task is figuring out the command to run, right?

re: bundle exec and -w, that sounds like a bug, and should be dealt with separately, no?

My point wasn't that we should keep skip_bundler specifically because of this issue, but rather, that there are occasional issues with bundler (like all software, it has bugs) and the skip_bundler option is a nice-to-have way to work around these occasional issues. The -w issue is just an example of one case where I found skip_bundler useful. I imagine there will be other times in the future where it may prove useful.

I think I'm OK with using BUNDLE_GEMFILE to detect the default setting of whether or not bundle exec should be used, but I also like the flexibility of keeping the skip_bundler option. One change does not necessitate the other.

Owner

dchelimsky commented Sep 12, 2011

The end goal is to either have Bundler manage the runtime environment of the subshell or not. bundle exec is one way. Bundle.setup in code is another. They generally have the same result provided you want to run the latest rspec installed in the current environment. So while Bundle.setup in spec_helper won't cause bundle exec to prefix the command, it will still modify the load path for you.

I'm OK with leaving skip_bundler as an "only do this if something is broken" sort of workaround.

Owner

myronmarston commented Sep 12, 2011

Thanks for the explanation. I think this approach makes a lot of sense.

And thanks for being willing to keep skip_bundler around :).

Owner

dchelimsky commented Sep 12, 2011

Can anybody else think of a reason not to do this?

One small drawback ... before I started running bundle install --binstubs on my projects I would sometimes forget to prefix a command with bundle exec (it's much easier to just type rspec and pretty easy to type bin/rspec).

When I forget to prefix bundle exec and I have newer versions of gem dependencies installed in my system I can sometimes get confusing bugs when the newer gems are loaded.

This will not be a drawback for me anymore because I now add the bin stubs.

It would probably be good to let the Bundler developers know about your usage so both groups are clearly on the same page about future expectations for BUNDLER_GEM... though I just looked through the Bundler code and it looks pretty engrained.

Owner

dchelimsky commented Oct 9, 2011

@indirect, @wycats, @carllerche - any reason rspec-core should not depend on BUNDLE_GEMFILE as proposed above?

@dchelimsky dchelimsky closed this in d731a27 Oct 9, 2011

Owner

dchelimsky commented Oct 9, 2011

@indirect, @wycats, @carllerche - I'm planning to release this in an rc, but am willing to roll back if you guys convince me that's the right way to go.

indirect commented Oct 9, 2011

If BUNDLE_GEMFILE is already set by Bundler, then it's superfluous to invoke bundle exec. Anytime Bundler has set up the ENV, it has also set up RUBYOPT, and therefore any ruby process will automatically be set up to use the bundled gems. So while BUNDLE_GEMFILE is a great way to tell if you're already inside a bundled environment, but if it's there you already don't need to do anything.

The only exception I can think of is some of the more esoteric use-cases that involve setting BUNDLE_GEMFILE manually before Bundler has been invoked, to ensure that Bundler will find the correct Gemfile when it eventually loads.

On Oct 9, 2011, at 6:31 AM, David Chelimskyreply@reply.github.com wrote:

@indirect, @wycats, @carllerche - I'm planning to release this in an rc, but am willing to roll back if you guys convince me that's the right way to go.

Reply to this email directly or view it on GitHub:
#454 (comment)

Owner

dchelimsky commented Oct 9, 2011

@indirect - just so I'm clear: if rake is invoked with bundle exec, or if Bundler.setup appears in the Rakefile, when the rspec rake task shells out using rspec, it will operate as though the command was bundle exec rspec. Correct?

Owner

dchelimsky commented Oct 9, 2011

@indirect re: the edge case you mention, that should still work the same way, no? i.e. if I run BUNDLE_GEMFILE=special.gemfile rake spec, the subshell will get BUNDLE_GEMFILE, and bundler will also have set RUBYOPT so the subshell runs using bundler with the gemfile prescribed in the parent shell. Right?

indirect commented Oct 9, 2011

Yes on both counts. Here's an example of how it works (and why rspec can, afaict, just default to skip_bundler = true at all times): https://gist.github.com/8f59bedcd961c3da1c28

As that example hopefully shows, subshells inherit the bundle, so rspec doesn't need to include bundle exec. As an upside, that will speed up each rspec invocation, too. :)

dchelimsky added a commit that referenced this issue Oct 9, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment