--path fails under jruby due to rake gem, but succeeds second time

[dtropp]

(bundler 1.0.0.rc.6, jruby 1.5.1)

'bundle install' works fine with no parameters to install into the system jruby gems.

But we'd to install with --path to checkin our gems to later use to deploy into production. However, 'bundle install --path dan' fails with the following error.

The error is:
$ bundle install --path dan
Fetching source index for http://rubygems.org/
Installing rake (0.8.7) /home/drp/.rvm/rubies/jruby-1.5.1/lib/ruby/site_ruby/1.8/rubygems/format.rb:37:in from_file_by_path': can't convert nil into String (TypeError) from /home/drp/.rvm/rubies/jruby-1.5.1/lib/ruby/site_ruby/1.8/rubygems/installer.rb:118:ininitialize'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/source.rb:99:in new' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/source.rb:99:ininstall'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/installer.rb:55:in run' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/spec_set.rb:12:ineach'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/spec_set.rb:12:in each' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/installer.rb:44:inrun'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/installer.rb:8:in install' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/cli.rb:217:ininstall'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor/task.rb:21:in run' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor/invocation.rb:118:ininvoke_task'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor.rb:246:in dispatch' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor/base.rb:389:instart'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/bin/bundle:13
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/bin/bundle:19:in `load'
from /home/drp/.rvm/gems/jruby-1.5.1/bin/bundle:19

Strangely, this leaves a rake-0.8.7.gem file in dan/jruby/.../cache and running the same command a second time works:
Fetching source index for http://rubygems.org/
Installing rake (0.8.7)
Installing abstract (1.0.0)
Installing activesupport (3.0.0.rc)
Installing builder (2.1.2)
Installing i18n (0.4.1)
Installing activemodel (3.0.0.rc)
Installing erubis (2.6.6)
Installing rack (1.2.1)
Installing rack-mount (0.6.12)
Installing rack-test (0.5.4)
Installing tzinfo (0.3.23)
Installing actionpack (3.0.0.rc)
Installing mime-types (1.16)
Installing polyglot (0.3.1)
Installing treetop (1.4.8)
Installing mail (2.2.5)
Installing actionmailer (3.0.0.rc)
Installing arel (0.4.0)
Installing activerecord (3.0.0.rc)
Installing activerecord-jdbc-adapter (0.9.2)
Installing activerecord-oracle_enhanced-adapter (1.3.0)
Installing activeresource (3.0.0.rc)
Using bundler (1.0.0.rc.5)
Installing culerity (0.2.12)
Installing weakling (0.0.4)
Installing nokogiri (1.4.3.1)
Installing ffi (0.6.3)
Installing json_pure (1.4.6)
Installing rubyzip (0.9.4)
Installing selenium-webdriver (0.0.28)
Installing capybara (0.3.9)
Installing diff-lcs (1.1.2)
Installing trollop (1.16.2)
Installing gherkin (2.1.5)
Installing term-ansicolor (1.0.5)
Installing cucumber (0.8.5)
Installing cucumber-rails (0.3.2)
Installing faker (0.3.1)
Installing sexp_processor (3.0.4)
Installing ruby_parser (2.0.4)
Installing flay (1.4.0)
Installing flog (2.4.0)
Installing machinist (1.0.6)
Installing thor (0.14.0)
Installing railties (3.0.0.rc)
Installing rails (3.0.0.rc)
Installing roodi (2.1.0)
Installing rspec-core (2.0.0.beta.19)
Installing rspec-expectations (2.0.0.beta.19)
Installing rspec-mocks (2.0.0.beta.19)
Installing rspec (2.0.0.beta.19)
Installing webrat (0.7.2.beta.1)
Installing rspec-rails (2.0.0.beta.19)
Your bundle is complete! Use bundle show [gemname] to see where a bundled gem is installed.

Your bundle was installed to dan

Our Gemfile is:
source 'http://rubygems.org'

gem 'rails', '3.0.0.rc'

platforms :jruby do
gem 'activerecord-jdbc-adapter', '0.9.2'
end

platforms :mri do
gem 'ruby-oci8', '2.0.4'
gem 'rake', '0.8.7'
end

gem 'activerecord-oracle_enhanced-adapter', '1.3.0'

group :test do
gem 'rspec', '2.0.0.beta.19'
gem 'rspec-rails', '2.0.0.beta.19'
gem 'machinist', '1.0.6'
gem 'faker', '0.3.1'
gem 'flog', '2.4.0'
gem 'flay', '1.4.0'
gem 'roodi', '2.1.0'
gem 'cucumber', '0.8.5'
gem 'cucumber-rails', '0.3.2'
gem 'capybara', '0.3.9'
end

[redbeard]

We've encountered the same problem on Bundler version 1.0.0, jruby 1.5.1 (ruby 1.8.7 patchlevel 249) (2010-06-06 f3a3480) (Java HotSpot(TM) Client VM 1.6.0_20) [i386-java]

[biilmann]

Having the same problem with rvm and jruby on a server - when deploying from capistrano the first gem bundler tries to install I get the "/usr/local/jruby/lib/ruby/site_ruby/1.8/rubygems/format.rb:37:in `from_file_by_path': can't convert nil into String (TypeError)" error

[paulwalker]

Same problem with rvm and jruby 1.5.2 locally.

[tarcieri]

This is happening the first time I use Bundler 1.0.0 on JRuby with a custom path anywhere... on our production servers, on the build server, locally... definitely easy to reproduce.

[headius]

We'd like to fix this if it can be narrowed down...

[BanzaiMan]

I tracked this down to http://github.com/carlhuda/bundler/blob/master/lib/bundler/source.rb#L20

When execution gets to this line, with JRuby, Gem.path is empty, but with MRI, it has the expected value. (Just put a simple p statements around this line to see for yourself.)

The funny thing is that, if you run jruby -rubygems -e 'require "bundler/source"; p Gem.path' straight up, I get the correct value.

[BanzaiMan]

This is quite maddening. Under some circumstances (not exactly clear yet), JRuby fails to calculate Gem.path once Gem.clear_paths is called. In particular, http://github.com/carlhuda/bundler/blob/master/lib/bundler.rb#L256 appears to be such a trigger.

A simple

jruby -rubygems -e 'p Gem.path; Gem.clear_paths; Gem.clear_paths; p Gem.path'

doesn't do it. So I'm not really sure what is causing this behavior.

[rstawarz]

I have the same issue using jruby 1.5.2 when trying to deploy remotely using capistrano. I was able to bypass the issue by setting:
default_environment["GEM_PATH"] = "/usr/local/jruby/lib/ruby/gems/1.8"
in my deploy.rb file

[tarcieri]

I tracked this down as well and can confirm BanzaiMan's findings regarding Gem.path being empty on JRuby but not on MRI. On the second run Gem.path on JRuby is PATH/jruby/1.8. So wherever that's getting inserted by Bundler into Gem.path, it's failing on JRuby the first time around.

Things start going awry in the first line of Bundler::Source::Rubygems (lib/bundler/source.rb:83):

path = cached_gem(spec)

This method is returning nil and there's no kind of check for it. Oddly enough, a similar method below (Bundler::Source::Rubygems#cache) includes this check:

raise GemNotFound, "Missing gem file '#{spec.full_name}.gem'." unless cached_path

At the very least, the Bundler::Source::Rubygems#install method should probably include a similar check in case cached_gem returns nil.

[tarcieri]

After a bit more digging:

Bundler is trying to remove the normal Gem.path and replace it with one that points to the custom path you specify with "bundle install PATH"

On MRI, it works. On JRuby, Gem.path is emptied, but the path you give to bundler is never added, at least the first time you run it.

[tarcieri]

BanzaiMan, looks like you've identified the key place where things are starting to go wrong in the Bundler.configure_gem_home_and_path method. Somewhere between there and Gem.path things are breaking, but I haven't managed to identify what, where, and why :/

[tarcieri]

Okay, finally tracked this one down:

http://gist.github.com/599708

Bundler is setting GEM_HOME to a nonexistent directory. For whatever reason, under MRI this works just fine, but under JRuby it fails.

I think the simple solution here is to mkdir_p GEM_HOME after calling Gem.clear_paths before Gem.path is called again.

[tarcieri]

Charlie of JRuby tracked this down to a monkeypatch to RubyGems' Gem.set_paths method. He's working on a fix.

[tarcieri]

This patch to JRuby corrects the problem for me:

http://gist.github.com/599754

[tarcieri]

And here's a link to this bug in JRuby's JIRA:

http://jira.codehaus.org/browse/JRUBY-5104

[tarcieri]

This change seems to work around the problem:

http://github.com/tarcieri/bundler/commit/403c1a5113b8ed1738d7894ff8e8d6626ed6c884

[tarcieri]

Create the GEM_HOME directory before setting it

When Bundler sets GEM_HOME before creating the path in question, JRuby's monkeypatch to Gem.set_paths ignores the directory because it doesn't exist.

There is a JRuby bug (and patch) at http://jira.codehaus.org/browse/JRUBY-5104, but it won't be included in JRuby until 1.5.4 at the earliest.

Closed by 1c05f01, #734