--path fails under jruby due to rake gem, but succeeds second time #602

Closed
dtropp opened this Issue Aug 24, 2010 · 17 comments

Comments

Projects
None yet
8 participants
@dtropp

dtropp commented Aug 24, 2010

(bundler 1.0.0.rc.6, jruby 1.5.1)

'bundle install' works fine with no parameters to install into the system jruby gems.

But we'd to install with --path to checkin our gems to later use to deploy into production. However, 'bundle install --path dan' fails with the following error.

The error is:
$ bundle install --path dan
Fetching source index for http://rubygems.org/
Installing rake (0.8.7) /home/drp/.rvm/rubies/jruby-1.5.1/lib/ruby/site_ruby/1.8/rubygems/format.rb:37:in from_file_by_path': can't convert nil into String (TypeError) from /home/drp/.rvm/rubies/jruby-1.5.1/lib/ruby/site_ruby/1.8/rubygems/installer.rb:118:ininitialize'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/source.rb:99:in new' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/source.rb:99:ininstall'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/installer.rb:55:in run' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/spec_set.rb:12:ineach'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/spec_set.rb:12:in each' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/installer.rb:44:inrun'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/installer.rb:8:in install' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/cli.rb:217:ininstall'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor/task.rb:21:in run' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor/invocation.rb:118:ininvoke_task'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor.rb:246:in dispatch' from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/lib/bundler/vendor/thor/base.rb:389:instart'
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/bin/bundle:13
from /home/drp/.rvm/gems/jruby-1.5.1/gems/bundler-1.0.0.rc.6/bin/bundle:19:in `load'
from /home/drp/.rvm/gems/jruby-1.5.1/bin/bundle:19

Strangely, this leaves a rake-0.8.7.gem file in dan/jruby/.../cache and running the same command a second time works:
Fetching source index for http://rubygems.org/
Installing rake (0.8.7)
Installing abstract (1.0.0)
Installing activesupport (3.0.0.rc)
Installing builder (2.1.2)
Installing i18n (0.4.1)
Installing activemodel (3.0.0.rc)
Installing erubis (2.6.6)
Installing rack (1.2.1)
Installing rack-mount (0.6.12)
Installing rack-test (0.5.4)
Installing tzinfo (0.3.23)
Installing actionpack (3.0.0.rc)
Installing mime-types (1.16)
Installing polyglot (0.3.1)
Installing treetop (1.4.8)
Installing mail (2.2.5)
Installing actionmailer (3.0.0.rc)
Installing arel (0.4.0)
Installing activerecord (3.0.0.rc)
Installing activerecord-jdbc-adapter (0.9.2)
Installing activerecord-oracle_enhanced-adapter (1.3.0)
Installing activeresource (3.0.0.rc)
Using bundler (1.0.0.rc.5)
Installing culerity (0.2.12)
Installing weakling (0.0.4)
Installing nokogiri (1.4.3.1)
Installing ffi (0.6.3)
Installing json_pure (1.4.6)
Installing rubyzip (0.9.4)
Installing selenium-webdriver (0.0.28)
Installing capybara (0.3.9)
Installing diff-lcs (1.1.2)
Installing trollop (1.16.2)
Installing gherkin (2.1.5)
Installing term-ansicolor (1.0.5)
Installing cucumber (0.8.5)
Installing cucumber-rails (0.3.2)
Installing faker (0.3.1)
Installing sexp_processor (3.0.4)
Installing ruby_parser (2.0.4)
Installing flay (1.4.0)
Installing flog (2.4.0)
Installing machinist (1.0.6)
Installing thor (0.14.0)
Installing railties (3.0.0.rc)
Installing rails (3.0.0.rc)
Installing roodi (2.1.0)
Installing rspec-core (2.0.0.beta.19)
Installing rspec-expectations (2.0.0.beta.19)
Installing rspec-mocks (2.0.0.beta.19)
Installing rspec (2.0.0.beta.19)
Installing webrat (0.7.2.beta.1)
Installing rspec-rails (2.0.0.beta.19)
Your bundle is complete! Use bundle show [gemname] to see where a bundled gem is installed.

Your bundle was installed to dan

Our Gemfile is:
source 'http://rubygems.org'

gem 'rails', '3.0.0.rc'

platforms :jruby do
gem 'activerecord-jdbc-adapter', '0.9.2'
end

platforms :mri do
gem 'ruby-oci8', '2.0.4'
gem 'rake', '0.8.7'
end

gem 'activerecord-oracle_enhanced-adapter', '1.3.0'

group :test do
gem 'rspec', '2.0.0.beta.19'
gem 'rspec-rails', '2.0.0.beta.19'
gem 'machinist', '1.0.6'
gem 'faker', '0.3.1'
gem 'flog', '2.4.0'
gem 'flay', '1.4.0'
gem 'roodi', '2.1.0'
gem 'cucumber', '0.8.5'
gem 'cucumber-rails', '0.3.2'
gem 'capybara', '0.3.9'
end

@redbeard

This comment has been minimized.

Show comment
Hide comment
@redbeard

redbeard Sep 7, 2010

We've encountered the same problem on Bundler version 1.0.0, jruby 1.5.1 (ruby 1.8.7 patchlevel 249) (2010-06-06 f3a3480) (Java HotSpot(TM) Client VM 1.6.0_20) [i386-java]

redbeard commented Sep 7, 2010

We've encountered the same problem on Bundler version 1.0.0, jruby 1.5.1 (ruby 1.8.7 patchlevel 249) (2010-06-06 f3a3480) (Java HotSpot(TM) Client VM 1.6.0_20) [i386-java]

@biilmann

This comment has been minimized.

Show comment
Hide comment
@biilmann

biilmann Sep 9, 2010

Having the same problem with rvm and jruby on a server - when deploying from capistrano the first gem bundler tries to install I get the "/usr/local/jruby/lib/ruby/site_ruby/1.8/rubygems/format.rb:37:in `from_file_by_path': can't convert nil into String (TypeError)" error

biilmann commented Sep 9, 2010

Having the same problem with rvm and jruby on a server - when deploying from capistrano the first gem bundler tries to install I get the "/usr/local/jruby/lib/ruby/site_ruby/1.8/rubygems/format.rb:37:in `from_file_by_path': can't convert nil into String (TypeError)" error

@paulwalker

This comment has been minimized.

Show comment
Hide comment
@paulwalker

paulwalker Sep 10, 2010

Same problem with rvm and jruby 1.5.2 locally.

Same problem with rvm and jruby 1.5.2 locally.

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 17, 2010

This is happening the first time I use Bundler 1.0.0 on JRuby with a custom path anywhere... on our production servers, on the build server, locally... definitely easy to reproduce.

This is happening the first time I use Bundler 1.0.0 on JRuby with a custom path anywhere... on our production servers, on the build server, locally... definitely easy to reproduce.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Sep 17, 2010

Contributor

We'd like to fix this if it can be narrowed down...

Contributor

headius commented Sep 17, 2010

We'd like to fix this if it can be narrowed down...

@BanzaiMan

This comment has been minimized.

Show comment
Hide comment
@BanzaiMan

BanzaiMan Sep 17, 2010

Contributor

I tracked this down to http://github.com/carlhuda/bundler/blob/master/lib/bundler/source.rb#L20

When execution gets to this line, with JRuby, Gem.path is empty, but with MRI, it has the expected value. (Just put a simple p statements around this line to see for yourself.)

The funny thing is that, if you run jruby -rubygems -e 'require "bundler/source"; p Gem.path' straight up, I get the correct value.

Contributor

BanzaiMan commented Sep 17, 2010

I tracked this down to http://github.com/carlhuda/bundler/blob/master/lib/bundler/source.rb#L20

When execution gets to this line, with JRuby, Gem.path is empty, but with MRI, it has the expected value. (Just put a simple p statements around this line to see for yourself.)

The funny thing is that, if you run jruby -rubygems -e 'require "bundler/source"; p Gem.path' straight up, I get the correct value.

@BanzaiMan

This comment has been minimized.

Show comment
Hide comment
@BanzaiMan

BanzaiMan Sep 18, 2010

Contributor

This is quite maddening. Under some circumstances (not exactly clear yet), JRuby fails to calculate Gem.path once Gem.clear_paths is called. In particular, http://github.com/carlhuda/bundler/blob/master/lib/bundler.rb#L256 appears to be such a trigger.

A simple

jruby -rubygems -e 'p Gem.path; Gem.clear_paths; Gem.clear_paths; p Gem.path'

doesn't do it. So I'm not really sure what is causing this behavior.

Contributor

BanzaiMan commented Sep 18, 2010

This is quite maddening. Under some circumstances (not exactly clear yet), JRuby fails to calculate Gem.path once Gem.clear_paths is called. In particular, http://github.com/carlhuda/bundler/blob/master/lib/bundler.rb#L256 appears to be such a trigger.

A simple

jruby -rubygems -e 'p Gem.path; Gem.clear_paths; Gem.clear_paths; p Gem.path'

doesn't do it. So I'm not really sure what is causing this behavior.

@rstawarz

This comment has been minimized.

Show comment
Hide comment
@rstawarz

rstawarz Sep 21, 2010

I have the same issue using jruby 1.5.2 when trying to deploy remotely using capistrano. I was able to bypass the issue by setting:
default_environment["GEM_PATH"] = "/usr/local/jruby/lib/ruby/gems/1.8"
in my deploy.rb file

I have the same issue using jruby 1.5.2 when trying to deploy remotely using capistrano. I was able to bypass the issue by setting:
default_environment["GEM_PATH"] = "/usr/local/jruby/lib/ruby/gems/1.8"
in my deploy.rb file

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 24, 2010

I tracked this down as well and can confirm BanzaiMan's findings regarding Gem.path being empty on JRuby but not on MRI. On the second run Gem.path on JRuby is PATH/jruby/1.8. So wherever that's getting inserted by Bundler into Gem.path, it's failing on JRuby the first time around.

Things start going awry in the first line of Bundler::Source::Rubygems (lib/bundler/source.rb:83):

path = cached_gem(spec)

This method is returning nil and there's no kind of check for it. Oddly enough, a similar method below (Bundler::Source::Rubygems#cache) includes this check:

raise GemNotFound, "Missing gem file '#{spec.full_name}.gem'." unless cached_path

At the very least, the Bundler::Source::Rubygems#install method should probably include a similar check in case cached_gem returns nil.

I tracked this down as well and can confirm BanzaiMan's findings regarding Gem.path being empty on JRuby but not on MRI. On the second run Gem.path on JRuby is PATH/jruby/1.8. So wherever that's getting inserted by Bundler into Gem.path, it's failing on JRuby the first time around.

Things start going awry in the first line of Bundler::Source::Rubygems (lib/bundler/source.rb:83):

path = cached_gem(spec)

This method is returning nil and there's no kind of check for it. Oddly enough, a similar method below (Bundler::Source::Rubygems#cache) includes this check:

raise GemNotFound, "Missing gem file '#{spec.full_name}.gem'." unless cached_path

At the very least, the Bundler::Source::Rubygems#install method should probably include a similar check in case cached_gem returns nil.

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 24, 2010

After a bit more digging:

Bundler is trying to remove the normal Gem.path and replace it with one that points to the custom path you specify with "bundle install PATH"

On MRI, it works. On JRuby, Gem.path is emptied, but the path you give to bundler is never added, at least the first time you run it.

After a bit more digging:

Bundler is trying to remove the normal Gem.path and replace it with one that points to the custom path you specify with "bundle install PATH"

On MRI, it works. On JRuby, Gem.path is emptied, but the path you give to bundler is never added, at least the first time you run it.

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 25, 2010

BanzaiMan, looks like you've identified the key place where things are starting to go wrong in the Bundler.configure_gem_home_and_path method. Somewhere between there and Gem.path things are breaking, but I haven't managed to identify what, where, and why :/

BanzaiMan, looks like you've identified the key place where things are starting to go wrong in the Bundler.configure_gem_home_and_path method. Somewhere between there and Gem.path things are breaking, but I haven't managed to identify what, where, and why :/

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 27, 2010

Okay, finally tracked this one down:

http://gist.github.com/599708

Bundler is setting GEM_HOME to a nonexistent directory. For whatever reason, under MRI this works just fine, but under JRuby it fails.

I think the simple solution here is to mkdir_p GEM_HOME after calling Gem.clear_paths before Gem.path is called again.

Okay, finally tracked this one down:

http://gist.github.com/599708

Bundler is setting GEM_HOME to a nonexistent directory. For whatever reason, under MRI this works just fine, but under JRuby it fails.

I think the simple solution here is to mkdir_p GEM_HOME after calling Gem.clear_paths before Gem.path is called again.

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 27, 2010

Charlie of JRuby tracked this down to a monkeypatch to RubyGems' Gem.set_paths method. He's working on a fix.

Charlie of JRuby tracked this down to a monkeypatch to RubyGems' Gem.set_paths method. He's working on a fix.

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 27, 2010

This patch to JRuby corrects the problem for me:

http://gist.github.com/599754

This patch to JRuby corrects the problem for me:

http://gist.github.com/599754

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Sep 27, 2010

And here's a link to this bug in JRuby's JIRA:

http://jira.codehaus.org/browse/JRUBY-5104

And here's a link to this bug in JRuby's JIRA:

http://jira.codehaus.org/browse/JRUBY-5104

@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

This comment has been minimized.

Show comment
Hide comment
@tarcieri

tarcieri Oct 2, 2010

Create the GEM_HOME directory before setting it

When Bundler sets GEM_HOME before creating the path in question, JRuby's monkeypatch to Gem.set_paths ignores the directory because it doesn't exist.

There is a JRuby bug (and patch) at http://jira.codehaus.org/browse/JRUBY-5104, but it won't be included in JRuby until 1.5.4 at the earliest.

Closed by 1c05f01, #734

tarcieri commented Oct 2, 2010

Create the GEM_HOME directory before setting it

When Bundler sets GEM_HOME before creating the path in question, JRuby's monkeypatch to Gem.set_paths ignores the directory because it doesn't exist.

There is a JRuby bug (and patch) at http://jira.codehaus.org/browse/JRUBY-5104, but it won't be included in JRuby until 1.5.4 at the earliest.

Closed by 1c05f01, #734

indirect added a commit that referenced this issue Apr 8, 2011

Rescue from mkdir_p errors in configure_gem_home_and_path
The mkdir_p was originally added for JRuby (see #602). As of JRuby
1.6, some paths will actually raise an error, so we need to rescue
and ignore those.

joevandyk pushed a commit to joevandyk/bundler that referenced this issue May 16, 2011

Create the GEM_HOME directory before setting it
When Bundler sets GEM_HOME before creating the path in question, JRuby's monkeypatch to Gem.set_paths ignores the directory because it doesn't exist.

There is a JRuby bug (and patch) at http://jira.codehaus.org/browse/JRUBY-5104, but it won't be included in JRuby until 1.5.4 at the earliest.

Closes #602, #734

kiela added a commit to kiela/bundler that referenced this issue Feb 13, 2014

kiela added a commit to kiela/bundler that referenced this issue Feb 13, 2014

kiela added a commit to kiela/bundler that referenced this issue Feb 13, 2014

kiela added a commit to kiela/bundler that referenced this issue Feb 14, 2014

kiela added a commit to kiela/bundler that referenced this issue Feb 16, 2014

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment