Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundler incorrect path resolution handling classloader URI #6045

Closed
fzakaria opened this issue Jan 24, 2020 · 13 comments · Fixed by #6239
Closed

Bundler incorrect path resolution handling classloader URI #6045

fzakaria opened this issue Jan 24, 2020 · 13 comments · Fixed by #6239
Milestone

Comments

@fzakaria
Copy link

@fzakaria fzakaria commented Jan 24, 2020

I have built a JAR using warbler where my Gemfile contains a gem using the :path argument.
The :path points to a sub-directory.

gem 'some-gem', :path => "ruby-gems/some-gem"

The gem is present (actually twice on account of jruby/warbler#465) in the built JAR:

jar -tf my-application.jar

gems/some-gem-1.0/
gems/some-gem-1.0/Gemfile
gems/some-gem-1.0/Gemfile.lock
gems/some-gem-1.0/some-gem.gemspec
gems/some-gem-1.0/lib/
gems/some-gem-1.0/lib/code.rb
gems/some-gem-1.0/test/
...
my-application/ruby-gems/some-gem/
my-application/ruby-gems/some-gem/Gemfile
my-application/ruby-gems/some-gem/Gemfile.lock
my-application/ruby-gems/some-gem/README.md
my-application/ruby-gems/some-gem/Rakefile
...

The JAR however fails to startup with the error: [!] There was an error parsing `Gemfile`: different prefix: "uri:classloader://" and "uri:classloader:/my-application". Bundler cannot continue.

You can find a small reproduction project: https://github.com/fzakaria/warbler-bundler-jruby-failure

bundle exec warble
java -jar warbler-example.jar

[!] There was an error parsing `Gemfile`: different prefix: "uri:classloader://" and "uri:classloader:/warbler-example". Bundler cannot continue.

 #  from uri:classloader:/warbler-example/Gemfile:4
 #  -------------------------------------------
 #  gem 'warbler', '2.0.5'
 >  gem 'jruby-jars', '9.1.17.0' #  ruby '2.3.3', engine: 'jruby', engine_version: '9.1.17.0'
 #  -------------------------------------------

The failure is present only once the following line is included:

require "bundler/setup"

Bundler 2.1.4 is needed to trigger the issue & I believe might be related to the commit rubygems/bundler@c7532ce
which introduced use of relative_path_from

Environment Information

Provide at least:

jruby 9.1.17.0 (2.3.3) 2018-04-20 d8b1ff9 OpenJDK 64-Bit Server VM 25.232-b09 on 1.8.0_232-b09 +jit [darwin-x86_64]

Darwin MacBook-Pro.localdomain 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

Expected Behavior

Run the JAR built with warbler successfully.

Actual Behavior

eval_gemfile at uri:classloader:/gems/bundler-2.1.4/lib/bundler/dsl.rb:53
          evaluate at uri:classloader:/gems/bundler-2.1.4/lib/bundler/dsl.rb:12
             build at uri:classloader:/gems/bundler-2.1.4/lib/bundler/definition.rb:34
        definition at uri:classloader:/gems/bundler-2.1.4/lib/bundler.rb:195
             setup at uri:classloader:/gems/bundler-2.1.4/lib/bundler.rb:143
   block in (root) at uri:classloader:/gems/bundler-2.1.4/lib/bundler/setup.rb:20
        with_level at uri:classloader:/gems/bundler-2.1.4/lib/bundler/ui/shell.rb:136
           silence at uri:classloader:/gems/bundler-2.1.4/lib/bundler/ui/shell.rb:88
            <main> at uri:classloader:/gems/bundler-2.1.4/lib/bundler/setup.rb:20

[!] There was an error parsing `Gemfile`: different prefix: "uri:classloader://" and "uri:classloader:/my-application". Bundler cannot continue.

 #  from uri:classloader:/my-application/Gemfile:13
 #  -------------------------------------------
 #    end
 >  end
@fzakaria
Copy link
Author

@fzakaria fzakaria commented Jan 24, 2020

Did some sleuthing/investigation just to make sure i'm not on a untrodden path:

  1. warbler does allow :path as long as it's relative https://github.com/jruby/warbler/blob/master/lib/warbler/traits/bundler.rb#L64
    There is also a test for it
    https://github.com/jruby/warbler/blob/572652bfd2f1b155085a70f0b69c5d9f40c2add2/spec/warbler/bundler_spec.rb#L101

  2. My hunch is the failing line is https://github.com/rubygems/bundler/blob/826bf309f63cb696f1eca47360bb4f24170d837b/lib/bundler/source/path.rb#L28
    Unfortunately, I can't seem to figure out how to connect the Ruby remote debugger when running from a JAR at the moment.

@fzakaria
Copy link
Author

@fzakaria fzakaria commented Jan 24, 2020

@headius so this problem was being hit when using bundler 2.1.4
Interestingly enough, when I downgraded the bundler to 2.0.2 the problem disappears.

Trying to understand if that's a pure bundler problem or an artifact of JRuby still.

@fzakaria
Copy link
Author

@fzakaria fzakaria commented Jan 24, 2020

Looks like this commit is fairly recent: rubygems/bundler@c7532ce

It introduces relative_path_from and seems to be not playing well with JRuby classloader

@fzakaria
Copy link
Author

@fzakaria fzakaria commented Jan 26, 2020

@headius I have a small reproduction: https://github.com/fzakaria/warbler-bundler-jruby-failure

If you run the example script with simply ruby, it works.

bundle exec ruby bin/warbler-example.rb
>> Starting the JAR!
>> hello world

however, if you build the JAR & run it you get the error from above.

java -jar warbler-example.jar

[!] There was an error parsing `Gemfile`: different prefix: "uri:classloader://" and "uri:classloader:/warbler-example". Bundler cannot continue.

 #  from uri:classloader:/warbler-example/Gemfile:4
 #  -------------------------------------------
 #  gem 'warbler', '2.0.5'
 >  gem 'jruby-jars', '9.1.17.0' #  ruby '2.3.3', engine: 'jruby', engine_version: '9.1.17.0'
 #  -------------------------------------------

The failure is only present once I wrote:

require "bundler/setup"

@fzakaria
Copy link
Author

@fzakaria fzakaria commented Jan 27, 2020

Updated main description with the latest info: bundler version, reproduction project & bundler possible commit

@fzakaria
Copy link
Author

@fzakaria fzakaria commented Jan 27, 2020

@deivid-rodriguez gave a great breakdown of the bug & needed resolution in JRuby in rubygems/bundler#7598 (comment)

@headius
Copy link
Member

@headius headius commented May 22, 2020

Current output from the script by @deivid-rodriguez:

uri:classloader:/foo/bar
uri:classloader:/foo/bar
uri:classloader://foo/bar
uri:classloader://foo/bar

Fixing.

@headius
Copy link
Member

@headius headius commented May 22, 2020

Ok, this is actually a bug in File.expand_path:

puts File.expand_path("uri:classloader://foo/bar")
puts File.expand_path("uri:classloader:/foo/bar");
puts File.expand_path("foo/bar", "uri:classloader://")
puts File.expand_path("foo/bar", "uri:classloader:/")

... has the same output.

@headius
Copy link
Member

@headius headius commented May 22, 2020

This appears to be a bug in the canonicalization portion of File.expand_path, where it unconditionally appends a "/" character to the base path when adding the next child. I have a fix.

@headius
Copy link
Member

@headius headius commented May 22, 2020

FWIW, CRuby appears to handle this case correctly, though it does not recognize the URI as being a root path:

$ rvm ruby-2.6.5 do ruby ../jruby/blah.rb 
/Users/headius/projects/ruby/uri:classloader:/foo/bar
/Users/headius/projects/ruby/uri:classloader:/foo/bar
/Users/headius/projects/ruby/uri:classloader:/foo/bar
/Users/headius/projects/ruby/uri:classloader:/foo/bar

@fzakaria
Copy link
Author

@fzakaria fzakaria commented May 23, 2020

Thank you looking at this -- hopefully the example project was helpful.

I wonder if looking at this gives you any insight into: rubygems/rubygems#3177

That above Bundler bug is quite annoying as it forces me to include test dependencies in our JAR so that they work :(

I haven't confirmed it to be JRuby specific.

@headius
Copy link
Member

@headius headius commented May 23, 2020

I've resolved this issue, assuming the root problem was what @deivid-rodriguez reported.

@fzakaria For this bug the example Pathname script really put it to rest; I was able to step through it and see why the two expand_path patterns did different things.

I have your other issue in hand though and will be looking at it tonight or over the weekend. I'm sure the sample project will be a big help!

@deivid-rodriguez
Copy link
Contributor

@deivid-rodriguez deivid-rodriguez commented May 23, 2020

Thanks for fixing this @headius! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants