Problems with Bundler 1.16.0

[andy-twosticks]

It seems as if Bundler 1.16.0 might have broken Warbler?

I get "Unable to detect bundler spec under '/usr/share/tomcat/.rvm/gems/jruby-9.1.8.0/gems/bundler-1.16.0'' and its sub-dirs".

This is a new problem, and if I force bundler < 1.16.0, it all works again. I'm using jRuby 9.1.13.0.

Here is some output (from a Jenkins log):

12:58:47 Successfully installed bundler-1.16.0
12:58:47 1 gem installed
12:58:47 + bundle install --without development test --with production

<skipping a lot of output here>

12:59:00 [UAT] $ /usr/share/tomcat/.rvm/gems/jruby-9.1.8.0/bin/bundle exec rake compile:all deploy:UAT
12:59:07 mkdir -p public/jhall/img
12:59:07 mkdir -p public/jhall/css
12:59:07 mkdir -p public/jhall/js
12:59:07 CoffeeScript.compile assets/coffee/common.coffee -> public/jhall/js/common.js
12:59:07 cp assets/other/favicon-cog.ico public/jhall/img/favicon.ico
12:59:07 sass --style compressed assets/sass/jh.sass public/jhall/css/jh.css
12:59:07 cp assets/other/logo.png public/jhall/img/logo.png
12:59:07 mkdir -p public/jhall/other
12:59:07 cp assets/other/remedium_tx_label.xml public/jhall/other/remedium_tx_label.xml
12:59:07 cp assets/other/remedium_patient_label.xml public/jhall/other/remedium_patient_label.xml
12:59:09 CoffeeScript.compile assets/coffee/jh_dymo.coffee -> public/jhall/js/jh_dymo.js
12:59:10 CoffeeScript.compile assets/coffee/onload.coffee -> public/jhall/js/onload.js
12:59:10 CoffeeScript.compile assets/coffee/pillomatic.coffee -> public/jhall/js/pillomatic.js
12:59:11 CoffeeScript.compile assets/coffee/selectotron.coffee -> public/jhall/js/selectotron.js
12:59:12 CoffeeScript.compile assets/coffee/soap_rebate_form.coffee -> public/jhall/js/soap_rebate_form.js
12:59:18 Unable to detect bundler spec under '/usr/share/tomcat/.rvm/gems/jruby-9.1.8.0/gems/bundler-1.16.0'' and its sub-dirs
12:59:18 mv UAT.war swingshift.war
12:59:18 rake aborted!

Here is the rake "deploy" task group, which is running bundler:

namespace :deploy do

  desc "Compile Swingshift to .war & install in Tomcat, this machine"
  task :UAT => :warble do
    cp "swingshift.war", "/var/lib/tomcat/webapps/swingshift.war"
  end

  desc "Compile Swingshift to .war & move file to deploy area for live"
  task :live => :warble do
    cp "swingshift.war", "/home/jonea/deploy/war/swingshift.war"
  end

  task :warble do
    dir = Pathname(__dir__).expand_path.basename.to_s
    `warble`
    mv "#{dir}.war", "swingshift.war"
  end

end # namespace :deploy

Here's my config/warble.rb:

Warbler::Config.new do |config|
  # Application directories to be included in the webapp.
  config.dirs = %w(config core projects public db)

  # Additional files/directories to include, above those in config.dirs
  config.includes = FileList[ 'config.ru',
                              'controllers.rb',
                              'version.rb',
                              'app.rb' ]

  config.features = %w(runnable)

  # Application booter to use, one of :rack, :rails,
  # or :merb (autodetected by default)
  config.webxml.booter = :rack
end

[the-michael-toy]

We are looking at this too. The problem seems to be that bundler 1.16.0 doesn't ship with a bundler.gemspec file, and warbler really wants to find that.

https://github.com/jruby/warbler/blob/master/lib/warbler/traits/bundler.rb#L43-L56

This really HORRIBLE workaround ... after bundle install has our CI builds running again while we figure out what to do about this ...

spec_file = File.join(g.gem_dir, "bundler.gemspec")
if ! File.exist?(spec_file)
  File.write(spec_file, g.to_ruby)
  puts "#{spec_file} written"
else
  puts "#{spec_file} exists"
end

[andy-twosticks]

I don't know if this is a better or worse workaround, but in my Jenkins build script where I had gem install bundler I now have

gem uninstall -a bundler
gem install --version "< 1.16" bundler

(I always build from scratch.)

[HarlemSquirrel]

I added this terrible hack to my local install of warbler after line 44. It downloads and saves a copy of bundler.gemspec from github for the current Bundler version the first time it cannot find it.

# Bundler 1.16 hack for missing bundler gemspec
# Download bundler.gemspec from github to warbler gem local root path
unless full_gem_path.join('bundler.gemspec').exist?
  warbler_gem_path = File.expand_path(File.join(File.dirname(__FILE__), '../../..'))
  full_gem_path = Pathname.new(warbler_gem_path)
  unless full_gem_path.join('bundler.gemspec').exist?
    bundler_min_version = spec.version.to_s.split('.')[0..1].join('-')
    require 'open-uri'
    IO.copy_stream(
      open("https://raw.githubusercontent.com/bundler/bundler/"\
        "#{bundler_min_version}-stable/bundler.gemspec"),
      full_gem_path.join('bundler.gemspec').to_s
    )
  end
end

Here's my full file: https://gist.github.com/HarlemSquirrel/cc4d8f70ac39a382c6863b585beec273

EDIT: This does not seem to yield a usable war file :(

[hsbt]

This issue will fix by bundler-1.16.1 with rubygems/bundler#6195 soon.

[kares]

seems like quite an ugly/old hack - from 89978dd
... maybe someone should attempt removing the .gemspec dependency + cleanup specs, any takers?

[Confusion]

I think the problems I reported below were due to a wonky rvm setup. After removing bundler from /home/user/.rvm/rubies/jruby-9.1.17.0/lib/ruby/gems/shared, reinstalling JRuby and running rvm gemset pristine in the default and project-specific gemsets, warble works fine.

---

I'm still seeing a related problem with bundler-1.16.1 on rvm:

$ rvm current
jruby-9.1.17.0@project

$ which gem
/home/user/.rvm/rubies/jruby-9.1.17.0/bin/gem

$ gem list | grep warbler
warbler (2.0.4)

$ warble
Unable to detect bundler spec under '/home/user/.rvm/rubies/jruby-9.1.17.0/lib/ruby/gems/shared/gems/bundler-1.16.1'' and its sub-dirs

This is sort of correct, in the sense that that directory only contains exe/bundle and nothing else. But shouldn't it be looking in /home/user/.rvm/gems/jruby-9.1.17.0@project/gems/bundler-1.16.1, where bundler.gemspec indeed exists?

---

In lib/warbler/traits/bundler.rb function add_bundler_gems, the bundler spec contains all the files and data from /home/user/.rvm/gems/jruby-9.1.17.0@project/gems/bundler-1.16.1, but spec.full_gem_path refers to the /home/user/.rvm/rubies/jruby-9.1.17.0/lib/ruby/gems/shared/gems/bundler-1.16.1. So it seems bundler is resolving things wrongly/inconsistently? Or is rvm doing something wrong?

[darmbrust]

no idea if this is related, I'm not really a jruby developer. But upgrading from 1.11.2 to 1.16.4 of bunder in our build process (jruby complete / jruby exec plugin in maven) results in completely unusable war files:

``
05-Sep-2018 02:04:48.882 INFO [main] org.apache.catalina.core.ApplicationContext.log An exception happened during JRuby-Rack startup
Could not find rainbow-2.2.2 in any of the sources
--- System
jruby 9.1.8.0 (2.3.1) 2017-03-06 90fc7ab Java HotSpot(TM) 64-Bit Server VM 25.181-b13 on 1.8.0_181-b13 +jit [linux-x86_64]
Time: 2018-09-05 02:04:48 +0000
Server: Apache Tomcat/9.0.10
jruby.home: uri:classloader://META-INF/jruby.home

--- Context Init Parameters:
jruby.max.runtimes = 1
jruby.min.runtimes = 1
public.root = /
rails.env = production

--- Backtrace
Bundler::GemNotFound: Could not find rainbow-2.2.2 in any of the sources
block in materialize at /apps/tomcat/webapps/editor/WEB-INF/gems/gems/bundler-1.16.4/lib/bundler/spec_set.rb:91
map! at org/jruby/RubyArray.java:2518

Gem.dir: /apps/tomcat/webapps/editor/WEB-INF/gems
Gem.path:
/apps/.gem/jruby/2.3.0
uri:classloader:/META-INF/jruby.home/lib/ruby/gems/shared
/apps/tomcat/webapps/editor/WEB-INF/gems
Activated gems:
did_you_mean-1.0.1
bundler-1.16.4

--- Bundler
Bundler.bundle_path: /apps/tomcat/webapps/editor/WEB-INF/gems
Bundler.root: /apps/tomcat/webapps/editor/WEB-INF
Gemfile: /apps/tomcat/webapps/editor/WEB-INF/Gemfile
Settings:
gemfile = /apps/tomcat/webapps/editor/WEB-INF/Gemfile
without = [:development, :test, :assets]

[darmbrust]

I also noted from the dump above, it referenced the path
/apps/.gem/jruby/2.3.0

This is a garbage path, that points to nowhere. This is a war file being deployed in tomcat - the war file exploded path is /apps/tomcat/webapps/editor/

Nothing /apps/.gem/ could be correct - now, I don't know if this has always been wrong, or if this is a symptom of the whatever broke the world in the recent releases...

[mkristian]

@darmbrust rubygems alsways adds $HOME/.gem/jruby/2.3.0 to GEM_PATH and if there is nothing it does not find anything there. so this bit is totally OK

[darmbrust]

I did a bit more isolation - 1.13.7 is the last version of bundler that worked for me. 1.14.6 does not work, nor does anything newer.
I haven't attempted to isolated it to a point release in between those two points....