Can't bundle bouncy-castle-java under jruby-1.7.10 #238

Closed
bennesher opened this Issue Feb 10, 2014 · 32 comments

Projects

None yet

4 participants

@bennesher

Background: Somewhere between jruby 1.7.6 and jruby 1.7.10 the location of the gemspec file for bouncy-castle-java seems to have moved from lib/ruby/gems/shared/specifications to lib/ruby/gems/shared/specifications/default, and the lib/ruby/gems/shared/gems/bouncy-castle-java-1.5.0147 directory went away. The contents of this gem still also appear in lib/ruby/shared. The command 'gem which bouncy-castle-java' reports lib/ruby/shared/bouncy-castle-java.rb under both jruby 1.7.6 and jruby 1.7.10.

The problem is that warbler is unable to pick up these files, and still looks for the (now non-existent) gem directory lib/ruby/gems/shared/gems/bouncy-castle-java-1.5.0147.

@mkristian
JRuby Team member
@pierrickrouxel

It is necessary to add these gems to Gemfile because the classloader can not access the jars present in the installation of JRuby.

I think a solution could be include all jars that are in $RUBY_PATH/lib/ruby/shared in the war.

With this solution it will be really simplier for a beginner to use warbler.

What do you think about that?

@mkristian
JRuby Team member
@pierrickrouxel

I don't understand the relation between these two issues. Bouncy castle, krypt and openssl are nested in an other jar. A jar in jar isn't loaded by the classloader. Previously we could force the addition these jars adding the respective gems to the Gemfile but now with the default gem no any jar is imported. The warning confirm that.

I don't know if my explanations are clear.

@mkristian
JRuby Team member
@mkristian
JRuby Team member

@pierrickrouxel if you have something which is not working please let us know.

@pierrickrouxel

Thank you for the clarification. In my case, since the warning appears (jruby-1.7.10), when I launch my application I have this error :

ERROR: initialization failed:.org.jruby.rack.RackInitializationException: missing class or uppercase package name (`org.jruby.ext.openssl.OSSLLibrary')

I already had this error in the past but I fixed it adding

gem jruby-openssl, require: false

to the Gemfile.

Doing this, the jruby-openssl jar was directly added to WEB-INF/lib and all work well.

@mkristian
JRuby Team member
@pierrickrouxel

The war is exploded by server (IBM Websphere Application Server 8.5.5). A simple empty rails app causes this problem. I'm not really familiar with jruby-openssl and I don't know if I can reproduce it without rails.

The configuration move_jars_to_webinf_lib doesn't change anything for this problem.

I use JRuby from two years and this problem is currently the last annoying thing I encounter to put my application into production.

@mkristian
JRuby Team member
@pierrickrouxel

I guess you need to build the warbler gem from git head to have that
option

I did that. I follow this issue from some days :)

@mkristian
JRuby Team member
@mkristian
JRuby Team member
@pierrickrouxel

I have only one runtime.
If I put the jruby-openssl directly to /lib it works but it's not a good solution.

@mkristian
JRuby Team member
@pierrickrouxel

Yes I copy bouncy-castle openssl and krypt. I will retry with config.move_jars_to_webinf_lib but my first tests doesn't add these jars to the lib folder.

@mkristian
JRuby Team member
@pierrickrouxel

Yes it's that ! I'm really excited by this fix that I expect for a long time <3

@pierrickrouxel

I tested at new with exploded war and config.move_jars_to_webinf_lib = true
and I get the message :

missing class or uppercase package name (`org.jruby.ext.openssl.OSSLLibrary')

You can easily test that with Websphere Liberty Profile :
https://www.ibmdw.net/wasdev/downloads/

@mkristian
JRuby Team member
@mkristian
JRuby Team member

I played around quite a bit with websphere the last two days and I can start a fresh rails app now:

I add the following gems to Gemfile: krypt, jruby-openssl, bouncy-castle-java
and install them manually before I warble them, so they get packed along all the other gems.

now there is this openssl bug jruby/jruby#1549 which needs a little change in config/application.rb to add require 'openssl' before the Bundler.require statement.

AND it needs the snapshot jruby build from jruby-1_7 branch

@jkutner @kares what can we do about 'adding' those default gems to the war. the gist from the jruby openssl bug does first install those gems manually and then you need to add them to the Gemfile. maybe just a warning with link to an howto ? maybe something else ?

@kares
JRuby Team member

seems quite unfortunate, I think this knowledge about "default jars" should be baked in somehow into Warbler? (I've read the all the comments but still would need to try this out to understand what's going on better) ... it's not clear to me whether this is WebSphere only or happens with say Tomcat as well (there's been previous issues that have work-arounds already in JRuby-Rack ... well even in JRuby itself, so maybe it's fixable with a class-loading trick one level above Warbler) ?

p.s. Also when this is fixed Pierrick will get us a few tickets to Paris and share a romantic weekend with us :) !

@bennesher
@mkristian
JRuby Team member

I do not see the same problem as with websphere on tomcat or jetty - not with warbler from git head.

@benfastmodel is there is a tomcat problem other then the warning with the head of warbler and tomcat, please tell more details.

the thing which works for websphere is to explicitly bundle those default gems which is not trivial since bundler does not install them since they are already installed. gem itself does install them, but you have to pick the right version.

to get the version from jruby itself you have to look out for default specifications on Gem.path

hardcoding is not really an options since for example the krypt version will change with jruby-1.7.12 and bouncy-castle-java and jruby-openssl already has a newer version on rubygems.org

I played around with the classloading a bit but do not understand websphere classloading internals and their special url protocol handler "wsjar" - jruby handles the jar protocol special at various places. but also the LoadService from jruby is not an easy class to understand (at least for me) ;)

@kares
JRuby Team member

notably, from my previous findings here's what WS does it's own way :

  • getResource() seems to always return null for jar "dirs" but works if you ask for a jar "file" entry e.g. getResource("/META-INF/jruby.home") returns 'null` no matter that it's an existing zip dir entry
  • jar entries are prefixed with wsjar: protocol instead of jar: e.g. getResource("/META-INF/jruby.home/bin/jrubyc") returns : wsjar:file:/opt/IBM/WebSphere/AppServer/profiles/AppSrv01/installedApps/debianNode01Cell/rails31_war.ear/rails31.war/WEB-INF/lib/jruby-stdlib-1.6.7.dev.jar!/META-INF/jruby.home/bin/jrubyc

also I do think it auto-wrapped my rails.war (tried some ~ 2 years ago jruby/jruby#123) implicitly into an .ear as seen from the returned path ... not sure if it can be configured.

@mkristian
JRuby Team member

@kares the ear-file part is gone. I made a PR for the issue I had with packed warfiles and websphere: jruby/jruby-rack#170

I did not get packed warfiles to work with jetty or tomcat at all. jboss always unpackes, same does resin.

anyways, one idea to get websphere back to use less 'workarounds' is to actually unpack the jruby-stdlib.jar into WEB-INF/classes which takes away one layer of packaging and it helped for websphere (when the war gets unpacked). still loading happens through the classloader which has still it set of problems. Dir[ 'bc-*.jar' ] which is used to find and load the bouncy-castle jars is not working for various servlet-containers - I will make a PR for jruby-1.7 to fix this.

for more details see: https://github.com/mkristian/war-pack

@kares
JRuby Team member

OK, thanks a lot for researching this ... I guess Warbler might need an "unpacking" feature (some day) if there's demand (not sure about the non-exploded scenario but I think I've seen it working ... maybe with 1.6).

@mkristian
JRuby Team member

@benfastmodel if I read the original issue, then that is not an issue since bouncy-castle-java is part of jruby itself. the gem command does not handle those default gems consistently (gem which shows an non existing directory) is more an issue with rubygems.

there is an issue with some classloaders to find those bouncy castle jars which is fixed for the coming 1.7.12 jruby. the workaround for warbler is to add WEB-INF/init.rb to the war file with
require 'bcpkix-jdk15on-1.47.jar'
require 'bcprov-jdk15on-1.47.jar

@pierrickrouxel the websphere needs more workarounds or different packing of the jruby-stdlib jar - please feel free to open a specific issue for your websphere problem.

I close the ticket since there is fix on jruby. in case jruby-1.7.12 still gives you guys problems please open a new issue.

@mkristian mkristian closed this Mar 12, 2014
@kares
JRuby Team member

@mkristian one side note, is the Dir[File.expand_path('bc*.jar', File.dirname(__FILE__))] code (going to be) refactored from bouncy-castle-java.rb (a simple Dir.entries would do or it's of no help) ?

@mkristian
JRuby Team member
@bennesher

OK, I have confirmed that the solution recently posted in jruby/jruby#1435 does the trick.

I'm mentioning that here in case anyone else is interested.

@ryanfb ryanfb added a commit to ryanfb/sosol that referenced this issue May 15, 2014
@ryanfb ryanfb jruby-openssl gem no longer required as it's included in jruby-1.7.x,…
… also fixes potential warbler issue with bouncy-castle-java gem (see: jruby/jruby#1435 jruby/warbler#238)
fc577d9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment