...ngs work just fine with the vendored jars inside the gems. fix issue jruby/jruby#1435 and others maybe.
currently I just comment out the respective code since there was a reason to include that code. but the problem is that such a copying of jars will divert for example from
$ jruby -S rails server
drastically. I haven't look into how executable jars are packed, i.e. whether they unpack the vendored jars as well.
I think actually the opposite approach would be actually better: to separate the jruby-classloader from its parent classloader, similar what you can configure with servlet-classloader to load its classes first to avoid clashed with the underlying parent-classloader. maven also separates the plugin from each other with the trick.
on the other hand I would not know how to extract the jars from jruby-stdlib-complete or jruby-complete or whatever jruby jars you have configured.
further the current setup duplicates the jars from jruby-jars.gem on fresh rails setup (rails new foobar; cd foobar; warble)
from my personal experience I used a simple maven setup to pack a warfile which always worked on jetty.
jars from jruby.jar itself do not get copied over to WEB-INF/lib and …
…things work just fine with the vendored jars inside the gems. fix issue jruby#1453 and others maybe
hmm, this kind of surprises me. i'm going to take some time and try to better understand the issue. but i should have this in soon. thanks!
yea, if this works without the move_jars_to_webinf_lib we should definitely get rid of it. I'll merge this and kill the commented sections. thanks again.
killed the move_jars_to_webinf_lib method for #241
interesting, I wonder if it would still work with those that do not expand the .war archive (rare but still).
might have some related issue as well at jruby/activerecord-jdbc-adapter#536 ... but it ain't confirmed yet
the problem before the merge was that warbler copied the jopenssl.jar to WEB-INF/lib and that needs bouncycastle jars. but BC were in jruby-classloader which has the webapp-classloader as parent, i.e. jopenssl.jar could not see the bouncy-castle jars
your problem might be related - I do understand to little what 'arjdbc/jdbc/java' does - but with that patch applied the whole warble warfile is much closer to jruby -S rails server which could solve the activerecord issue as well.
jruby -S rails server
Thanks for the clarification Christian, that call just triggers a require '.../adapter.jar' which contains the native extension classes compiled ... in this particular case it seems to be not getting loaded up correctly. Which is strange but might have something to do with how the application is warbled.
My concern (even as we ignore the issue mentioned) is if this would work with all the JRuby versions supported esp. if you consider not all application servers explode the .war archive - thus JRuby needs to load the jars (missing from WEB-INF/lib) using jar: references ... but maybe I assumed wrong and this does not effect such cases ?!
exactly - see it will probably work in most cases but probably not when say on Tomcat unpackWARs=false
... not an uncommon thing with other servers as well.
With your change in place our warfile lib directory went from:
I don't think this was the intention was it? I don't think this was the intention was it? Unless I'm not understanding? Reason for my interest is extremely rare
uninitialized constant Krypt::ASN1::BOOLEAN
errors upon starting tomcat. So rare that I can go almost half a day of starting and stopping tomcat and only see it once. So I was thinking an issue with warbler in copying out the jar files within the jar files.
@malletto thanks ... would you please maybe setup a minimal rails app where this case be reproduced ...
@malletto I do not get it completely. so before the change you saw never such
ans now you see it occasionally ? is that about what you are saying ?
@kares I wanted to had one more thing to our discussion. just look at a normal java servlet project they have to pack jruby-complete.jar (or the modulare maven artifact) into WEB-INF/lib and aspect it to work, there you do not have a 'warbler' which starts to move jar file from gems or move from within jruby-complete.jar. so if things are not working it is a loading problem with jruby itself.
@malletto looking at your jar list:
are the same jars as
Krypt::ASN1::BOOLEAN is part of ruby-stdlib-complete-1.7.10.jar or in gems-gems-jruby-jars-1.7.10-lib-jruby-stdlib-complete-1.7.10.jar
so it is really hard to see a difference between the two setups regarding Krypt::ASN1::BOOLEAN
would be nice to see the output of jruby when the system property
or the obvious question from my claim that there is no difference: did you try the same start+stop tomcat debug cycle with the old setup as well ?
@mkristian very true, but historically Warbler did the moving pretty "smart" ... this likely avoided some of the loading issues JRuby had (and might still have) e.g. the edge case with non-exploded .war(s), also some web-servers might pop up having issues. maybe what would probably the best - to move things forward while keeping an option for this "old but still useful" behaviour. so I would like to propose a Warbler configuration option something like config.move_jars_to_webinf_lib (by default false) ... ideas?
maybe ask the user to report an issue if they NEED to use the old behaviour ;)
@jkutner what do you think about the extra config ?
@kares for jruby-9000.dev the copy thing will have extra side-effects since the classloader there looks first to loaded jar via require before looking into the parent classloader. i.e. you can have a servlet using bouncy-castle-1.50 and jruby can safely use bouncy-castle-1.47 which is bundled right now.
Yea, I like the extra config. I'll create a ticket and I can do that work myself.
Have a look at #243 and see if that's what you had in mind.
@mkristian that BC loading does sounds pretty scary ... I'm not sure why and if I wanted that (no "option" to "override" the BC jar). I actually did some BC internals hacking with JRuby - might open a pull, would love to hear your input as well ... maybe it turns out all stupid but I'd like to avoid leakage with multiple runtimes and it seems quite necessary to "avoid" registering the provider.
@kares my BC loading was just example. any servlet container allows to reverse the classloader from parent container to separate your WEB-INF/lib from whatever is load in the parent classloader. next time I use jline.jar as example or xerces from nokogiri ;)
reagarding the registering the security provider in jriuby-openssl: I saw the code once and had the feeling that is not the job of jruby to register such a provider - just my thoughts. happy to give another pair of eyes on BC loading code of yours.
@kares I setup a barebones rails app. My gemfile contains:
# Package dependency manager
# Ruby on Rails
gem 'rails', '3.2.17'
platforms :jruby do
gem 'activerecord-jdbcsqlite3-adapter', '1.3.3'
gem 'jruby-rack', '188.8.131.52'
gem 'jruby-jars', '1.7.10'
# Not needed in production, but needed on our build server
# Must be >= 1.4.0 to pick up fixes #144 and #180
gem 'warbler', '1.4.0'
With the modified warbler my lib directory looks like:
drwxr-xr-x 2 68 Mar 3 09:16 assets
-rwxr-xr-x 1 13243641 Mar 3 09:16 jruby-core-complete-1.7.10.jar
-rwxr-xr-x 1 243757 Mar 3 09:16 jruby-rack-184.108.40.206.jar
-rwxr-xr-x 1 8639499 Mar 3 09:16 jruby-stdlib-complete-1.7.10.jar
drwxr-xr-x 2 68 Mar 3 09:16 tasks
With the original unmodified warbler it looks like:
drwxr-xr-x 2 68 Mar 3 09:15 assets
-rwxr-xr-x 1 134889 Mar 3 09:15 gems-gems-activerecord-jdbc-adapter-1.3.6-lib-arjdbc-jdbc-adapter_java.jar
-rwxr-xr-x 1 3201128 Mar 3 09:15 gems-gems-jdbc-sqlite3-220.127.116.11-lib-sqlite-jdbc-3.7.2.jar
-rwxr-xr-x 1 13243641 Mar 3 09:15 gems-gems-jruby-jars-1.7.10-lib-jruby-core-complete-1.7.10.jar
-rwxr-xr-x 1 8639499 Mar 3 09:15 gems-gems-jruby-jars-1.7.10-lib-jruby-stdlib-complete-1.7.10.jar
-rwxr-xr-x 1 243757 Mar 3 09:15 gems-gems-jruby-rack-18.104.22.168-lib-jruby-rack-22.214.171.124.jar
-rwxr-xr-x 1 36523 Mar 3 09:15 gems-gems-json-1.8.1-java-lib-json-ext-generator.jar
-rwxr-xr-x 1 29097 Mar 3 09:15 gems-gems-json-1.8.1-java-lib-json-ext-parser.jar
-rwxr-xr-x 1 16846 Mar 3 09:15 gems-gems-warbler-1.4.0-lib-warbler_jar.jar
-rwxr-xr-x 1 13243641 Mar 3 09:15 jruby-core-complete-1.7.10.jar
-rwxr-xr-x 1 243757 Mar 3 09:15 jruby-rack-126.96.36.199.jar
-rwxr-xr-x 1 8639499 Mar 3 09:15 jruby-stdlib-complete-1.7.10.jar
drwxr-xr-x 2 68 Mar 3 09:15 tasks
Created the war with:
RAILS_ENV=production bundle exec warble executable war
Running the 2 war files (since they are executable) with java -jar they both start. So maybe this is just a cosmetic thing? Not sure. This doesn't get me any closer to figuring out our extremely random (maybe 4% of the time) Krypt::ASN1::BOOLEAN error problem that has been bothering us for months though, and I have no idea if its related.
@malletto I just downloaded your testapp from jruby/jruby#1119 (comment)
and the first thing I could see were some warning:
WARNING: while creating new bindings for class org.jruby.ext.krypt.asn1.RubyTemplate$RubyAsn1Template,
and you have 8 jruby runtimes. so the first thought is that all the problems you see are concurrency issue with krypt gem ! especially since they are rare and thus depending on execution timings.
is the strong reason for having 8 runtimes and not only 1 ?
there is krypt-0.0.2.rc1 gem now, could you add that to your Gemfile and see if that works better. that new gem is already part of jruby-1.7.12-SNAPSHOT and on jruby-head
@mkristian Yeah we aren't running in threadsafe mode, yet. So we need to have a pool of runtimes. I'll look at the new rc1 of krypt and see if I can add that to my test app.
Not sure why we would have concurrency issues if there are 8 separate runtimes? Aren't they all self contained?
@malletto I have one more thing for you since it is really a startup problem:
with jruby-rack you can set the context-param
that should work around your problem (not solving the underlying issue with the krypt extension)
@mkristian I'll check that out, thanks. What does it do though? I looked in the documentation:
When using runtime pooling, this flag indicates that the pool should be created serially in the foreground rather than spawning (background) threads, it's by default off (set to false). For environments where creating threads is not permitted.
Any negatives to setting to true?