loading (default) gems varies between jruby 1.7.x versions #1566

Closed
mkristian opened this Issue Mar 17, 2014 · 6 comments

Projects

None yet

3 participants

@mkristian
Member

the Gem.path changed from 1.7.5 to append the jruby internal path at the end. which matches MRI (as far I can tell)

from 1.7.5 onwards the internal "default" gems get preference over the gems installed somewhere else. 1.7.12-SNAPSHOT/9000.dev does even ignore those installed altogether and only loads the internal ones. same did 1.7.9,1.7.8,1.7.6

1.7.11 sees the jruby-openssl gem and use it.

but jruby-openssl installed outside of jruby is version 0.9.4 and the default gem is version 0.9.3. for bouncy-castle-java gem it 1.7.5 or newer always loads the internal default gems.

https://gist.github.com/mkristian/9599309

however the given Gem.path indicates that the given GEM_PATH is first in the path and I would expect those gems get preference to the gems further behind in the Gem.path - as 1.7.4 does it.

if I add a gem 'jruby-openssl' in the beginning of the test script the situation is slightly different.

altogether I would expect with or without the extra gem 'jruby-openssl' that the externally installed gems have preference as the Gem.path indicates (as 1.7.4 did it) and those krypt gems are used then they are loaded gems, as 1.7.11 does show but 1.7.12-SNAPSHOT not anymore.

it is not clear what is the right behaviour should be.

@mkristian mkristian added this to the JRuby 1.7.12 milestone Mar 17, 2014
@mkristian mkristian added the openssl label Mar 17, 2014
@enebo enebo modified the milestone: JRuby 1.7.13, JRuby 1.7.12 Apr 15, 2014
@jayjlawrence

I was impacted by this issue as well. Identical matter to https://groups.google.com/forum/#!msg/jruby-users/cx-1gODKPWM/zKDREFZqfcYJ

I'm a bit perplexed because it seems like bouncy castle was loaded, but exactly as Michael Pitman did, adding an explicit reference to the BC 147 jars solved the issue. This occurred under jruby 1.7.8-1.7.12 for me.

I'm going to watch this issue and if a fix is proposed then I'll try to confirm as well.

@mkristian
Member

I am still not able to reproduce the issue from Michael.

what exactly do you mean with 'explicit reference to the BC 147 jars' ? I
guess I know but to be on the safe side.

could you provide me more info on your setup - maybe your setup allows me
to reproduce it on my side !!!!!

@jayjlawrence

I too made a stand-alone app that failed to reproduce the issue. In my case this is a rails app and the reference to the encryption code is quite deep (of course) in my app. The app works fine in my dev (mac) environment but when it gets bundled via warbler and deployed as a war on windows it experiences this issue. Who knows what's triggering the effect!

I've added at the top of my application.rb, right after rails is loaded

if defined?(Java)
bc_path=File.expand_path('../../java/BouncyCastle/', FILE)
load File.join(bc_path, 'bcpkix-jdk15on-147.jar')
load File.join(bc_path, 'bcprov-jdk15on-147.jar')
end

and then placed copies of the jar files into my build as shown above. This clears the issue.

While troubleshooting I used the example code to exercise the issue and receive the same exception "Wrong algorithm: DESede or TripleDES required".

org.jruby.ext.openssl.x509store.PEMInputOutput.readPrivateKey(java.io.StringReader.new(File.read(file_name).to_java()), pwd.to_java.to_char_array)

Basically I put this in my bundled war file code right before I'd expect to use a private key so that I could confirm that it was this method call that croaks. As you know the initialize method in PKeyRSA.java just tosses these exceptions as it works through a list of different key loading strategies. (That is a costly decision for developers using this code - as the source of the failure is a total mystery and to make things more perplexing my public key loads - now I see why)

I did review the LOAD_PATH and CLASSPATH values and I did see in my CLASSPATH contains:
"jar:file:/E:/PW-AMP/PW_UAT_KIO/webapps/ROOT/WEB-INF/lib/jruby-stdlib-complete-1.7.12.jar!/META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-1.47.jar"
"jar:file:/E:/PW-AMP/PW_UAT_KIO/webapps/ROOT/WEB-INF/lib/jruby-stdlib-complete-1.7.12.jar!/META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-1.47.jar",

Confirming the jars are in the jruby jars:
$ unzip -l jruby-stdlib-complete-1.7.12.jar |grep bcp
515071 04/15/2014 09:36 META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-1.47.jar
515071 04/15/2014 09:36 META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-147.jar
1997327 04/15/2014 09:36 META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-1.47.jar
1997327 04/15/2014 09:36 META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-147.jar�
����������������������������������������������
The only thing I was unsure of how to do is get a list of loaded jars. I could put that right before the issue with and without the temporary fix and see what is the delta. Please advise what I should check and I'll do that while the issue is still hot.

@mkristian
Member

which JDK are you using on windows ? oracle jkd-8 ?

the issue could be similar to
#1543 (comment)

where the problem is how the jar got loaded and gets not verified.

as you say it is difficult to see the real cause of the problem since a lot
of exception get swallowed.

one more thing to test would be to build the ext/openssl gem from jruby git
master branch

cd ext/openssl
mvn package

and use the pkg/jruby-openssl-0.9.5.gem for you tests. here Kares took some
care to at least log intermediate exceptions when -Djruby.openssl.debug=true

thanx for all the help . . . .

On Tue, May 6, 2014 at 2:23 PM, jayjlawrence notifications@github.comwrote:

I too made a stand-alone app that failed to reproduce the issue. In my
case this is a rails app and the reference to the encryption code is quite
deep (of course) in my app. The app works fine in my dev (mac) environment
but when it gets bundled via warbler and deployed as a war on windows it
experiences this issue. Who knows what's triggering the effect!

I've added at the top of my application.rb, right after rails is loaded
TEMPORARY FIX

if defined?(Java)
bc_path=File.expand_path('../../java/BouncyCastle/', FILE)
load File.join(bc_path, 'bcpkix-jdk15on-147.jar')
load File.join(bc_path, 'bcprov-jdk15on-147.jar')
end

and then placed copies of the jar files into my build as shown above. This
clears the issue.

While troubleshooting I used the example code to exercise the issue and
receive the same exception "Wrong algorithm: DESede or TripleDES required".

org.jruby.ext.openssl.x509store.PEMInputOutput.readPrivateKey(java.io.StringReader.new(File.read(file_name).to_java()),
pwd.to_java.to_char_array)

Basically I put this in my bundled war file code right before I'd expect
to use a private key so that I could confirm that it was this method call
that croaks. As you know the initialize method in PKeyRSA.java just tosses
these exceptions as it works through a list of different key loading
strategies. (That is a costly decision for developers using this code - as
the source of the failure is a total mystery and to make things more
perplexing my public key loads - now I see why)

I did review the LOAD_PATH and CLASSPATH values and I did see in my
CLASSPATH contains:

"jar:file:/E:/PW-AMP/PW_UAT_KIO/webapps/ROOT/WEB-INF/lib/jruby-stdlib-complete-1.7.12.jar!/META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-1.47.jar"

"jar:file:/E:/PW-AMP/PW_UAT_KIO/webapps/ROOT/WEB-INF/lib/jruby-stdlib-complete-1.7.12.jar!/META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-1.47.jar",

Confirming the jars are in the jruby jars:
$ unzip -l jruby-stdlib-complete-1.7.12.jar |grep bcp
515071 04/15/2014 09:36
META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-1.47.jar
515071 04/15/2014 09:36
META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-147.jar
1997327 04/15/2014 09:36
META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-1.47.jar
1997327 04/15/2014 09:36
META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-147.jar

The only thing I was unsure of how to do is get a list of loaded jars. I
could put that right before the issue with and without the temporary fix
and see what is the delta. Please advise what I should check and I'll do
that while the issue is still hot.


Reply to this email directly or view it on GitHubhttps://github.com/jruby/jruby/issues/1566#issuecomment-42300910
.

@jayjlawrence

Reading through the problem it seems that this could be related. What tipped me off is the that the Ciphers were not initialized when this problem occurs - and so the proposed cause seems plausible.

I'm running Oracle 1.7.0_45-b15 on Windows. On Mac Oracle 1.7.0_25-b15

Thanks for the openssl build process ... so if I just load the openssl 0.9.5 into my current app that's running JRuby 1.7.12 I will be ok, or should I build 1.7.13-snapshot?

Unfortunately the builds don't work for me at this time....I've got the head of the master branch checked out and ext/openssl give this error:

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/jaylawrence/Projects/jruby/ext/openssl/src/main/java/org/jruby/ext/openssl/BN.java:[47,1] class, interface, or enum expected
[ERROR] /Users/jaylawrence/Projects/jruby/ext/openssl/src/main/java/org/jruby/ext/openssl/BN.java:[49,1] class, interface, or enum expected
[ERROR] /Users/jaylawrence/Projects/jruby/ext/openssl/src/main/java/org/jruby/ext/openssl/BN.java:[51,1] class, interface, or enum expected
[INFO] 3 errors

and the jruby package over all fails with:
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] JRuby
[INFO] JRuby Core
[INFO] JRuby Lib Setup
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building JRuby 9000.dev-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jruby-parent ---
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building JRuby Core 9000.dev-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jruby-core ---
[INFO]
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties (properties) @ jruby-core ---
[WARNING] Ignoring missing properties file: /Users/jaylawrence/Projects/jruby/core/../build.properties
[INFO]
[INFO] --- buildnumber-maven-plugin:1.2:create (jruby-revision) @ jruby-core ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd /Users/jaylawrence/Projects/jruby/core && git rev-parse --verify --short=7 HEAD
[INFO] Working directory: /Users/jaylawrence/Projects/jruby/core
[INFO] Storing buildNumber: d3ed74e at timestamp: 1399388881389
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd /Users/jaylawrence/Projects/jruby/core && git rev-parse --verify --short=7 HEAD
[INFO] Working directory: /Users/jaylawrence/Projects/jruby/core
[INFO] Storing buildScmBranch: UNKNOWN
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jruby-core ---
[INFO] Using 'utf-8' encoding to copy filtered resources.
[INFO] Copying 38 resources
[INFO] Copying 1 resource
[INFO] Copying 1 resource to /Users/jaylawrence/Projects/jruby/core/src/main/java
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (anno) @ jruby-core ---
[INFO] Compiling 7 source files to /Users/jaylawrence/Projects/jruby/core/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] javac: invalid target release: 1.7
Usage: javac
use -help for a list of possible options

[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] JRuby ............................................. SUCCESS [0.472s]
[INFO] JRuby Core ........................................ FAILURE [2.286s]
[INFO] JRuby Lib Setup ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.991s
[INFO] Finished at: Tue May 06 11:08:02 EDT 2014
[INFO] Final Memory: 9M/81M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (anno) on project jruby-core: Compilation failure
[ERROR] javac: invalid target release: 1.7
[ERROR] Usage: javac
[ERROR] use -help for a list of possible options
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :jruby-core

@mkristian
Member

sorry - I did not update my repo so I did not see the error. just pushed
the fix for the compilation error on ext/openssl

the jruby itself you do not need. and jruby-1.7.12 is OK to use. it is
meant as gem which will be (hopefully) released soon.

@mkristian mkristian added a commit that referenced this issue Jun 17, 2014
@mkristian mkristian generate gemspecs from gem and NOT use the ones installed by rubygems…
… in specifications - fixes #1566
7556a3a
@mkristian mkristian closed this in 1d03c28 Jun 23, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment