Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jruby-complete 1.7.14 appears to be missing some files #1927

Closed
rtyler opened this Issue Aug 28, 2014 · 12 comments

Comments

Projects
None yet
3 participants
@rtyler
Copy link

commented Aug 28, 2014

More like jruby-incomplete, AMIRITE?!

[17:22:56] tyler:~ $ (cd /tmp/gradle7468542493140908025projectDir && /usr/local/openjdk8/bin/java -Dfile.encoding=UTF-8 -cp /tmp/gradle7468542493140908025projectDir/userHome/caches/modules-2/files-2.1/org.jruby/jruby-complete/1.7.14/ad70c18834a143afa6686ebcda27351ebfef7385/jruby-complete-1.7.14.jar org.jruby.Main -S gem install /tmp/gradle7468542493140908025projectDir/userHome/caches/modules-2/files-2.1/rubygems/credit_card_validator/1.1.0/4a3be270d8d28ca49b30ccd2c8ac0ceb2d17a986/credit_card_validator-1.1.0.gem --ignore-dependencies --install-dir=/usr/home/tyler/source/github/jruby-gradle/jruby-gradle-plugin/build/tmp/integrationTest/root/jreeis/vendor -N)
LoadError: library `java' could not be loaded: java.lang.RuntimeException: native error calling lstat: No such file or directory /tmp/gradle7468542493140908025projectDir/userHome/caches/modules-2/files-2.1/org.jruby/jruby-complete/1.7.14/ad70c18834a143afa6686ebcda27351ebfef7385/jruby-complete-1.7.14.jar!/META-INF/jruby.home/lib/ruby/1.9/site_ruby/jruby/java.rb
``

I've seen this also when trying use RVM to build and install jruby-1.7.14:

23:26 < rtyler> [ERROR] Failed to execute goal
io.tesla.polyglot:tesla-polyglot-maven-plugin:0.1.1:execute (install_gems) on project
jruby-lib: Execution install_gems of goal
io.tesla.polyglot:tesla-polyglot-maven-plugin:0.1.1:execute failed: (LoadError)
library `java' could not be loaded: java.lang.RuntimeException: native error calling
lstat: No such file or directory
/home/tyler/scratch/tools/apache-maven-3.1.1/boot/plexus-classworlds-2.5.1.jar/jruby/java.rb -> [Help
1]

@mkristian

This comment has been minimized.

Copy link
Member

commented Aug 28, 2014

OK now I know how to reproduce #1914 without an android ;)

no it is not missing, there is some odd loading going on. the file jruby/java.rb at jruby-complete.jar!/jruby/java.rb

@enebo

This comment has been minimized.

Copy link
Member

commented Aug 28, 2014

This looks bad and likely a result of load service refactoring. Can someone whittle this down a little bit more? 1.7.15 might be in the not so distant future?

@enebo

This comment has been minimized.

Copy link
Member

commented Aug 28, 2014

@rtyler I think if you add -Djruby.debug.loadService=true and run with 1.7.13 and 1.7.14 then hopefully we will see something that 1.7.14 can not load that 1.7.1.3 did.

@rtyler

This comment has been minimized.

Copy link
Author

commented Aug 28, 2014

With loadService=true and JRuby 1.7.15-SNAPSHOT;

(cd /tmp/gradle7816039793277592345projectDir && /usr/local/openjdk8/bin/java -Dfile.encoding=UTF-8 -Djruby.debug.loadService=true -cp /tmp/gradle7816039793277592345projectDir/userHome/caches/modules-2/files-2.1
/org.jruby/jruby-complete/1.7.15-SNAPSHOT/4c13865dfc1cbfa57ed2d6ace77cff4e10636927/jruby-complete-1.7.15-SNAPSHOT.jar org.jruby.Main -S gem install /tmp/gradle7816039793277592345projectDir/userHome/caches/modules-2/files-2.1/rubygems/credit_card_validator/1.1.0/4a3be270d8d28ca49b30ccd2c8ac0ceb2d17a986/credit_card_validator-1.1.0.gem --ignore-dependencies --install-dir=/usr/home/tyler/source/github/jruby-gradle/jruby-gradle-plugin/build/tmp/integrationTest/root/jreeis/vendor -N)                                                                                                                                                    2014-08-28T16:02:44.332-07:00: LoadService: trying builtinLib: jruby.rb
2014-08-28T16:02:44.409-07:00: LoadService: found builtinLib: jruby.rb
2014-08-28T16:02:44.410-07:00: LoadService: trying builtinLib: java.rb
2014-08-28T16:02:44.410-07:00: LoadService: found builtinLib: java.rb
2014-08-28T16:02:44.668-07:00: LoadService: trying builtinLib: jruby/java.rb
2014-08-28T16:02:44.668-07:00: LoadService: trying fileResource: file:/tmp/gradle7816039793277592345projectDir/userHome/caches/modules-2/files-2.1/org.jruby/jruby-complete/1.7.15-SNAPSHOT/4c13865dfc1cbfa57ed2d6
ace77cff4e10636927/jruby-complete-1.7.15-SNAPSHOT.jar!/META-INF/jruby.home/lib/ruby/1.9/site_ruby/jruby/java.rb                                                                                                   LoadError: library `java' could not be loaded: java.lang.RuntimeException: native error calling lstat: No such file or directory /tmp/gradle7816039793277592345projectDir/userHome/caches/modules-2/files-2.1/org.
jruby/jruby-complete/1.7.15-SNAPSHOT/4c13865dfc1cbfa57ed2d6ace77cff4e10636927/jruby-complete-1.7.15-SNAPSHOT.jar!/META-INF/jruby.home/lib/ruby/1.9/site_ruby/jruby/java.rb                                        [16:02:45] tyler:jruby-gradle-plugin git:(master*) $
[16:02:55] tyler:jruby-gradle-plugin git:(master*) $ java -version
openjdk version "1.8.0_05"
OpenJDK Runtime Environment (build 1.8.0_05-b13)
OpenJDK 64-Bit Server VM (build 25.5-b02, mixed mode)
[16:03:12] tyler:jruby-gradle-plugin git:(master*) $ 

@mkristian

This comment has been minimized.

Copy link
Member

commented Aug 28, 2014

@ratnikov the log looks like the file:/tmp/gradle7816039793277592345projectDir/userHome/caches/modules-2/files-2.1/org.jruby/jruby-complete/1.7.15-SNAPSHOT/4c13865dfc1cbfa57ed2d6
ace77cff4e10636927/jruby-complete-1.7.15-SNAPSHOT.jar!/META-INF/jruby.home/lib/ruby/1.9/site_ruby/jruby/java.rb

gets treated as RegularFileResource and Posix.lstat fails since it is not a file on the file-system

@mkristian

This comment has been minimized.

Copy link
Member

commented Aug 29, 2014

diff --git a/core/src/main/java/org/jruby/util/RegularFileResource.java b/core/src/main/java/org/jruby/util/RegularFileResource.java
index 277240b..33b95ff 100644
--- a/core/src/main/java/org/jruby/util/RegularFileResource.java
+++ b/core/src/main/java/org/jruby/util/RegularFileResource.java
@@ -113,6 +113,8 @@ class RegularFileResource implements FileResource {

     @Override
     public FileStat lstat(POSIX posix) {
+        // TODO quick fix for https://github.com/jruby/jruby/issues/1927
+        if (file.getName().contains("!/")) return new JarFileStat(this);
         return posix.lstat(file.getAbsolutePath());
     }

did not help. the biggest problem right now is: not be able to reproduce the issue locally

@enebo

This comment has been minimized.

Copy link
Member

commented Aug 29, 2014

Hmm can zip files have unfollowable soft links in them? In any case, if it messes up the resource type for lstat it really will not go well for it when it tries to read the file :)

So somehow a jar resource is getting creates as a regularfileresource...This is something to go on, but it would be great if I could fail locally. @rtyler what do I need to do to get to the command-line you are showing in this issue. The 'gradew clean check' did not trigger this.

@enebo

This comment has been minimized.

Copy link
Member

commented Aug 29, 2014

Brainstorming a bit. Textually and code-path-wise it is impossible for this to not be a JarResource (more specifically a JarFileResource). So assuming this is true (which I have pretty good confdence at this point), then createJarResource has to be returning null and then it follows through and assumes it is really a file out on the filesystem somewhere.

With that assumption out of the way it can really only be one of these reasons:

  1. JarResource.jarCache has not entry for this jar file. Either the path used to key off of which jar is different or somehow multiple runtimes are loaded and somehow cross checking the wrong static.
  2. getIndex from the jarcache is not returning the proper entry. Either processing of the cache is broken and some entry is not there or the path we are supplying to the index is mismatched.

I guess the good thing is even without a local reproduction we can put prints in at these important places and see where things drive off the road....

@rtyler

This comment has been minimized.

Copy link
Author

commented Aug 29, 2014

Alright, running the jruby-gradle-plugin with 1.7.14 and 1.7.15 on Ubuntu 14.04 with JDK 8 works:

java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)

The steps I've been following are to run:

  1. git clone git://github.com/jruby-gradle/jruby-gradle-plugin.git
  2. cd jruby-gradle-plugin
  3. git checkout 908516ca00cc1a82bc8726a37c60d9141739c6b4
  4. ./gradlew clean check
@rtyler

This comment has been minimized.

Copy link
Author

commented Aug 29, 2014

It looks like this is definitely reproducible on FreeBSD. I used ami-e8b4d5d8 which is in us-west-2 on AWS for testing this.

Specific pre-requisite steps on FreeBSD, after the AMI comes online:

  1. pkg update
  2. pkg install -y git bash openjdk8 gcc (gcc is apparently needed for some of the gradle native extension code to work)

Then you should be able to run the steps below to see the reproduction of the failure:

  1. git clone git://github.com/jruby-gradle/jruby-gradle-plugin.git

  2. cd jruby-gradle-plugin

  3. git checkout 908516ca00cc1a82bc8726a37c60d9141739c6b4

  4. ./gradlew clean check

    The output is as follows below:

$ ./gradlew clean check --stacktrace
:clean
:compileJava UP-TO-DATE
:compileGroovy
:processResources
:classes
:compileTestJava UP-TO-DATE
:compileTestGroovy
Download http://jcenter.bintray.com/junit/junit/4.12-beta-1/junit-4.12-beta-1.pom
Download http://jcenter.bintray.com/org/spockframework/spock-core/0.7-groovy-2.0/spock-core-0.7-groovy-2.0.pom
Download http://jcenter.bintray.com/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.pom
Download http://jcenter.bintray.com/org/hamcrest/hamcrest-parent/1.3/hamcrest-parent-1.3.pom
Download http://jcenter.bintray.com/junit/junit-dep/4.10/junit-dep-4.10.pom
Download http://jcenter.bintray.com/junit/junit/4.12-beta-1/junit-4.12-beta-1.jar
Download http://jcenter.bintray.com/org/spockframework/spock-core/0.7-groovy-2.0/spock-core-0.7-groovy-2.0.jar
Download http://jcenter.bintray.com/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar
Download http://jcenter.bintray.com/junit/junit-dep/4.10/junit-dep-4.10.jar
:processTestResources
:testClasses
:test

com.github.jrubygradle.JRubyPluginTest > jrubyPluginSetsRepositoriesCorrectly STANDARD_OUT
    18:31:01.563 [Test worker] INFO  o.g.c.project.BuildScriptProcessor - Evaluating root project 'test' using empty build file.

com.github.jrubygradle.JRubyPluginTest > defaultRubyGemRepository STANDARD_OUT
    18:31:02.805 [Test worker] INFO  o.g.c.project.BuildScriptProcessor - Evaluating root project 'test' using empty build file.

com.github.jrubygradle.JRubyExecSpec > Changing the jruby version on a JRubyExec task STANDARD_OUT
    18:31:05.280 [Test worker] INFO  o.g.c.project.BuildScriptProcessor - Evaluating root project 'test' using empty build file.

com.github.jrubygradle.JRubyPrepareGemsSpec > Configuring JRubyPrepareGems STANDARD_OUT
    18:31:06.830 [Test worker] INFO  o.g.c.project.BuildScriptProcessor - Evaluating root project 'test' using empty build file.
:compileIntegrationTestJava UP-TO-DATE
:compileIntegrationTestGroovy
:processIntegrationTestResources
:integrationTestClasses
:jar
:integrationTest

com.github.jrubygradle.JRubyExecExtensionIntegrationSpec > Run a script with minimum parameters FAILED
    org.gradle.process.internal.ExecException at JRubyExecExtensionIntegrationSpec.groovy:39

com.github.jrubygradle.JRubyExecExtensionIntegrationSpec > Run a script containing a conditional FAILED
    org.gradle.process.internal.ExecException at JRubyExecExtensionIntegrationSpec.groovy:54
    org.gradle.process.internal.ExecException at JRubyExecExtensionIntegrationSpec.groovy:54

com.github.jrubygradle.JRubyExecExtensionIntegrationSpec > Running a script that requires a gem, a separate jRuby and a separate configuration FAILED
    org.gradle.process.internal.ExecException at JRubyExecExtensionIntegrationSpec.groovy:84

com.github.jrubygradle.JRubyExecIntegrationSpec > Running a Hello World script FAILED
    org.gradle.process.internal.ExecException at JRubyExecIntegrationSpec.groovy:61

com.github.jrubygradle.JRubyPrepareGemsIntegrationSpec > Unpack our gem as normal FAILED
    org.gradle.process.internal.ExecException at JRubyPrepareGemsIntegrationSpec.groovy:52

com.github.jrubygradle.JRubyPrepareGemsIntegrationSpec > Unpack our gem, but without transitives FAILED
    org.gradle.process.internal.ExecException at JRubyPrepareGemsIntegrationSpec.groovy:73

com.github.jrubygradle.JRubyPrepareGemsIntegrationSpec > Check that default 'jrubyPrepareGems' uses the correct directories FAILED
    org.gradle.process.internal.ExecException at JRubyPrepareGemsIntegrationSpec.groovy:91

11 tests completed, 7 failed, 2 skipped
:integrationTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':integrationTest'.
> There were failing tests. See the report at: file:///usr/home/ec2-user/jruby-gradle-plugin/build/reports/tests/index.html

* Try:
Run with --info or --debug option to get more log output.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':integrationTest'.
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
    at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:296)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:86)
    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
    at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
    at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)
    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:54)
    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:148)
    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:105)
    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:85)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:81)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:39)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:29)
    at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:50)
    at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
    at org.gradle.launcher.Main.doAction(Main.java:33)
    at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
    at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
    at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
    at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
    at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:56)
Caused by: org.gradle.api.GradleException: There were failing tests. See the report at: file:///usr/home/ec2-user/jruby-gradle-plugin/build/reports/tests/index.html
    at org.gradle.api.tasks.testing.Test.handleTestFailures(Test.java:1111)
    at org.gradle.api.tasks.testing.Test.executeTests(Test.java:542)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:570)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:553)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
    ... 47 more


BUILD FAILED

Total time: 5 mins 26.737 secs
$ 

I've tarballed up the build/reports directory and uploaded it here which should allow you to browse the HTML test report with the full standard error and output

@enebo

This comment has been minimized.

Copy link
Member

commented Aug 29, 2014

This appears to be a problem with the FreeBSD box not being able to load JNR-POSIX. Not sure if this is from recent changes to jnr-posix or whether this is a configuration of FreeBSD jnr-posix is having issues with:

$ ./bin/jruby -Xnative.verbose=true -e 1
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.NoClassDefFoundError: Could not initialize class jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder
    at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:223)
    at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:36)
    at jnr.posix.FreeBSDPOSIX.<init>(FreeBSDPOSIX.java:41)
    at jnr.posix.POSIXFactory.loadFreeBSDPOSIX(POSIXFactory.java:107)
    at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:72)
    at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:38)
    at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:35)
    at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:31)
    at jnr.posix.LazyPOSIX.isatty(LazyPOSIX.java:201)
    at org.jruby.Main.handleRaiseException(Main.java:509)
    at org.jruby.Main.main(Main.java:202)
LoadError: library `java' could not be loaded: java.lang.RuntimeException: native error calling lstat: No such file or directory /usr/home/ec2-user/jruby/lib/ruby/1.9/site_ruby/jruby/java.rb

@enebo enebo added this to the JRuby 1.7.15 milestone Sep 2, 2014

@enebo

This comment has been minimized.

Copy link
Member

commented Sep 2, 2014

Fixed by commits 38ea3c8 0163e4b

@enebo enebo closed this Sep 2, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.