Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for DragonFly BSD #5768

Closed
tuxillo opened this issue Jun 21, 2019 · 36 comments
Closed

Support for DragonFly BSD #5768

tuxillo opened this issue Jun 21, 2019 · 36 comments
Milestone

Comments

@tuxillo
Copy link
Contributor

@tuxillo tuxillo commented Jun 21, 2019

Hi,

No JRuby available for DragonFly BSD (www.dragonflybsd.org) and I'd like to know what would be the steps to have that OS included in the binary distribution.

We, at least, took what we thought the initial requirements, see: jnr/jnr-posix#127, jnr/jnr-ffi#184, jnr/jffi#64

Now, I cloned jruby's git repo and built distribution with this diff:

diff --git a/core/pom.rb b/core/pom.rb
index 5292aa1767..bf3be2f325 100644
--- a/core/pom.rb
+++ b/core/pom.rb
@@ -43,9 +43,9 @@ project 'JRuby Core' do
   jar 'com.github.jnr:jnr-netdb:1.1.6', :exclusions => ['com.github.jnr:jnr-ffi']
   jar 'com.github.jnr:jnr-enxio:0.19', :exclusions => ['com.github.jnr:jnr-ffi']
   jar 'com.github.jnr:jnr-unixsocket:0.20', :exclusions => ['com.github.jnr:jnr-ffi']
-  jar 'com.github.jnr:jnr-posix:3.0.47', :exclusions => ['com.github.jnr:jnr-ffi']
+  jar 'com.github.jnr:jnr-posix:3.0.50', :exclusions => ['com.github.jnr:jnr-ffi']
   jar 'com.github.jnr:jnr-constants:0.9.12', :exclusions => ['com.github.jnr:jnr-ffi']
-  jar 'com.github.jnr:jnr-ffi:2.1.9'
+  jar 'com.github.jnr:jnr-ffi:2.1.10'
   jar 'com.github.jnr:jffi:${jffi.version}'
   jar 'com.github.jnr:jffi:${jffi.version}:native'

diff --git a/pom.rb b/pom.rb
index 13a50c928e..fc0cf7d5f2 100644
--- a/pom.rb
+++ b/pom.rb
@@ -83,7 +83,7 @@ project 'JRuby', 'https://github.com/jruby/jruby' do
               'jruby-launcher.version' => '1.1.6',
               'ant.version' => '1.9.8',
               'asm.version' => '6.2.1',
-              'jffi.version' => '1.2.18',
+              'jffi.version' => '1.2.19',
               'joda.time.version' => '2.9.9' )
 
   plugin_management do

And mvnw -Pdist creates a .tar.gz with libjffi for DragonFly BSD but I see no stdlib/ffi.

Thanks,
Antonio Huete

@headius
Copy link
Member

@headius headius commented Jun 24, 2019

The stdlib FFI files are mostly generated for each platform, so I think we'd need to figure out how to regenerate them. The alternative, which might be easier (the generation process was not documented and we'd have to dig a bit) would be to just take one of the other BSDs and fix up any type widths etc.

This would be great to get done for 9.2.8!

@headius headius added this to the JRuby 9.2.8.0 milestone Jun 24, 2019
@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Jun 24, 2019

But are they generated for every platform during the jruby build time, or is it pulled externally?

@headius
Copy link
Member

@headius headius commented Jun 25, 2019

@tuxillo They're generated manually, since we don't have a good rig to generate automatically on all these weird platforms.

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Jun 26, 2019

Okay then, if you come across the instructions to generate it maybe I can give it a shot :)

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Jul 16, 2019

By any chance you happened to come across the instructions? Thanks!

@headius
Copy link
Member

@headius headius commented Aug 2, 2019

@tuxillo Sorry for delay. I did not see this update...

I believe if you cd to lib/ruby/stdlib/ffi/tools and run rake it should generate for your local platform, though I had some trouble with this on MacOS (some C conventions and libs on MacOS may have changed).

Can you give that a try?

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Aug 5, 2019

Not sure if I'm doing something wrong. After building JRuby (from master branch) and setting PATH I get this just by running rake:

$ rake
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:13: warning: already initialized constant VERSION
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:118: warning: already initialized constant RUBYGEMS_DIR
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:123: warning: already initialized constant WIN_PATTERNS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:132: warning: already initialized constant GEM_DEP_FILES
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:142: warning: already initialized constant REPOSITORY_SUBDIRECTORIES
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:154: warning: already initialized constant REPOSITORY_DEFAULT_GEM_SUBDIRECTORIES
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:163: warning: already initialized constant READ_BINARY_ERRORS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:173: warning: already initialized constant WRITE_BINARY_ERRORS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:179: warning: already initialized constant USE_BUNDLER_FOR_GEMDEPS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:186: warning: already initialized constant LOADED_SPECS_MUTEX
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:1373: warning: already initialized constant MARSHAL_SPEC_DIR
LoadError: load error: rubygems -- java.lang.NullPointerException: null
  require at org/jruby/RubyKernel.java:987
   <main> at /home/antonioh/s/jruby/bin/rake:9
$ which rake
/home/antonioh/s/jruby/bin/rake

Maybe something is wrong with my setup?

@headius
Copy link
Member

@headius headius commented Aug 6, 2019

@tuxillo Hmm that's interesting. This doesn't seem to be loading right. Can you check if there's any GEM-related environment variables?

You can get the longer trace for the NullPointerException by passing -d to JRuby, e.g. JRUBY_OPTS=-d rake.

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Aug 8, 2019

I don't see any gem-related env variables.

Here's the longer trace:

java.lang.NullPointerException
	at org.jruby.util.io.OpenFile.fdFixCloexec(OpenFile.java:2602)
	at org.jruby.RubyIO.cloexecOpen(RubyIO.java:1276)
	at org.jruby.RubyIO.sysopenFunc(RubyIO.java:1257)
	at org.jruby.RubyIO.sysopenInternal(RubyIO.java:1249)
	at org.jruby.RubyIO.sysopen(RubyIO.java:1234)
	at org.jruby.RubyFile.fileOpenGeneric(RubyFile.java:1473)
	at org.jruby.RubyIO.ioOpenGeneric(RubyIO.java:3691)
	at org.jruby.RubyIO.ioOpen(RubyIO.java:3667)
	at org.jruby.RubyIO.openKeyArgs(RubyIO.java:3650)
	at org.jruby.RubyIO.read(RubyIO.java:3769)
	at org.jruby.RubyIO$INVOKER$s$0$3$read.call(RubyIO$INVOKER$s$0$3$read.gen)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:363)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:201)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:325)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:171)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:158)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:342)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:114)
	at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:151)
	at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:187)
	at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
	at org.jruby.runtime.Block.yield(Block.java:170)
	at org.jruby.ir.runtime.IRRuntimeHelpers.yield(IRRuntimeHelpers.java:477)
	at org.jruby.ir.instructions.YieldInstr.interpret(YieldInstr.java:85)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.processOtherOp(StartupInterpreterEngine.java:178)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:104)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:114)
	at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:151)
	at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:187)
	at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
	at org.jruby.runtime.Block.yield(Block.java:170)
	at org.jruby.RubyArray.each(RubyArray.java:1800)
	at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:299)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:82)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:89)
	at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:537)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:114)
	at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:151)
	at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:187)
	at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
	at org.jruby.runtime.Block.yield(Block.java:170)
	at org.jruby.RubyArray.each(RubyArray.java:1800)
	at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:299)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:82)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:89)
	at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:537)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:171)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:158)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:353)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:180)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:337)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:171)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:158)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:353)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:180)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:337)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:138)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:125)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:321)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:139)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:94)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:80)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:867)
	at org.jruby.Ruby.loadFile(Ruby.java:2964)
	at org.jruby.runtime.load.LibrarySearcher$ResourceLibrary.load(LibrarySearcher.java:236)
	at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:34)
	at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:887)
	at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:535)
	at org.jruby.runtime.load.LoadService.require(LoadService.java:402)
	at org.jruby.RubyKernel.requireCommon(RubyKernel.java:994)
	at org.jruby.RubyKernel.require(RubyKernel.java:987)
	at org.jruby.RubyKernel$INVOKER$s$1$0$require.call(RubyKernel$INVOKER$s$1$0$require.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:417)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:342)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:94)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:80)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:867)
	at org.jruby.Ruby.loadFile(Ruby.java:2964)
	at org.jruby.runtime.load.LibrarySearcher$ResourceLibrary.load(LibrarySearcher.java:236)
	at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:34)
	at org.jruby.runtime.load.LoadService.load(LoadService.java:343)
	at org.jruby.RubyKernel.loadCommon(RubyKernel.java:1052)
	at org.jruby.RubyKernel.load(RubyKernel.java:1022)
	at org.jruby.RubyKernel$INVOKER$s$load.call(RubyKernel$INVOKER$s$load.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:417)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:342)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:94)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:80)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:867)
	at org.jruby.Ruby.loadFile(Ruby.java:2964)
	at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:59)
	at org.jruby.runtime.load.LoadService.loadFromClassLoader(LoadService.java:371)
	at org.jruby.Ruby.initRubyPreludes(Ruby.java:1775)
	at org.jruby.Ruby.init(Ruby.java:1289)
	at org.jruby.Ruby.newInstance(Ruby.java:368)
	at org.jruby.Main.internalRun(Main.java:273)
	at org.jruby.Main.run(Main.java:234)
	at org.jruby.Main.main(Main.java:206)
RubyGems not found; disabling gems
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:13: warning: already initialized constant VERSION
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:118: warning: already initialized constant RUBYGEMS_DIR
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:123: warning: already initialized constant WIN_PATTERNS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:132: warning: already initialized constant GEM_DEP_FILES
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:142: warning: already initialized constant REPOSITORY_SUBDIRECTORIES
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:154: warning: already initialized constant REPOSITORY_DEFAULT_GEM_SUBDIRECTORIES
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:163: warning: already initialized constant READ_BINARY_ERRORS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:173: warning: already initialized constant WRITE_BINARY_ERRORS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:179: warning: already initialized constant USE_BUNDLER_FOR_GEMDEPS
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:186: warning: already initialized constant LOADED_SPECS_MUTEX
/home/antonioh/s/jruby/lib/ruby/stdlib/rubygems.rb:1373: warning: already initialized constant MARSHAL_SPEC_DIR
java.lang.NullPointerException
	at org.jruby.util.io.OpenFile.fdFixCloexec(OpenFile.java:2602)
	at org.jruby.RubyIO.cloexecOpen(RubyIO.java:1276)
	at org.jruby.RubyIO.sysopenFunc(RubyIO.java:1257)
	at org.jruby.RubyIO.sysopenInternal(RubyIO.java:1249)
	at org.jruby.RubyIO.sysopen(RubyIO.java:1234)
	at org.jruby.RubyFile.fileOpenGeneric(RubyFile.java:1473)
	at org.jruby.RubyIO.ioOpenGeneric(RubyIO.java:3691)
	at org.jruby.RubyIO.ioOpen(RubyIO.java:3667)
	at org.jruby.RubyIO.openKeyArgs(RubyIO.java:3650)
	at org.jruby.RubyIO.read(RubyIO.java:3769)
	at org.jruby.RubyIO$INVOKER$s$0$3$read.call(RubyIO$INVOKER$s$0$3$read.gen)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:199)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:325)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:171)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:158)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:114)
	at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:151)
	at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:187)
	at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
	at org.jruby.runtime.Block.yield(Block.java:170)
	at org.jruby.ir.runtime.IRRuntimeHelpers.yield(IRRuntimeHelpers.java:477)
	at org.jruby.ir.instructions.YieldInstr.interpret(YieldInstr.java:85)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.processOtherOp(StartupInterpreterEngine.java:178)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:104)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:114)
	at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:151)
	at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:187)
	at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
	at org.jruby.runtime.Block.yield(Block.java:170)
	at org.jruby.RubyArray.each(RubyArray.java:1800)
	at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:80)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:89)
	at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:537)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:114)
	at org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:151)
	at org.jruby.runtime.IRBlockBody.doYield(IRBlockBody.java:187)
	at org.jruby.runtime.BlockBody.yield(BlockBody.java:116)
	at org.jruby.runtime.Block.yield(Block.java:170)
	at org.jruby.RubyArray.each(RubyArray.java:1800)
	at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:80)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:89)
	at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:537)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:171)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:158)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:178)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:337)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:171)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:158)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:178)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:337)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:138)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:125)
	at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:321)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:139)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:94)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:80)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:867)
	at org.jruby.Ruby.loadFile(Ruby.java:2964)
	at org.jruby.runtime.load.LibrarySearcher$ResourceLibrary.load(LibrarySearcher.java:236)
	at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:34)
	at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:887)
	at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:535)
	at org.jruby.runtime.load.LoadService.require(LoadService.java:402)
	at org.jruby.RubyKernel.requireCommon(RubyKernel.java:994)
	at org.jruby.RubyKernel.require(RubyKernel.java:987)
	at org.jruby.RubyKernel$INVOKER$s$1$0$require.call(RubyKernel$INVOKER$s$1$0$require.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:417)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:342)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
	at home.antonioh.s.jruby.bin.rake.invokeOther1:require(/home/antonioh/s/jruby/bin/rake:9)
	at home.antonioh.s.jruby.bin.rake.RUBY$script(/home/antonioh/s/jruby/bin/rake:9)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
	at org.jruby.ir.Compiler$1.load(Compiler.java:89)
	at org.jruby.Ruby.runScript(Ruby.java:854)
	at org.jruby.Ruby.runNormally(Ruby.java:777)
	at org.jruby.Ruby.runNormally(Ruby.java:795)
	at org.jruby.Ruby.runFromMain(Ruby.java:607)
	at org.jruby.Main.doRunFromMain(Main.java:412)
	at org.jruby.Main.internalRun(Main.java:304)
	at org.jruby.Main.run(Main.java:234)
	at org.jruby.Main.main(Main.java:206)
LoadError: load error: rubygems -- java.lang.NullPointerException: null
  require at org/jruby/RubyKernel.java:987
   <main> at /home/antonioh/s/jruby/bin/rake:9

@headius
Copy link
Member

@headius headius commented Aug 8, 2019

Ok that's peculiar. The line in question is here:

throw new AssertionError(String.format("BUG: rb_maygvl_fd_fix_cloexec: fcntl(%d, F_GETFD) failed: %s", fd, posix.getErrno().description()));

We know posix would not be null since it was just used above to call fcntlGetFd above, so that means it has a null errno after a -1 return value from said function.

And looking at the logic inside that posix function, we see that errno is never set to anything:

public int fcntlGetFD(int fd) {
int ret = posix.fcntl(fd, Fcntl.F_GETFD);
return ret;
}

I'll commit a fix that retrieves the errno and captures it in the posix shim, so you can try it again.

@headius
Copy link
Member

@headius headius commented Aug 8, 2019

I've pushed e255434 but of course it occurred to me that it's just going to raise an error here. Based on that error hopefully we'll have a better idea what's going on.

I'll try to get my own DragonFly VM going also.

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Aug 8, 2019

Thanks @headius !

Now I indeed get a different error:

$ env JRUBY_OPTS=-d rake
[...]
LoadError: load error: rubygems -- java.lang.AssertionError: BUG: rb_maygvl_fd_fix_cloexec: fcntl(18, F_GETFD) failed: EFAULT
  require at org/jruby/RubyKernel.java:987
   <main> at /home/antonioh/s/jruby/bin/rake:9

Doing a ktrace (similar to linux's strace) I can see this:

276814:2    java     CALL  fcntl(0x12,F_SETLK,0x12)
276814:2    java     RET   fcntl -1 errno 14 Bad address
276814:2    java     CALL  fcntl(0x12,F_SETLKW,0xffffffffffffffff)
276814:2    java     RET   fcntl -1 errno 14 Bad address
276814:2    java     CALL  fstat(0x12,0x7fdfdfffb350)
276814:2    java     RET   fstat 0
276814:2    java     CALL  fcntl(0x12,F_SETLK,0x12)
276814:2    java     RET   fcntl -1 errno 14 Bad address

So in a call to fcntl(18, F_GETFD ..) under the hood it is really calling a different operation which could indicate that the definition for F_GETFD is wrong.

I added a lib/ruby/stdlib/ffi/platform/x86_64-dragonfly/fcntl.rb with the correct values but I am still getting the same error, I'm probably missing something.

@headius
Copy link
Member

@headius headius commented Aug 9, 2019

Ah that's interesting! So the really good news here is that it is making the native call. But the fcntl flags appear to not line up.

This logic should be trying to do F_GETFD to get the file descriptor flags. On my system, that flag is "1" which does appear to match my system includes:

[] ~/projects/jruby $ jruby -e 'p Java::jnr.constants.platform.Fcntl::F_GETFD.value'
1

[] ~/projects/jruby $ grep 'define F_GETFD' /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/sys/fcntl.h 
#define F_GETFD         1               /* get file descriptor flags */

In your case it appears to be sending F_SETLK, which is clearly not F_GETFD. On my system F_SETLK is 8, which shouldn't overlap. The values on DragonFly seem to match, though:

https://github.com/DragonFlyBSD/DragonFlyBSD/blob/master/sys/sys/fcntl.h#L190-L224

Can you show me the output of the following command line on your system? I wonder if it's picking the wrong set of Fcntl values...

$ jruby -e 'Java::jnr.constants.platform.Fcntl.values.each {|flag| p [flag.name, flag.value]}'
["FAPPEND", 8]
["FREAD", 1]
["FWRITE", 2]
["FASYNC", 64]
["FFSYNC", 128]
["FNONBLOCK", 4]
["FNDELAY", 4]
["F_DUPFD", 0]
["F_GETFD", 1]
["F_SETFD", 2]
["F_GETFL", 3]
["F_SETFL", 4]
["F_GETOWN", 5]
["F_SETOWN", 6]
["F_GETLK", 7]
["F_SETLK", 8]
["F_SETLKW", 9]
["F_CHKCLEAN", 41]
["F_PREALLOCATE", 42]
["F_SETSIZE", 43]
["F_RDADVISE", 44]
["F_RDAHEAD", 45]
["F_READBOOTSTRAP", 20000]
["F_WRITEBOOTSTRAP", 20001]
["F_NOCACHE", 48]
["F_LOG2PHYS", 49]
["F_GETPATH", 50]
["F_FULLFSYNC", 51]
["F_PATHPKG_CHECK", 52]
["F_FREEZE_FS", 53]
["F_THAW_FS", 54]
["F_GLOBAL_NOCACHE", 55]
["F_ADDSIGS", 59]
["F_MARKDEPENDENCY", 20002]
["F_RDLCK", 1]
["F_UNLCK", 2]
["F_WRLCK", 3]
["F_ALLOCATECONTIG", 2]
["F_ALLOCATEALL", 4]
["__UNKNOWN_CONSTANT__", 20003]

@headius
Copy link
Member

@headius headius commented Aug 9, 2019

Another couple to try:

[] ~/projects/jruby $ jruby -e 'p org.jruby.platform.Platform::IS_BSD'
true

[] ~/projects/jruby $ jruby -e 'puts Java::jnr.constants.Platform::OS'
darwin

@headius
Copy link
Member

@headius headius commented Aug 9, 2019

Another!

[] ~/projects/jruby $ jruby -e 'puts java.lang.System.getProperty("os.name")'
Mac OS X

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Aug 11, 2019

Hi,

Yeah, it's definitely picking up the wrong values:

$ jruby -e 'p Java::jnr.constants.platform.Fcntl::F_GETFD.value'
8

$ jruby -e 'Java::jnr.constants.platform.Fcntl.values.each {|flag| p [flag.name, flag.value]}'
["FAPPEND", 0]
["FREAD", 1]
["FWRITE", 2]
["FASYNC", 3]
["FFSYNC", 4]
["FNONBLOCK", 5]
["FNDELAY", 6]
["F_DUPFD", 7]
["F_GETFD", 8]
["F_SETFD", 9]
["F_GETFL", 10]
["F_SETFL", 11]
["F_GETOWN", 12]
["F_SETOWN", 13]
["F_GETLK", 14]
["F_SETLK", 15]
["F_SETLKW", 16]
["F_CHKCLEAN", 17]
["F_PREALLOCATE", 18]
["F_SETSIZE", 19]
["F_RDADVISE", 20]
["F_RDAHEAD", 21]
["F_READBOOTSTRAP", 22]
["F_WRITEBOOTSTRAP", 23]
["F_NOCACHE", 24]
["F_LOG2PHYS", 25]
["F_GETPATH", 26]
["F_FULLFSYNC", 27]
["F_PATHPKG_CHECK", 28]
["F_FREEZE_FS", 29]
["F_THAW_FS", 30]
["F_GLOBAL_NOCACHE", 31]
["F_ADDSIGS", 32]
["F_MARKDEPENDENCY", 33]
["F_RDLCK", 34]
["F_UNLCK", 35]
["F_WRLCK", 36]
["F_ALLOCATECONTIG", 37]
["F_ALLOCATEALL", 38]
["__UNKNOWN_CONSTANT__", 20000]

$ jruby -e 'p org.jruby.platform.Platform::IS_BSD'
false

$ jruby -e 'puts java.lang.System.getProperty("os.name")'
DragonFlyBSD

I've created this, not sure if i'ts completely correct: #5828

@headius
Copy link
Member

@headius headius commented Aug 12, 2019

@tuxillo That's great! It was exactly what we suspected...those values are from the "fake" set of constants when we can't determine the platform. I've raised the issue with @enebo that we should regenerate these to mimic something a little closer to normal POSIX so that at least when it falls back most of the values should be right.

As for your PR, it looks right to me. Does it make things work for you?

I think we've missed the window for 9.2.8 but will be working to spin a 9.2.9 release MUCH more quickly. That will give us time to ensure things are working properly on your system.

In the meantime, you might try this as a workaround and see what happens:

jruby -J-Dos.name=FreeBSD ...

It might pick the wrong native library and go haywire, though.

@headius headius removed this from the JRuby 9.2.8.0 milestone Aug 12, 2019
@headius headius added this to the JRuby 9.2.9.0 milestone Aug 12, 2019
@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Aug 12, 2019

Thanks!

It indeed detects the OS and uses the correct constants there:

$ jruby -e 'p Java::jnr.constants.platform.Fcntl::F_GETFD.value'
8
$ jruby -J-Dos.name=FreeBSD  -e 'p Java::jnr.constants.platform.Fcntl::F_GETFD.value'
1

I wonder whether FreeBSD uses its own constants or so.

@headius
Copy link
Member

@headius headius commented Aug 12, 2019

I'd be surprised if there were much difference between the BSDs as far as these constants go, so as long as it's able to load native support (-Xnative.logging should print success) you should be good to go with that property until we get 9.2.9 out.

@headius
Copy link
Member

@headius headius commented Aug 12, 2019

Sorry that's -Xnative.verbose.

Adithya-copart pushed a commit to Adithya-copart/jruby that referenced this issue Aug 25, 2019
@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Sep 5, 2019

After the "Prepare errno here, to prevent NPE from ..." I still get the same valeu from F_GETFD=8

Sorry if I'm missing something here!

@ahorek
Copy link
Contributor

@ahorek ahorek commented Oct 15, 2019

since there was work done to support this platform, it would be nice to have jnr/jnr-constants#61 for 9.2.9. ping @headius

headius added a commit that referenced this issue Oct 27, 2019
@headius
Copy link
Member

@headius headius commented Oct 27, 2019

@ahorek Are we good here? I also merged in your openbsd update.

@ahorek
Copy link
Contributor

@ahorek ahorek commented Oct 27, 2019

thanks @headius !

there's one more thing, I tested it and unfortunatelly correct constants are still not loaded

the reason is because the platform name to generate those constants is get from FFI
https://github.com/jnr/jnr-constants/blob/f90dce5c0b0a2c060c733d60a880c254c4637f6d/Rakefile#L11

jruby -e 'require "ffi"; p FFI::Platform::OS'
dragonfly5

but to use them we get a name from a java property
https://github.com/jnr/jnr-constants/blob/aff74258c6db27b31f1612c6aad0f386e29f14d1/src/main/java/jnr/constants/Platform.java#L94

jruby -e 'p Java::jnr.constants.platform::OS'
dragonflybsd

I think that FFI gem should also be updated to identify dragonflybsd as a platform
https://github.com/ffi/ffi/blob/6e631e06d7539543bccac967720e3410fb9e710c/lib/ffi/platform.rb#L37
and dragonflybsd should be the correct platform name (without a version), but I'm not sure, they have a separate platform for FreeBSD and FreeBSD12.

What would you recommend?

related PRs
jnr/jffi#77
jnr/jnr-constants#63
ffi/ffi#724

I think we're very close!

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Oct 27, 2019

We don't keep backwards compatibility for the most part so I think having a platform called "dragonflybsd" is okay, but I am not sure if that's the question.

@ahorek
Copy link
Contributor

@ahorek ahorek commented Oct 27, 2019

I think the intention of https://github.com/ffi/ffi/pull/724/files#diff-db0e38803c251431bdf21ae8936ac3f6R38 was to unify the platform name that omits the specific version of a platform, but that doesn't happen if the platform is unknown, because RbConfig::CONFIG['host_os'] contains a version.
We can add a new platform for future versions (DragonFlyBSD 6 ?), if there's any incompatibility, but for all existing platforms that situation happened only once with FreeBSD 12.

at least we have to choose one name everywhere, so we don't have to search for dragonflybsd libraries in the dragonfly5 folder etc. I'm open to other suggestions.

@headius
Copy link
Member

@headius headius commented Oct 28, 2019

Avoiding versioning in the platform is certainly best. I'm thinking a better long-term path would be to have a separate version field that can be used only when needed and populated from whatever properties make sense. As @ahorek points out we've only had a version-specific problem with FreeBSD; if we have another one at some point it may force the issue but I'm inclined to stick with version free platforms as much as possible right now.

So I think if we get these changes merged in (and I'll manually make the change to JRuby's built-in FFI) we should be good.

@headius
Copy link
Member

@headius headius commented Oct 28, 2019

I've pulled in the jnr pull requests and updated JRuby to point at the new releases. I'm now working on a PR for JRuby 9.2.10 that will update all related files from the FFI gem so we can start single-sourcing them (See #5947 #5948).

Thanks for the help everyone! Hopefully you can also help me get the shared files working properly!

@headius headius closed this Oct 28, 2019
@ahorek
Copy link
Contributor

@ahorek ahorek commented Oct 28, 2019

I can confirm it works fine! thanks again

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Oct 29, 2019

Me too!

# jruby -e 'p Java::jnr.constants.platform.Fcntl::F_GETFD.value'
1

Many thanks guys!

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Oct 29, 2019

@headius what about the stdlib thing we mentioned at the beginning of this issue?

@headius
Copy link
Member

@headius headius commented Oct 30, 2019

@tuxillo That will have to wait a bit... See #5948 and the related PR and if you can help me get things green again that would be great! I want to make 9.2.10 use the exact same files as the ffi gem, so we can just default install it as part of the build to pick up all types configs and generated files.

@tuxillo
Copy link
Contributor Author

@tuxillo tuxillo commented Jan 7, 2020

@headius I am not sure what I should do :)

@headius
Copy link
Member

@headius headius commented Jan 7, 2020

@tuxillo Most of the errors appear to be issues in the native parts of JRuby's FFI, probably things that evolved in the gem but were never propagated to the JRuby version. A large number of these are "got type X and expected Y" where Y should be able to coerce into X...so we're probably not doing the right coercion call (e.g. to_int or to_str) on the native side.

If you can pull that branch and run those FFI specs, it would be a big help to reduce them down to just the couple of lines that fails, so we can iterate on the errors quickly and fix the internals.

@headius
Copy link
Member

@headius headius commented Jan 8, 2020

I started looking into it a bit and it seems like the main change we'll need to make is reconciling the ruby/native split to match the FFI gem. Several of the classes that were originally defined in C are now moved into Ruby, and with JRuby's versions still in Java we end up with conflicting logic. The first example I found was that the Enums and Enum class are now defined in Ruby, but we expect them to behave based on how they're defined in Java. Commenting out the entire file got more specs passing, but that's not the solution going forward.

I'll poke at it a bit and push some changes to the PR that may help others assist.

@headius
Copy link
Member

@headius headius commented Jan 8, 2020

I've pushed a number of changes that reduces the failure count to 13. Mostly I'm just going spec by spec and looking to make logic match between the C FFI and the JRuby FFI. In many cases this just means removing the JRuby native version of some logic and allowing the Ruby logic to win.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants