JRuby fails to load FFI support with obscure error when /tmp is not writable #1302

qerub · 2013-12-04T13:44:42Z

$ sudo chmod 1775 /tmp
$ java -Djruby.native.verbose=true -jar jruby-complete.jar -rffi -e :ok
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
        at jnr.ffi.provider.InvalidProvider$1.loadLibrary(InvalidProvider.java:30)
        at jnr.ffi.LibraryLoader.load(LibraryLoader.java:228)
        at jnr.ffi.Library.loadLibrary(Library.java:123)
        at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:219)
        at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:223)
        at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:37)
        at jnr.posix.LinuxPOSIX.<init>(LinuxPOSIX.java:20)
        at jnr.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:95)
        at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:69)
        at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:38)
        at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:33)
        at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:29)
        at jnr.posix.LazyPOSIX.isatty(LazyPOSIX.java:183)
        at org.jruby.Main.handleRaiseException(Main.java:509)
        at org.jruby.Main.main(Main.java:202)
Caused by: java.lang.ExceptionInInitializerError
        at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:49)
        at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
        at java.lang.Class.newInstance0(Class.java:374)
        at java.lang.Class.newInstance(Class.java:327)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:60)
        at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:49)
        at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
        at jnr.ffi.Runtime$SingletonHolder.<clinit>(Runtime.java:85)
        at jnr.ffi.Runtime.getSystemRuntime(Runtime.java:70)
        at jnr.posix.NativePOSIX.<init>(NativePOSIX.java:9)
        at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:35)
        ... 9 more
Caused by: java.lang.IllegalStateException: Can't overwrite cause
        at java.lang.Throwable.initCause(Throwable.java:337)
        at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:252)
        at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
        at com.kenai.jffi.Type.resolveSize(Type.java:155)
        at com.kenai.jffi.Type.size(Type.java:138)
        at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:178)
        at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:57)
        at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
        at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:53)
        ... 24 more
LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
 See http://jira.codehaus.org/browse/JRUBY-4583
  require at org/jruby/RubyKernel.java:1084
  require at jar:file:/tmp/jruby-sandbox/vendor/jruby-complete-with-bundler.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
   (root) at jar:file:/tmp/jruby-sandbox/vendor/jruby-complete-with-bundler.jar!/META-INF/jruby.home/lib/ruby/shared/ffi/ffi.rb:69
  require at org/jruby/RubyKernel.java:1084
   (root) at jar:file:/tmp/jruby-sandbox/vendor/jruby-complete-with-bundler.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
  require at jar:file:/tmp/jruby-sandbox/vendor/jruby-complete-with-bundler.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
  require at org/jruby/RubyKernel.java:1084
   (root) at jar:file:/tmp/jruby-sandbox/vendor/jruby-complete-with-bundler.jar!/META-INF/jruby.home/lib/ruby/shared/ffi.rb:1

$ sudo chmod 1777 /tmp
$ java -Djruby.native.verbose=true -jar jruby-complete.jar -rffi -e :ok
Successfully loaded native POSIX impl.

I noticed this when I accidentally changed permissions for /tmp. A more informative message would be helpful. :)

blendmaster · 2014-10-27T21:39:35Z

I just ran into this issue, can this error message be made better somehow? 😞

jkraemer · 2015-12-18T10:06:28Z

Much better would be a fix for the problem - how can I get around this issue if making /tmp writable is not an option?

mkristian · 2015-12-18T10:39:50Z

@jkraemer you can set java.io.tmpdir property:

java -Djruby.native.verbose=true  -Djava.io.tmpdir=jruby-temp-dir -jar jruby-complete.jar -rffi -e :ok

ranjankumar03 · 2017-09-01T08:44:58Z

Anyone aware of this?

datastax.driver.core.Native:113 -Could not load JNR C Library, native system calls through this library will not be available
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
at jnr.ffi.provider.InvalidProvider$1.loadLibrary(InvalidProvider.java:48)
at jnr.ffi.LibraryLoader.load(LibraryLoader.java:290)
at jnr.ffi.LibraryLoader.load(LibraryLoader.java:269)
at com.datastax.driver.core.Native$LibCLoader.(Native.java:108)
at com.datastax.driver.core.Native.isGettimeofdayAvailable(Native.java:190)
at com.datastax.driver.core.ClockFactory.newInstance(Clock.java:51)
at com.datastax.driver.core.AbstractMonotonicTimestampGenerator.(AbstractMonotonicTimestampGenerator.java:43)
at com.datastax.driver.core.LoggingMonotonicTimestampGenerator.(LoggingMonotonicTimestampGenerator.java:49)
at com.datastax.driver.core.AtomicMonotonicTimestampGenerator.(AtomicMonotonicTimestampGenerator.java:50)
at com.datastax.driver.core.AtomicMonotonicTimestampGenerator.(AtomicMonotonicTimestampGenerator.java:37)
at com.datastax.driver.core.policies.Policies.defaultTimestampGenerator(Policies.java:121)
at com.datastax.driver.core.policies.Policies$Builder.build(Policies.java:286)
at com.datastax.driver.core.Cluster$Builder.getConfiguration(Cluster.java:1317)
at com.datastax.driver.core.Cluster.(Cluster.java:113)
at com.datastax.driver.core.Cluster.buildFrom(Cluster.java:178)
at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:1335)

rdp · 2019-09-10T15:08:28Z

Just ran into this, (CentOS 7 some distros have /tmp non-writable by default :( ). For google followers, the error messages now look like this:

 java -jar jruby-complete-9.2.8.0.jar -S irb
NotImplementedError: fstat unimplemented unsupported or native support failed to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
  initialize at org/jruby/RubyIO.java:1015
        open at org/jruby/RubyIO.java:1156
  initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
  initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
  initialize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
       start at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:381
      <main> at uri:classloader:/META-INF/jruby.home/bin/jirb:13

and

 java -Djruby.native.verbose=true -jar jruby-complete-9.2.8.0.jar  -S irb
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
	at jnr.ffi.provider.InvalidProvider$1.loadLibrary(InvalidProvider.java:48)
	at jnr.ffi.LibraryLoader.load(LibraryLoader.java:325)
	at jnr.ffi.Library.loadLibrary(Library.java:127)
	at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:289)
	at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:318)
	at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:38)
	at jnr.posix.LinuxPOSIX.<init>(LinuxPOSIX.java:19)
	at jnr.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:149)
	at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:124)
	at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:93)
	at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:38)
	at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:32)
	at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:402)
	at org.jruby.util.io.FilenoUtil.<init>(FilenoUtil.java:32)
	at org.jruby.Ruby.<init>(Ruby.java:318)
	at org.jruby.Ruby.newInstance(Ruby.java:367)
	at org.jruby.Main.internalRun(Main.java:273)
	at org.jruby.Main.run(Main.java:234)
	at org.jruby.Main.main(Main.java:206)
Caused by: java.lang.UnsatisfiedLinkError: could not get native definition for type: POINTER
	at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:251)
	at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
	at com.kenai.jffi.Type.resolveSize(Type.java:155)
	at com.kenai.jffi.Type.size(Type.java:138)
	at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:178)
	at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:57)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
	at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:53)
	at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:49)
	at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
	at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
	at jnr.ffi.Library.loadLibrary(Library.java:114)
	... 16 more
Caused by: java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: /tmp/jffi7684241648010829190.so: /tmp/jffi7684241648010829190.so: failed to map segment from shared object: Operation not permitted
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:809)
	at java.lang.System.load(System.java:1086)
	at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:380)
	at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:262)
	at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:453)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at com.kenai.jffi.Init.load(Init.java:68)
	at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:49)
	at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:45)
	at com.kenai.jffi.Foreign.getInstance(Foreign.java:103)
	at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
	at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
	at com.kenai.jffi.Type.resolveSize(Type.java:155)
	at com.kenai.jffi.Type.size(Type.java:138)
	at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:178)
	at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:57)
	at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
	at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:53)
	at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:49)
	at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
	at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
	at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
	at jnr.ffi.Library.loadLibrary(Library.java:114)
	at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:289)
	at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:318)
	at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:38)
	at jnr.posix.LinuxPOSIX.<init>(LinuxPOSIX.java:19)
	at jnr.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:149)
	at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:124)
	at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:93)
	at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:38)
	at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:32)
	at jnr.posix.LazyPOSIX.isNative(LazyPOSIX.java:402)
	at org.jruby.util.io.FilenoUtil.<init>(FilenoUtil.java:32)
	at org.jruby.Ruby.<init>(Ruby.java:318)
	at org.jruby.Ruby.newInstance(Ruby.java:367)
	at org.jruby.Main.internalRun(Main.java:273)
	at org.jruby.Main.run(Main.java:234)
	at org.jruby.Main.main(Main.java:206)

	at com.kenai.jffi.Foreign.newLoadError(Foreign.java:72)
	at com.kenai.jffi.Foreign.access$300(Foreign.java:42)
	at com.kenai.jffi.Foreign$InValidInstanceHolder.getForeign(Foreign.java:98)
	at com.kenai.jffi.Foreign.getInstance(Foreign.java:103)
	at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
	... 35 more

suggestion/feature request: if /tmp not executable, fallback to ~/.jruby-cache or something. Cheers!..

headius · 2019-09-11T20:38:29Z

Hmm I suppose we could fall back, but I don't like the thought that we'd now dumping binaries into an even more unexpected location. The error should definitely be clearer though.

How can a system work with a read-only /tmp? Are we not paying attention to some user TMP variable or something?

rdp · 2019-09-12T03:26:33Z

hmm maybe /run/whatever or something like that?

…

On Wed, Sep 11, 2019 at 2:38 PM Charles Oliver Nutter < ***@***.***> wrote: Hmm I suppose we could fall back, but I don't like the thought that we'd now dumping binaries into an even more unexpected location. The error should definitely be clearer though. How can a system work with a read-only /tmp? Are we not paying attention to some user TMP variable or something? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1302?email_source=notifications&email_token=AAADBUDRFGUWA6MPQPDEEPTQJFJM3A5CNFSM4AKHADB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6P2GVI#issuecomment-530555733>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAADBUESZR2JNRSJVVZDIKLQJFJM3ANCNFSM4AKHADBQ> .

headius · 2019-09-25T22:04:45Z

@rdp I think the issue you have there is that it's actually disallowing executables from /tmp, which is a feature of some "secure" Linux distributions. I believe if you set your own TMP or TEMP or whatever, it will work. Can you confirm?


        Don't attempt to initCause this exception.

In jruby/jruby#1302 we see that attempting to load FFI type defs when the jffi stub can't be loaded results in IllegalStateException being raised when we try to initCause on an UnsatisfiedLinkError. It appears that (sometimes?) simply constructing this exception initializes its cause, which can only be done once. Our attempt to initialize it again triggers the error. Since ULE does not have a constructor that accepts a cause, the only option here is to include the message from the triggering exception.


        Update to jffi 1.2.21 for better linkage errors.

Fixes #5900, #1302, and probably others.

headius · 2019-10-08T21:04:21Z

The original weirdness of this error was due to JFFI attempting to initCause on an UnsatisfiedLinkError that appears to get its own cause (sometimes? always?). I have fixed that in jffi and added better error messages (see #5900) that should help folks diagnose this problem in the future.

Note that you will still have to set the appropriate property or flag for verbose native loading in order to see the updated error, but it now explains that we need a writable, exec-able temp location.

headius · 2019-10-08T21:35:03Z

Also FWIW, JRuby should be attempting to fall back on the current directory for this temporary library, which isn't great but it's the behavior we have in place.

rdp · 2019-10-17T20:46:02Z

Sweet. With 9.2.8.0 it was failing but with 9.2.9.0-SNAPSHOT seems to be working great, even if the CWD isn't executable, sweet!

headius mentioned this issue Oct 1, 2019

reporting a better error when tmp doesn't work for temporary executables #5900

Closed

headius added this to the JRuby 9.2.9.0 milestone Oct 8, 2019

headius closed this Oct 8, 2019

jruby / jruby

JRuby fails to load FFI support with obscure error when /tmp is not writable #1302

JRuby fails to load FFI support with obscure error when /tmp is not writable #1302

qerub commented Dec 4, 2013

This comment has been minimized.

blendmaster commented Oct 27, 2014

This comment has been minimized.

jkraemer commented Dec 18, 2015

This comment has been minimized.

mkristian commented Dec 18, 2015

This comment has been minimized.

ranjankumar03 commented Sep 1, 2017

This comment has been minimized.

rdp commented Sep 10, 2019 •

edited

This comment has been minimized.

headius commented Sep 11, 2019

This comment has been minimized.

rdp commented Sep 12, 2019

This comment has been minimized.

headius commented Sep 25, 2019

This comment has been minimized.

headius commented Oct 8, 2019

This comment has been minimized.

headius commented Oct 8, 2019

This comment has been minimized.

rdp commented Oct 17, 2019

jruby / jruby

Join GitHub today

JRuby fails to load FFI support with obscure error when /tmp is not writable #1302

JRuby fails to load FFI support with obscure error when /tmp is not writable #1302

Comments

qerub commented Dec 4, 2013

This comment has been minimized.

blendmaster commented Oct 27, 2014

This comment has been minimized.

jkraemer commented Dec 18, 2015

This comment has been minimized.

mkristian commented Dec 18, 2015

This comment has been minimized.

ranjankumar03 commented Sep 1, 2017

This comment has been minimized.

rdp commented Sep 10, 2019 • edited

This comment has been minimized.

headius commented Sep 11, 2019

This comment has been minimized.

rdp commented Sep 12, 2019

This comment has been minimized.

headius commented Sep 25, 2019

This comment has been minimized.

headius commented Oct 8, 2019

This comment has been minimized.

headius commented Oct 8, 2019

This comment has been minimized.

rdp commented Oct 17, 2019

rdp commented Sep 10, 2019 •

edited