Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some File::Stat operations don't work for Oracle JDK #2913

Closed
jsvd opened this Issue May 6, 2015 · 8 comments

Comments

Projects
None yet
5 participants
@jsvd
Copy link

jsvd commented May 6, 2015

Installed Oracle JDK using:

sudo add-apt-repository -y ppa:webupd8team/java
sudo apt-get update
sudo apt-get -y install oracle-java8-installer
$ lsb_release -d
Description:    Ubuntu 14.04.2 LTS

Then tried using blockdev? and dev_minor:

ubuntu@ip-172-30-0-98:~/jruby-9.0.0.0.pre2$ bin/jruby -v
jruby 9.0.0.0.pre2 (2.2.2) 2015-04-28 2755ae0 Java HotSpot(TM) 64-Bit Server VM 25.45-b02 on 1.8.0_45-b14 +jit [linux-amd64]
ubuntu@ip-172-30-0-98:~/jruby-9.0.0.0.pre2$ bin/jruby -S irb
irb(main):001:0> File::Stat.new("/").blockdev?
NotImplementedError: block device detection unsupported or native support failed to load
    from org/jruby/RubyFileStat.java:172:in `blockdev?'
    from (irb):1:in `<eval>'
    from org/jruby/RubyKernel.java:1005:in `eval'
    from org/jruby/RubyKernel.java:1315:in `loop'
    from org/jruby/RubyKernel.java:1125:in `catch'
    from org/jruby/RubyKernel.java:1125:in `catch'
    from /home/ubuntu/jruby-9.0.0.0.pre2/bin/jirb:13:in `<top>'
irb(main):002:0> File::Stat.new("/").dev_minor
NotImplementedError: stat.st_dev unsupported or native support failed to load
    from org/jruby/RubyFileStat.java:233:in `dev_minor'
    from (irb):2:in `<eval>'
    from org/jruby/RubyKernel.java:1005:in `eval'
    from org/jruby/RubyKernel.java:1315:in `loop'
    from org/jruby/RubyKernel.java:1125:in `catch'
    from org/jruby/RubyKernel.java:1125:in `catch'
    from /home/ubuntu/jruby-9.0.0.0.pre2/bin/jirb:13:in `<top>'
irb(main):003:0> 

Seeing this in elastic/logstash#3127

@jsvd

This comment has been minimized.

Copy link
Author

jsvd commented May 7, 2015

Further investigations:

Stock Ubuntu VM on EC2

ubuntu@ip-172-30-0-98:~$ lsb_release -d
Description:    Ubuntu 14.04.2 LTS
ubuntu@ip-172-30-0-98:~$ uname -a
Linux ip-172-30-0-98 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ip-172-30-0-98:~$ jruby-1.7.20/bin/jruby -v
jruby 1.7.20 (1.9.3p551) 2015-05-04 3086e6a on Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14 +jit [linux-amd64]
ubuntu@ip-172-30-0-98:~$ jruby-1.7.20/bin/jruby -e "File.stat('/').dev_minor"
NotImplementedError: stat.st_dev unsupported or native support failed to load
  dev_minor at org/jruby/RubyFileStat.java:211
     (root) at -e:1

Stock RHEL VM on EC2:

[ec2-user@ip-172-30-0-53 ~]$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.1 (Maipo)
[ec2-user@ip-172-30-0-53 ~]$ uname -a
Linux ip-172-30-0-53.eu-west-1.compute.internal 3.10.0-229.el7.x86_64 #1 SMP Thu Jan 29 18:37:38 EST 2015 x86_64 x86_64 x86_64 GNU/Linux
[ec2-user@ip-172-30-0-53 ~]$ jruby-1.7.20/bin/jruby -v
jruby 1.7.20 (1.9.3p551) 2015-05-04 3086e6a on Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14 +jit [linux-amd64]
[ec2-user@ip-172-30-0-53 ~]$ jruby-1.7.20/bin/jruby -e "puts File.stat('/').dev_minor"
51714
@headius

This comment has been minimized.

Copy link
Member

headius commented May 8, 2015

I suspect the Ubuntu VM is failing to load our native stub. Can you try passing -Xnative.verbose=true to JRuby and show us the output?

@headius

This comment has been minimized.

Copy link
Member

headius commented May 8, 2015

@mkristian I see we still have a jni dir under lib, but I think we're still letting jffi create a tempfile and load native bits from there. Perhaps we just need a launcher or Main change to include lib/jni in classpath? That would save us from issues unpacking the binary to /tmp and having security policies prevent loading it.

@jsvd

This comment has been minimized.

Copy link
Author

jsvd commented May 8, 2015

ubuntu@ip-172-30-0-26:~/jruby-9.0.0.0.pre2$ bin/jruby -e "puts File.stat('/').dev_minor"
NotImplementedError: stat.st_dev unsupported or native support failed to load
  dev_minor at org/jruby/RubyFileStat.java:233
      <top> at -e:1
ubuntu@ip-172-30-0-26:~/jruby-9.0.0.0.pre2$ sudo apt-get install libc6-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  glibc-doc
The following NEW packages will be installed:
  libc6-dev
0 upgraded, 1 newly installed, 0 to remove and 42 not upgraded.
Need to get 0 B/1910 kB of archives.
After this operation, 12.4 MB of additional disk space will be used.
Selecting previously unselected package libc6-dev:amd64.
(Reading database ... 54594 files and directories currently installed.)
Preparing to unpack .../libc6-dev_2.19-0ubuntu6.6_amd64.deb ...
Unpacking libc6-dev:amd64 (2.19-0ubuntu6.6) ...
Setting up libc6-dev:amd64 (2.19-0ubuntu6.6) ...
ubuntu@ip-172-30-0-26:~/jruby-9.0.0.0.pre2$ bin/jruby -e "puts File.stat('/').dev_minor"
51713
@jsvd

This comment has been minimized.

Copy link
Author

jsvd commented May 8, 2015

Also, the output of the command with native.verbose: https://gist.github.com/jsvd/486e4b5770264887e5fa

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented May 8, 2015

@headius the jffi-native.jar gets shaded into lib/jruby.jar and we still unpack those file under lib/jni

guess addin lib/jni to classpath on the launcher would do AND remove the same files from lib/jruby.jar

BUT we also should allow to set a different tmp dir for unpacking those files ( unless this is already possible ) for all the folks using embedded jruby.

jordansissel added a commit to jordansissel/logstash that referenced this issue May 11, 2015

Add libc6-dev dependency to the deb package
This is required to resolve a Java 8 + JRuby FFI bug which causes
File::Stat#dev_minor and similar functions to raise this exception:

> NotImplementedError: block device detection unsupported or native
> support failed to load

References
* elastic#3127
* jruby/jruby#2913

Fixes elastic#3216
@jordansissel

This comment has been minimized.

Copy link
Contributor

jordansissel commented May 11, 2015

In digging, I found that Ubuntu 14.04's libc6 package doesn't have a file named "libcrypt.so" and JRuby tries to load that. There are two known workarounds at this time:

  • Install the libc6-dev package
  • or, run ln -s /lib/x86_64-linux-gnu/libcrypt.so.1 /usr/lib/x86_64-linux-gnu/libcrypt.so

I've seen this before specifically with ruby ffi where it tries to search for libfoo.so but doesn't find it, all the while libfoo.so.1 is lying there waiting to be used. I've only seen this on Debian/Ubuntu systems, I think.

@jordansissel

This comment has been minimized.

Copy link
Contributor

jordansissel commented May 11, 2015

Full details of my debugging can be found here: elastic/logstash#3127 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.