AIX 6.1: stat.st_gid unsupported or native support failed to load #587

Closed
jstoneky opened this Issue Mar 15, 2013 · 7 comments

Projects

None yet

1 participant

@jstoneky
Contributor

Looks like an issue with the jffi lib for AIX. This error occurs when loading a bare rails app as well as assets:precompile (so I used that trace for simplicity)

Output of some other troubleshooting task I read thus far:

jruby -v -e 'File.stat("/tmp")'
jruby 1.7.4.dev (1.9.3p385) 2013-03-13 4f459db on IBM J9 VM pap3260sr10-20111208_01 (SR10) [AIX-ppc]
jruby -v -w -rffi -e 'p FFI::Platform::ARCH'
jruby 1.7.4.dev (1.9.3p385) 2013-03-13 4f459db on IBM J9 VM pap3260sr10-20111208_01 (SR10) [AIX-ppc]
LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
 See http://jira.codehaus.org/browse/JRUBY-4583
  require at org/jruby/RubyKernel.java:1028
  require at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/rubygems/custom_require.rb:36
   (root) at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/ffi/ffi.rb:69
  require at org/jruby/RubyKernel.java:1028
   (root) at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/rubygems/custom_require.rb:1
  require at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/rubygems/custom_require.rb:36
  require at org/jruby/RubyKernel.java:1028
   (root) at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/ffi.rb:1
jruby -rffi -e 'module LibC; extend FFI::Library; ffi_lib "c"; attach_function :getpid, [], :pid_t;end; p LibC.getpid'
LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
 See http://jira.codehaus.org/browse/JRUBY-4583
  require at org/jruby/RubyKernel.java:1028
  require at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/rubygems/custom_require.rb:36
   (root) at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/ffi/ffi.rb:69
  require at org/jruby/RubyKernel.java:1028
   (root) at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/rubygems/custom_require.rb:1
  require at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/rubygems/custom_require.rb:36
  require at org/jruby/RubyKernel.java:1028
   (root) at /dsap/gpfs/home/jstone1/jruby/lib/ruby/shared/ffi.rb:1
** Invoke assets:precompile (first_time)
** Execute assets:precompile
/dsap/gpfs/home/jstone1/jruby/bin/jruby /dsap/gpfs/home/jstone1/jruby/bin/rake assets:precompile:all RAILS_ENV=production RAILS_GROUPS=assets --trace
** Invoke assets:precompile:all (first_time)
** Execute assets:precompile:all
** Invoke assets:precompile:primary (first_time)
** Invoke assets:environment (first_time)
** Execute assets:environment
** Invoke environment (first_time)
** Execute environment
** Invoke tmp:cache:clear (first_time)
** Execute assets:precompile:primary
rake aborted!
stat.st_gid unsupported or native support failed to load
org/jruby/RubyFileStat.java:429:in `gid'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.12/lib/active_support/core_ext/file/atomic.rb:39:in `atomic_write'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.12/lib/active_support/cache/file_store.rb:91:in `write_entry'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.12/lib/active_support/cache/strategy/local_cache.rb:140:in `write_entry'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.12/lib/active_support/cache.rb:364:in `write'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.12/lib/active_support/cache.rb:520:in `instrument'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.12/lib/active_support/cache.rb:362:in `write'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/caching.rb:90:in `cache_set'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/caching.rb:53:in `cache_set_hash'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/caching.rb:24:in `cache_asset'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/index.rb:92:in `build_asset'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/base.rb:169:in `find_asset'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/index.rb:60:in `find_asset'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/static_compiler.rb:19:in `compile'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/base.rb:219:in `each_logical_path'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/base.rb:206:in `each_file'
org/jruby/RubyArray.java:1614:in `each'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/base.rb:196:in `each_entry'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/base.rb:204:in `each_file'
org/jruby/RubyArray.java:1614:in `each'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/base.rb:203:in `each_file'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/sprockets-2.2.2/lib/sprockets/base.rb:217:in `each_logical_path'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/static_compiler.rb:18:in `compile'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/assets.rake:56:in `internal_precompile'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/assets.rake:70:in `(root)'
org/jruby/RubyProc.java:256:in `call'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:228:in `execute'
org/jruby/RubyArray.java:1614:in `each'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:223:in `execute'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:166:in `invoke_with_call_chain'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/1.9/monitor.rb:211:in `mon_synchronize'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:159:in `invoke_with_call_chain'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:152:in `invoke'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/assets.rake:60:in `(root)'
org/jruby/RubyProc.java:256:in `call'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:228:in `execute'
org/jruby/RubyArray.java:1614:in `each'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:223:in `execute'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:166:in `invoke_with_call_chain'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/1.9/monitor.rb:211:in `mon_synchronize'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:159:in `invoke_with_call_chain'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:152:in `invoke'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:143:in `invoke_task'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:101:in `top_level'
org/jruby/RubyArray.java:1614:in `each'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:101:in `top_level'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:110:in `run_with_threads'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:95:in `top_level'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:73:in `run'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:160:in `standard_exception_handling'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:70:in `run'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/bin/rake:33:in `(root)'
org/jruby/RubyKernel.java:1047:in `load'
/dsap/gpfs/home/jstone1/jruby/bin/rake:23:in `(root)'
Tasks: TOP => assets:precompile:primary
rake aborted!
Command failed with status (1): [/dsap/gpfs/home/jstone1/jruby/bin/jruby /d...]
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/file_utils.rb:53:in `create_shell_runner'
org/jruby/RubyProc.java:256:in `call'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/file_utils.rb:45:in `sh'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/file_utils_ext.rb:44:in `sh'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/file_utils.rb:80:in `ruby'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/file_utils_ext.rb:44:in `ruby'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/assets.rake:12:in `ruby_rake_task'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/assets.rake:21:in `invoke_or_reboot_rake_task'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/actionpack-3.2.12/lib/sprockets/assets.rake:29:in `(root)'
org/jruby/RubyProc.java:256:in `call'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:228:in `execute'
org/jruby/RubyArray.java:1614:in `each'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:223:in `execute'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:166:in `invoke_with_call_chain'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/1.9/monitor.rb:211:in `mon_synchronize'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:159:in `invoke_with_call_chain'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/task.rb:152:in `invoke'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:143:in `invoke_task'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:101:in `top_level'
org/jruby/RubyArray.java:1614:in `each'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:101:in `top_level'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:110:in `run_with_threads'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:95:in `top_level'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:73:in `run'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:160:in `standard_exception_handling'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/lib/rake/application.rb:70:in `run'
/dsap/gpfs/home/jstone1/jruby/lib/ruby/gems/shared/gems/rake-10.0.3/bin/rake:33:in `(root)'
org/jruby/RubyKernel.java:1047:in `load'
/dsap/gpfs/home/jstone1/jruby/bin/rake:23:in `(root)'
Tasks: TOP => assets:precompile
@jstoneky
Contributor

I compiled the jffi-ppc-AIX.jar and added that to my lib/native and I've made it to this error now.

 jruby -rffi -e 'module LibC; extend FFI::Library; ffi_lib "c"; attach_function :getpid, [], :pid_t;end; p LibC.getpid'
LoadError: Could not open library 'c' : Could not load module .
System error: No such file or directory. Could not open library 'libc.a(shr.o).a' : Could not load module .
System error: No such file or directory
  ffi_lib at /dsap/gpfs/home/jstone1/jruby-1.7.3/lib/ruby/shared/ffi/library.rb:83
      map at org/jruby/RubyArray.java:2361
  ffi_lib at /dsap/gpfs/home/jstone1/jruby-1.7.3/lib/ruby/shared/ffi/library.rb:65
     LibC at -e:1
   (root) at -e:1

Maybe you all have a better idea. It looks a lot like Issue #352 but I don't see anything breaking my LIBPATH in AIX

@ghost
ghost commented Mar 15, 2013

Try eliminating as much of the FFI machinery as you can, to narrow it down a bit.
e.g. try

jruby -rffi -e 'p FFI::DynamicLibrary.open("/usr/lib/libc.a", FFI::DynamicLibrary::RTLD_LAZY | FFI::DynamicLibrary::RTLD_GLOBAL)'

That is a very thin wrapper around dlopen(), so it is a better indicator that the jffi library is working. You might need to tweak what path is loads - the way FFI maps from 'c' => libc.a may be wrong.

@jstoneky
Contributor

Interesting error when I run that which others seem to be attributing to 64/32bit issues. We are on AIX 64bit and I am using 32bit java 6.

 jruby -rffi -e 'p FFI::DynamicLibrary.open("/usr/lib/libc.a", FFI::DynamicLibrary::RTLD_LAZY | FFI::DynamicLibrary::RTLD_GLOBAL)'
LoadError: Could not open library '/usr/lib/libc.a' : Could not load module /usr/lib/libc.a.
        The module has an invalid magic number.
    open at org/jruby/ext/ffi/jffi/DynamicLibrary.java:71
  (root) at -e:1
@ghost
ghost commented Mar 15, 2013

Try different values for the library path - I'm only guessing here - AIX boxes aren't exactly plentiful :-)

e.g. (non-absolute paths):

libc.a
libc.a(shr.o)

By not specifying the path, it should allow dlopen to search the appropriate path for the binary (i.e. if 32bit binaries load libs from a different path).

@jstoneky
Contributor

Looks like the libc.a(shr.o) worked for this snippet. Now to figure out what that means ;). Also I can provide the ppc-aix jffi jar if you want to add it in.

jruby -rffi -e 'p FFI::DynamicLibrary.open("libc.a(shr.o)", FFI::DynamicLibrary::RTLD_LAZY | FFI::DynamicLibrary::RTLD_GLOBAL)'
#<FFI::DynamicLibrary:0x219a219a>

I can also use the full path, but it seems i have to specify the library. I cannot load the archive without that magic number issue.

jruby -rffi -e 'p FFI::DynamicLibrary.open("/usr/lib/libc.a(shr.o)", FFI::DynamicLibrary::RTLD_LAZY | FFI::DynamicLibrary::RTLD_GLOBAL)'
#<FFI::DynamicLibrary:0xaa20aa2>

I also noticed I can fix the issue by checking if the library is an object before appending .a to it in map_library_name. This only fixes the case above when i was loading ffi_lib "c", but does not fix rake assets:precompile

module FFI

  def self.map_library_name(lib)
    lib = lib.to_s
    # Mangle the library name to reflect the native library naming conventions
    lib = Platform::LIBC if lib == 'c'
    if lib && File.basename(lib) == lib
      is_object = lib =~ /\((.*?)\)/
      lib = Platform::LIBPREFIX + lib unless lib =~ /^#{Platform::LIBPREFIX}/
      r = Platform::IS_LINUX ? "\\.so($|\\.[1234567890]+)" : "\\.#{Platform::LIBSUFFIX}$"
      lib += ".#{Platform::LIBSUFFIX}" unless lib =~ /#{r}/ or is_object
    end
    lib
  end

This results in the following output

jruby -rffi -e 'module LibC; extend FFI::Library; ffi_lib "c"; attach_function :getpid, [], :pid_t;end; p LibC.getpid'
47842052
@ghost
ghost commented Mar 16, 2013

@jstoneky to get posix full support, we also need to update jnr-posix with struct stat and struct passwd definitions for AIX.

See https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/AixFileStat.java and https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/AixPasswd.java

I copied those from FreeBSD just as place holders - you'll need to examine struct stat (or more likely stat64), and struct passwd on aix and mod those structs to be compatible.

If you can past the contents of /usr/include/sys/stat.h and /usr/include/pwd.h into a gist, I can help you with it.

@ghost
ghost commented Mar 27, 2013

jruby-master has AIX stubs for jffi and jnr-posix support. Closing.

@ghost ghost closed this Mar 27, 2013
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment