unknown error - Failed to initialize Augeas (SystemCallError)

[HarryWeppner]

On OS X 10.10 (Yosemite) using augeas 1.1.0 installed via MacPorts and ruby-augeas 0.5.0 ruby bindings fail with an unknown error - Failed to initialize Augeas (SystemCallError).

This can be reproduced via a simple test program such as:

require 'Augeas'
Augeas::open { |aug| aug.get("/files/etc/apache2") }  

It fails with:

$ ruby test.rb
/Users/harry/.rvm/gems/ruby-2.1.2@puppet/gems/ruby-augeas-0.5.0/lib/Augeas.rb:48:in `open3': unknown error - Failed to initialize Augeas (SystemCallError)
    from /Users/harry/.rvm/gems/ruby-2.1.2@puppet/gems/ruby-augeas-0.5.0/lib/Augeas.rb:48:in `open'
    from test.rb:2:in `<main>'

I debugged into it and it fails because dyld resolves hash_create to libsystem_c.dylib and not libfa.dylib as illustrated by this backtrace.

$ lldb ruby
(lldb) target create "ruby"
Current executable set to 'ruby' (x86_64).
(lldb) br s --shlib libsystem_c.dylib --name hash_create
Breakpoint 1: where = libsystem_c.dylib`hash_create, address = 0x00000000000640a9
(lldb) run test.rb
Process 19553 launched: '/Users/harry/.rvm/rubies/ruby-2.1.2/bin/ruby' (x86_64)
Process 19553 stopped
* thread #1: tid = 0x39e3aa, 0x00007fff8573e0a9 libsystem_c.dylib`hash_create, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x00007fff8573e0a9 libsystem_c.dylib`hash_create
libsystem_c.dylib`hash_create:
-> 0x7fff8573e0a9:  pushq  %rbp
   0x7fff8573e0aa:  movq   %rsp, %rbp
   0x7fff8573e0ad:  pushq  %r14
   0x7fff8573e0af:  pushq  %rbx
(lldb) bt
* thread #1: tid = 0x39e3aa, 0x00007fff8573e0a9 libsystem_c.dylib`hash_create, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
  * frame #0: 0x00007fff8573e0a9 libsystem_c.dylib`hash_create
    frame #1: 0x0000000100f60a07 libfa.1.dylib`state_set_hash_add + 53
    frame #2: 0x0000000100f5b6bb libfa.1.dylib`determinize + 186
    frame #3: 0x0000000100f5bbb8 libfa.1.dylib`fa_complement + 29
    frame #4: 0x0000000100f5bd6b libfa.1.dylib`fa_minus + 74
    frame #5: 0x0000000100f1d7e4 libaugeas.0.dylib`regexp_minus + 123
    frame #6: 0x0000000100f10547 libaugeas.0.dylib`compile_exp + 1282
    frame #7: 0x0000000100f1026a libaugeas.0.dylib`compile_exp + 549
    frame #8: 0x0000000100f0f502 libaugeas.0.dylib`load_module_file + 1037
    frame #9: 0x0000000100f0fbdb libaugeas.0.dylib`load_module + 287
    frame #10: 0x0000000100f0ddb2 libaugeas.0.dylib`lookup_internal + 129
    frame #11: 0x0000000100f120b2 libaugeas.0.dylib`ctx_lookup_bnd + 155
    frame #12: 0x0000000100f0eb85 libaugeas.0.dylib`check_exp + 713
    frame #13: 0x0000000100f0f243 libaugeas.0.dylib`load_module_file + 334
    frame #14: 0x0000000100f0fbdb libaugeas.0.dylib`load_module + 287
    frame #15: 0x0000000100f0fa36 libaugeas.0.dylib`interpreter_init + 447
    frame #16: 0x0000000100f029af libaugeas.0.dylib`aug_init + 1170
    frame #17: 0x00000001009e6f50 _augeas.bundle`augeas_init + 96
    frame #18: 0x000000010028460b ruby`vm_call_cfunc + 1067
    frame #19: 0x0000000100283f44 ruby`vm_call_method + 868
    frame #20: 0x000000010026d213 ruby`vm_exec_core + 12643
    frame #21: 0x000000010027acf1 ruby`vm_exec + 129
    frame #22: 0x000000010027bb78 ruby`rb_iseq_eval_main + 392
    frame #23: 0x0000000100144d54 ruby`ruby_exec_internal + 148
    frame #24: 0x0000000100144c7e ruby`ruby_run_node + 78
    frame #25: 0x00000001000012cf ruby`main + 79
    frame #26: 0x0000000100001274 ruby`start + 52
(lldb) 

If I force loading libfa.dylib first via export DYLD_INSERT_LIBRARIES=/opt/local/lib/libfa.dylib it resolves correctly and the test program no longer causes an exception. This confirmed that it's caused by how dyld resolves hash_create.

Looking at libfa.dylib the function hash_create is in fact defined.

$ nm -m /opt/local/lib/libfa.dylib | grep hash_create
0000000000009d62 (__TEXT,__text) external _hash_create

but disassembling libfa.dylib shows that there's a stub to call hash_create from state_set_hash_add instead of a direct call to hash_create!?

...
0000000000009a02    callq   0x1f5d2 ## symbol stub for: _hash_create
...

NOTE: if compiled manually from the release tarball, i.e. not via MacPorts, the calls to hash_create are direct as expected (and thus avoids this problem altogether).

otool -tV /usr/local/lib/libfa.1.dylib | grep hash_create
0000000000005890    callq   _hash_create

[HarryWeppner]

Issue turned out to be a known problem with configure scripts generated by old versions of libtool when MACOSX_DEPLOYMENT_TARGET is set to "10.10".

https://trac.macports.org/changeset/129962 fixes the issue.

[larryv]

You can read my dissections elsewhere, but the gist is that the sloppy parsing of MACOSX_DEPLOYMENT_TARGET used by old versions of Libtool broke once the OS X minor version went into the double digits.

The bug was fixed in the recent Libtool 2.4.3 release, but any configure scripts generated by older Libtool still have the bug. The best fix is to regenerate your next release's configure using a recent Libtool.