Installing hiredis-client on macos with M1 fails #58
Comments
|
Hum, works fine on my machine: What does |
|
Thanks for the feedback.
Just tried with 2.7.6: I get the same error when building as with ruby 2.7.5. When I use the x86_64-darwin21 architecture, it works.. |
|
Hum, that's interesting, I'll try to reproduce. What you pasted suggest your ruby was built with How did you build that ruby and did you Also could you run |
|
I'm seeing the same issue: $ gem install hiredis-client -v 0.11.0 -- --with-openssl-dir=/opt/homebrew/opt/openssl@3
Building native extensions with: '--with-openssl-dir=/opt/homebrew/opt/openssl@3'
This could take a while...
ERROR: Error installing hiredis-client:
ERROR: Failed to build gem native extension.
current directory: /Users/arian/.rvm/gems/ruby-3.1.2@my-app/gems/hiredis-client-0.11.0/ext/redis_client/hiredis
/Users/arian/.rvm/rubies/ruby-3.1.2/bin/ruby -I /Users/arian/.rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0 -r ./siteconf20221101-81291-r5ixvl.rb extconf.rb --with-openssl-dir\=/opt/homebrew/opt/openssl@3
checking for rb_hash_new_capa() in ruby.h... no
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb alloc.c
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb net.c
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb hiredis.c
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb sds.c
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb async.c
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb read.c
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb sockcompat.c
ar rcs libhiredis.a alloc.o net.o hiredis.o sds.o async.o read.o sockcompat.o
cc -std=c99 -pedantic -c -O3 -fPIC "-I/opt/homebrew/opt/openssl@3/include" -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb ssl.c
ar rcs libhiredis_ssl.a ssl.o
creating Makefile
current directory: /Users/arian/.rvm/gems/ruby-3.1.2@my-app/gems/hiredis-client-0.11.0/ext/redis_client/hiredis
make DESTDIR\= clean
current directory: /Users/arian/.rvm/gems/ruby-3.1.2@my-app/gems/hiredis-client-0.11.0/ext/redis_client/hiredis
make DESTDIR\=
compiling hiredis_connection.c
linking shared-object redis_client/hiredis_connection.bundle
Undefined symbols for architecture arm64:
"_ruby_abi_version", referenced from:
-exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [hiredis_connection.bundle] Error 1
make failed, exit code 2
Gem files will remain installed in /Users/arian/.rvm/gems/ruby-3.1.2@my-app/gems/hiredis-client-0.11.0 for inspection.
Results logged to /Users/arian/.rvm/gems/ruby-3.1.2@my-app/extensions/arm64-darwin-22/3.1.0/hiredis-client-0.11.0/gem_make.out$ ruby -e "puts RbConfig::CONFIG['CC']"
gcc |
|
@arianf thanks for the report, can you run |
$ cc --version
Apple clang version 14.0.0 (clang-1400.0.29.201)
Target: arm64-apple-darwin22.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin |
|
Oh that's a weird one, even macOS 13 (Ventura) is supposed to ship with I'll try to see if that's what cause the issue, not sure how I can upgrade though. |
|
See an similar issue here: grpc/grpc#30976 |
|
Yeah I know, @peterzhu2118 pointed the same GRPC error to me. I suppose we now know that clang-14 is the trigger. Now the question is how to fix it. |
|
I'm not able to reproduce this issue on Xcode 14 either. There is an upstream Ruby bug report about this, though: https://bugs.ruby-lang.org/issues/19005 |
|
Thank you very much @stanhu. If I understood that ticket correctly, ruby interpreters compiled with Xcode 14 have some missing flags in @dup2 / @arianf I'd appreciate if you could share the value of I'll try to repro and provide a fix tomorrow. |
|
Oh, I forgot I had my compiler set with When I used Ruby 3.1.2 without those flags, it fails on XCode 14. irb(main):001:0> RbConfig::CONFIG['DLDFLAGS']
=> "-L/Users/stanhu/.asdf/installs/ruby/3.1.2/lib -Wl,-multiply_defined,suppress" |
|
Upgrading from Ruby |
Yes, that's a workaround, but I believe gems should be able to be compiled without those flags, though that's not a commonly-tested path. Examples of other fixes: |
Ruby 3.2.0-preview2 ( 3.2.0-preview2 :001 > p RbConfig::CONFIG['DLDFLAGS']
"-Wl,-undefined,dynamic_lookup -Wl,-multiply_defined,suppress"Ruby 3.1.2 ( 3.1.2 :001 > p RbConfig::CONFIG['DLDFLAGS']
"-L/opt/homebrew/opt/libyaml/lib -L/opt/homebrew/opt/libksba/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/openssl@1.1/lib -Wl,-multiply_defined,suppress"Ruby 3.0.4 ( 3.0.4 :001 > p RbConfig::CONFIG['DLDFLAGS']
"-L/opt/homebrew/opt/libyaml/lib -L/opt/homebrew/opt/libksba/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/openssl@1.1/lib -Wl,-multiply_defined,suppress" |
|
Dropping this line fixed the compilation issue for me: diff --git a/hiredis-client/ext/redis_client/hiredis/export.clang b/hiredis-client/ext/redis_client/hiredis/export.clang
index 10065f7..883acc2 100644
--- a/hiredis-client/ext/redis_client/hiredis/export.clang
+++ b/hiredis-client/ext/redis_client/hiredis/export.clang
@@ -1,2 +1 @@
_Init_hiredis_connection
-_ruby_abi_version |
Yeah, because it doesn't exist on 3.1 and older, but then that will break on 3.2.0 when this field will need to be exported. @peterzhu2118 can correct me if I'm wrong on that one. |
|
I wonder if another option is to ignore the missing symbol via: diff --git a/hiredis-client/ext/redis_client/hiredis/extconf.rb b/hiredis-client/ext/redis_client/hiredis/extconf.rb
index 39204c3..a7c13c1 100644
--- a/hiredis-client/ext/redis_client/hiredis/extconf.rb
+++ b/hiredis-client/ext/redis_client/hiredis/extconf.rb
@@ -45,6 +45,7 @@ if RUBY_ENGINE == "ruby" && !RUBY_PLATFORM.match?(/mswin/)
$CFLAGS << " -I#{hiredis_dir}"
$LDFLAGS << " -lssl -lcrypto"
+ $DLDFLAGS << " -Wl,-U,_ruby_abi_version"
$libs << " #{hiredis_dir}/libhiredis.a #{hiredis_dir}/libhiredis_ssl.a "
$CFLAGS << " -std=c99 "
if ENV["EXT_PEDANTIC"] |
|
Yeah, this export is required for Ruby 3.2 compatibility. I wonder if this is a regression in xcode 14.1 beta. I remember being able to repro this issue in gRPC but I'm now unable to. Looking at @arianf's output, it seems like you're on xcode 14.1 beta 3, whereas I'm on the release version of xcode 14.1 (released today, Nov 1 2022) with a newer build of clang: I will try to downgrade to xcode 14.1 beta 3 tomorrow and see if I can repro this issue again. |
As I mentioned in the Ruby bug report, XCode 14.0 now warns with the message when % gcc -o test -Wl,-undefined,dynamic_lookup test.c
ld: warning: -undefined dynamic_lookup may not work with chained fixupsThis warning was enough to make the Ruby I believe the patches in https://github.com/ruby/ruby/pull/6441/files fix the |
|
Thanks for your quick reaction, lads - much appreciated!
|
Fix: #58 Ref: https://bugs.ruby-lang.org/issues/19005 Rubies compiled with Xcode 14 are missing some flags in LDFLAGS which cause the hiredis gem to fail to compile. We work around this by adding the flags back if we detect this bug.
|
Ok, so I made a PR with what I think is a decent workaround: #59 I'd appreciate if someone who experience this bug could give that branch a try to confirm the workaround doesn't only work on my machine. gem "hiredis-client", github: "redis-rb/redis-client"Also I'm far from a linker and compiler expert, so if you believe that is a better way to handle this, I'm all ears. |
Thanks! Unfortunately, this does not work for me. Most likely because the check for |
Fix: #58 Ref: https://bugs.ruby-lang.org/issues/19005 Rubies compiled with Xcode 14 are missing some flags in LDFLAGS which cause the hiredis gem to fail to compile. We work around this by adding the flags back if we detect this bug.
|
Thank you @dup2. I modified my PR a bit more to try to account for this. Could you try again? |
Thanks @casperisfine - I get another error now: Please note also that the reference in the Gemfile had to be like this to test the commit in the linked PR #59 : otherwise just the master branch would have been used.. |
|
Hum, interesting, IMO that's a bug in I pushed another commit. And yes indeed I forgot to add the |
|
Is there a reason to use a symbol list (like #58 (comment)) in this gem? Dropping that list would be the most future-proof solution and easiest to maintain. The symbol list will also make the gem not build on TruffleRuby for instance which uses a different symbol (which predates |
|
FYI I've filed https://bugs.ruby-lang.org/issues/19289 on the Ruby side to see how we could improve things related to |
Yes, it's to allow both I wish I didn't have to do this, but the two gems might need to co-exist for a while. If you know of a better way to fix that root cause I'd be happy to switch, otherwise, I might remove that hack in a year or so when I can expect most people will have migrated out of the |
|
The same error exists for Update to version 0.12.0 fixed the problem. |
|
Yes, the issue was never specific to M1. |
|
🤦 that makes so much sense... |
|
Damn, somehow I also didn't think about that, but it's definitely cleaner than checking the version and patchlevel. |
|
Maybe I'm doing something wrong, but it doesn't seem to work: |
|
@casperisfine Could you try without the parens too, i.e., |
|
As usual the |
|
without parens: with parens: |
|
Hum, wait: it's looking into my final |
Did you configure it with |
|
Rather I recommend decorating |
|
@nobu your solution works great, but I tried it and I don't think it works in this case: $ nm hiredis-client/lib/redis_client/hiredis_connection.bundle
...
000000000000dab8 T _redisAsyncConnectWithOptions
000000000000e4f4 T _redisAsyncDisconnect
000000000000faf8 T _redisAsyncFormattedCommand
000000000000df08 T _redisAsyncFree
000000000000ed44 T _redisAsyncHandleRead
000000000000f0f8 T _redisAsyncHandleTimeout
000000000000f0a8 T _redisAsyncHandleWrite
000000000000ec40 T _redisAsyncRead
0000000000012460 t _redisAsyncRead.cold.1
...I think the reason is that we first compile the $libs << " #{hiredis_dir}/libhiredis.a #{hiredis_dir}/libhiredis_ssl.a "If I try to compile Not sure if there is a way to fix this, I'm really not good enough with linkers and such to really figure that out. I'd love to make it work that way though, it seems much cleaner and simpler. |
|
Nevermind, I think i figured it out. |
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
Rather than to use an exported symbols list, we compile with -fvisibility=hidden and mark the init function with RUBY_FUNC_EXPORTED. This was suggested by @nobu. Ref: #58 (comment)
When compiling Ruby 3.2.0 on an Apple M1, this error results:
```
compiling ../../../../ext/oj-introspect/introspect.c
linking shared-object oj/introspect/introspect_ext.bundle
Undefined symbols for architecture arm64:
"_ruby_abi_version", referenced from:
-exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
```
`_ruby_abi_version` is only exported on development builds
(redis-rb/redis-client#58 (comment)).
When compiling Ruby 3.2.0 on an Apple M1, this error results:
```
compiling ../../../../ext/oj-introspect/introspect.c
linking shared-object oj/introspect/introspect_ext.bundle
Undefined symbols for architecture arm64:
"_ruby_abi_version", referenced from:
-exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
```
`_ruby_abi_version` is only exported on development builds
(redis-rb/redis-client#58 (comment)).
* Fix _ruby_abi_version symbol not found with Ruby 3.2.0
When compiling Ruby 3.2.0 on an Apple M1, this error results:
```
compiling ../../../../ext/oj-introspect/introspect.c
linking shared-object oj/introspect/introspect_ext.bundle
Undefined symbols for architecture arm64:
"_ruby_abi_version", referenced from:
-exported_symbol[s_list] command line option
ld: symbol(s) not found for architecture arm64
```
`_ruby_abi_version` is only exported on development builds
(redis-rb/redis-client#58 (comment)).
* Restrict symbol handling to macOS
Closes #4
Trying to install this as part of a bundle fails
Ruby: ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [arm64-darwin21]
OS: macos 12.6.1 / Apple M1 Pro
The text was updated successfully, but these errors were encountered: