Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fails on macOS Sierra (10.12.6) due to undefined symbols #263

Closed
lovebug356 opened this issue Oct 13, 2017 · 27 comments
Closed

Build fails on macOS Sierra (10.12.6) due to undefined symbols #263

lovebug356 opened this issue Oct 13, 2017 · 27 comments

Comments

@lovebug356
Copy link

Building git2-rs failes to build on my system, a macOS Sierra 10.12.6. (However, I see that travis correctly builds the same checkout)

Some system information:

$ rustc --version
rustc 1.21.0 (3b72af97e 2017-10-09)
$ rustup --version
rustup 1.6.0 ( )
$ cargo --version
cargo 0.22.0 (3423351a5 2017-10-06)
$ cc --version
Apple LLVM version 9.0.0 (clang-900.0.37)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

The cli-output:

$ cargo test
   Compiling libgit2-sys v0.6.16 (file:///Users/tvermeir/devel/rust-sandbox/git2-rs/libgit2-sys)
   Compiling git2 v0.6.8 (file:///Users/tvermeir/devel/rust-sandbox/git2-rs)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-L" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/git2-607ffebb677e754d.0.o" "-o" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/git2-607ffebb677e754d" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/git2-607ffebb677e754d.crate.allocator.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps" "-L" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/build/libgit2-sys-4d562b53da46a05e/out/lib" "-L" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/libbitflags-05a76aa5d1c1cf07.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libtest-db9e5401f64b6e8c.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/liburl-66962c7e15a9d3db.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/libpercent_encoding-e8491e011d12f115.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/libidna-0b5a450752e57dd0.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/libunicode_bidi-36e10feff83f59e7.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/liblibgit2_sys-89dbd6098f7d321f.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/libtempdir-2d0a2dcb33406b89.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libgetopts-2f170e25267c318b.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/libmatches-1ebd31cb2b60a18b.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/libunicode_normalization-50b1c2d2498198a6.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/liblibz_sys-f1d6b4bae61f4d14.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/librand-58382db5cc56e62c.rlib" "/Users/tvermeir/devel/rust-sandbox/git2-rs/target/debug/deps/liblibc-ffafbea21f5b9743.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libterm-ce9e0ebf918666dd.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-a4cb6a5cdd439628.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-518d4b2a103f523e.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-bf40ace258e8c400.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-ae44887349ff2666.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_system-232b33d68f8d8793.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-547714683d755c8a.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-ee8c941dfe26cb03.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd_unicode-ed28cc142b5072c0.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librand-d0d8801a35685ad6.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-2e89074855638f2b.rlib" "/Users/tvermeir/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-3cf04f5feabf9eee.rlib" "-l" "iconv" "-framework" "Security" "-framework" "CoreFoundation" "-l" "z" "-l" "System" "-l" "resolv" "-l" "pthread" "-l" "c" "-l" "m"
  = note: Undefined symbols for architecture x86_64:
            "_libiconv", referenced from:
                _git_path_iconv in liblibgit2_sys-89dbd6098f7d321f.rlib(path.c.o)
            "_libiconv_close", referenced from:
                _git_path_iconv_clear in liblibgit2_sys-89dbd6098f7d321f.rlib(path.c.o)
            "_libiconv_open", referenced from:
                _git_path_iconv_init_precompose in liblibgit2_sys-89dbd6098f7d321f.rlib(path.c.o)
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)
          

error: aborting due to previous error

error: linking with `cc` failed: exit code: 1
  |
error: linking with `cc` failed: exit code: 1
... (and much more of the same)
@alexcrichton
Copy link
Member

Do you have a custom build of libiconv somewhere perhaps? It looks like -liconv is on the linker line...

@lovebug356
Copy link
Author

Not that I'm aware of. I'm running this in an up-to-date homebrew environment. I have a another mac I can try it on.

@alexcrichton
Copy link
Member

Sorry I'm not really sure what's going on here :(

@lilyball
Copy link

I'm getting this same error when trying to compile exa (see ogham/exa#353) on macOS 10.13. Interestingly, cloning git2-rs and running cargo test works just fine, even when I check out an older version (such as 0.6.8 from this ticket, or 0.6.4 which exa is using).

@lilyball
Copy link

Oh now that's interesting. cargo test --no-default-features in git2-rs reproduces the issue.

@lilyball
Copy link

Adding either of the ssh or https features works around this problem (adding curl does not).

@lilyball
Copy link

Curiouser and curiouser. /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libiconv.2.tbd declares the following:

  - archs:           [ i386 ]
    symbols:         [ _libiconv, _libiconv_close, _libiconv_open, _libiconvctl, 
                       _libiconvlist ]
  - archs:           [ i386, x86_64 ]
    symbols:         [ ___iconv_2VersionNumber, ___iconv_2VersionString, __libiconv_version, 
                       _aliases2_lookup, _aliases_lookup, _iconv, _iconv_canonicalize, 
                       _iconv_close, _iconv_open, _iconvctl, _iconvlist, _libiconv_relocate, 
                       _libiconv_set_relocation_prefix, _locale_charset, _utf8_decodestr, 
                       _utf8_encodestr, _utf_extrabytes ]

The 3 missing symbols (_libiconv, _libiconv_close, and _libiconv_open) are declared as being present in i386 but not x86_64. But the real weird part is the libgit2 source is using the non-lib-prefixed versions, e.g.

rv = iconv(ic->map, &nfd, &nfdlen, &nfc, &nfclen);

So why is it trying to link against the 32-bit libiconv* functions?

@lilyball
Copy link

lilyball commented Feb 28, 2018

Great, it's finding /opt/local/include/iconv.h, which changes the symbols (if LIBICONV_PLUG isn't set, which it isn't), but actually trying to link against the system-provided libiconv. I found this in the build folder's CMakeCache.txt where it says

ICONV_INCLUDE_DIR:PATH=/opt/local/include

Though this doesn't explain why adding the ssh or https features fixes the problem.

@lilyball
Copy link

At this point I'm going to stop debugging. I just ran port uninstall cmake; nix-env -iA nixpkgs.cmake to install CMake from Nix instead of MacPorts and the issue has now gone away. Presumably CMake was including /opt/local/include in the search paths because that's where it itself lives, although that doesn't really make too much sense. It's likely that passing some argument to FIND_PATH in the CMake module that locates libiconv will fix this. Or alternatively, having that module just check if (APPLE) and hard-coding the paths instead of searching for them. Both of these changes of course have to be done in libgit2 itself.

In any case, since I've now worked around the problem locally, I can't do anything else on this front.

@freesamael
Copy link

I hit the same error and I found it was because there's another copy of libiconv installed under /Library/Frameworks/libiconv.framework. I have no idea what app installed that. Removing the libiconv.framework works for me.

It's a bit annoying that there are so many different places to check on macOS.

@photis
Copy link

photis commented May 2, 2018

I got the same error trying to cargo build grin (https://github.com/mimblewimble/grin) on macOS High Sierra 10.13.4 and you guys explained my problem succinctly. I followed @freesamael's suggestion and that solved it. It seems to be a rust/cargo problem, rather than a specific application's problem. Thanks a lot

@tbrand
Copy link

tbrand commented Aug 14, 2018

I faced the same situation for cargo-update but I cannot find /Library/Frameworks/libiconv.framework.

  • OS X 10.13.6
  • rustc 1.30.0-nightly (73c78734b 2018-08-05)

Does anyone solve this issue by another way?

@photis
Copy link

photis commented Aug 14, 2018

Did you search for libiconv.framework in any other (sub-)directories on the Mac? If you use the Finder, don't forget to enable looking for System files and Hidden files as well.

@tbrand
Copy link

tbrand commented Aug 16, 2018

Thanks for the advise. Yeah I'd searched by find command for whole directories.
I misunderstood that my error is not directory related to this issue.
It's related to libssh2 and openssl.

I have no idea how to fix it now.
So I'll dig other issues.

(Sorry for missing my logs. I'm using another PC without the problem.)

bcully referenced this issue in fink/fink-distributions Oct 30, 2018
dyld: Symbol not found: _iconv
dyld: Symbol not found: _iconv
  Referenced from: /usr/lib/libcups.2.dylib
  Expected in: /sw/lib/libiconv.2.dylib
 in /usr/lib/libcups.2.dylib
  Referenced from: /usr/lib/libcups.2.dylib
  Expected in: /sw/lib/libiconv.2.dylib
 in /usr/lib/libcups.2.dylib

some context in https://github.com/alexcrichton/git2-rs/issues/263
@behnam
Copy link

behnam commented Jan 2, 2019

I've been having this problem for a while and still hasn't been able to find a stable configuration for getting around it.

The main problem is that I'm not sure what's the supported configuration on macos. Specifically, is git2-rs expect to find libgit2 via MacPorts, or from another source?

@alexcrichton
Copy link
Member

It's expected that git2-rs builds its own copy of libgit2 and doesn't use the system version, as the system version is likely incompatible

@alexcrichton
Copy link
Member

I think this has likely long since been fixed, so I'm going to close.

@jpgoldberg
Copy link

For what it is worth, if people still search for this issue (as I have), here is what worked for me:

In my .profile, I now have

## Rust flags to avoid conflict with MacPorts libiconv
export RUSTFLAGS="-L/usr/lib/libiconv"

It may suck to end up looking in that lib first, even for things that don't depend on -liconv, but I'm happy to have the commuter do a tiny bit of extra build for each build then for me to figure out which cases this is needed in.

I've also filed a MacPorts request for a libiconv-select to help deal with the problem as it interacts with other things beyond rust.

@valin4tor
Copy link

I think this should be re-opened as I've also encountered the issue when attempting to install cargo-edit and cargo-outdated on macOS Big Sur, with libiconv installed via MacPorts.

Setting RUSTFLAGS as suggested in #263 (comment) did not work for me since this version of macOS does not provide /usr/lib/libiconv.a however the following workaround was successful: https://stackoverflow.com/a/34140282/4247209

@rdbisme
Copy link

rdbisme commented Feb 20, 2021

I also think this should be reopened. I'm hitting the same problem trying to build pyOxidizer. I have libiconv installed by Macports and I think whatever logic libgit2-sys is using to find libiconv, it's buggy.

@rdbisme
Copy link

rdbisme commented Apr 15, 2021

Ehi @kencu. By your you mean me specifically or git2-rs?

@kencu
Copy link

kencu commented Apr 15, 2021

Hi @rubendibattista -- I realized the comment was just a duplicate of one above, where this was already identified.

s/a rust-lang/cargo#1032

@rdbisme
Copy link

rdbisme commented Apr 15, 2021

@kencu Sorry if I don't follow, but I don't understand why changing the linker options is needed.

My questions:

  • What's different with Macports libiconv?
  • So the problem is that the build script of git2-rs (or its dependencies) is broken since it links against non-default libiconv in Macports path. Is that it?

@kencu
Copy link

kencu commented Apr 18, 2021

On purpose, the libiconv headers and libraries in /usr and /opt use different sets of symbols:

% nm -a /usr/lib/libiconv.dylib | grep iconv
00000000000f2710 D __libiconv_version
0000000000002390 T _iconv
0000000000014170 s _iconv_2VersionNumber
0000000000014140 s _iconv_2VersionString
00000000000026ab T _iconv_canonicalize
00000000000023b2 T _iconv_close
0000000000001095 T _iconv_open
00000000000023bf T _iconvctl
00000000000024b4 T _iconvlist
0000000000013fac T _libiconv_set_relocation_prefix

% nm -a /opt/local/lib/libiconv.dylib | grep iconv
00000000000f60c0 D __libiconv_version
0000000000002ae9 T _iconv_canonicalize
0000000000002353 T _libiconv
0000000000002375 T _libiconv_close
0000000000001058 T _libiconv_open
0000000000002382 T _libiconv_open_into
0000000000015b87 t _libiconv_relocate
0000000000015c59 t _libiconv_relocate2
0000000000015ad1 T _libiconv_set_relocation_prefix
00000000000027f8 T _libiconvctl
00000000000028f0 T _libiconvlist

This is specifically done by libiconv to prevent you from using one set of headers with the other (non-matching) library.

So if your build is somehow set up to find the headers in one place, but link the library from the other place, you get an error that you are supposed to fix in your build system.

I suspect that way back in the day the libiconv people received a bazillion bug reports that in the end were caused by mismatched header/library set, and said to themselves "We'll fix this once and for all!".

It's a Good Thing, all in all, although it can be frustrating to actually fix your build system.

For MacPorts, I suspect the fact that want to use openssl (in /opt/local) but not libiconv (but it is also in /opt/local) is the source of our troubles. libiconv is not tucked away, the header is right out there:

 /opt/local/include/iconv.h

and so we pick that header up when we don't actually want it.

@kencu
Copy link

kencu commented Apr 18, 2021

we could hack in a blocker -- some define we add to iconv.h like:

MACPORTS_DONT_USE_OUR_LIBICONV

that, if defined, would stop our iconv.h from ever being loaded, and pass through to the system iconv.h instead. That kind of hack is really really hard to sell to the MacPorts team, and for good reason :>

@kencu
Copy link

kencu commented Apr 18, 2021

Oh, and you may wonder why this only happens with MacPorts and not with the main competition: the main competition puts everything in /usr/local/include and /usr/local/lib, and because those locations are always automatically searched by the compiler no matter what the build system does or does not do, the errors never show up there, as there is always a match between header and library.

@rdbisme
Copy link

rdbisme commented Apr 18, 2021

@kencu This is all very cool and correct IMO. But... Is it then that libgit-sys build script is broken? Since it finds a different library to link against? I checked the script, I'm no expert, but It seems to me that they skip cmake altogether and probably the call here doesn't work as expected.

Don't know if it's a specific build.rs problem or a cargo problem.

facebook-github-bot pushed a commit to facebook/sapling that referenced this issue May 6, 2021
Summary:
eagerepo -> metalog -> git2 -> libgit2-sys -> libgit2 conflicts with edenfs'
non-Rust libgit2 dependency. Rust git2 crate does not seem to provide a way to
depend on specified libgit2.

Quote rust-lang/git2-rs#263 (comment):

> It's expected that git2-rs builds its own copy of libgit2 and doesn't use the
> system version, as the system version is likely incompatible

It also seems non-trivial to make buck C++ use the libgit2 frm `libgit2-sys` crate.

Let's just avoid depending on eagerepo from edenapi directly for now to solve the
issue. This basically revives D27948369 and D27951632.

Reviewed By: xavierd

Differential Revision: D28243784

fbshipit-source-id: 0c38c20c2d3a80c550732129da572fe26a229799
SharkBaitDLS pushed a commit to SharkBaitDLS/dotfiles that referenced this issue Mar 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests