Skip to content

Loading…

Failed to build boost 1.52.0 with MPI and ICU #16142

Closed
dazzlezhang opened this Issue · 27 comments

8 participants

@dazzlezhang

Hi,

I tried to install boost 1.52.0 with MPI and ICU via homebrew. Seems that there's a problem when clang linking the icu library. The building log is as follows.

Homebrew 0.9.3
/usr/local/bin/mpicc --version
/usr/local/bin/mpicxx --version
==> Downloading http://downloads.sourceforge.net/project/boost/boost/1.52.0/boost_1_52_0.tar.bz2
Already downloaded: /Library/Caches/Homebrew/boost-1.52.0.tar.bz2
/usr/bin/tar xf /Library/Caches/Homebrew/boost-1.52.0.tar.bz2
==> ./bootstrap.sh --prefix=/usr/local/Cellar/boost/1.52.0 --libdir=/usr/local/Cellar/boost/1.52.0/lib --with-icu=/usr/local/opt/icu4c
./bootstrap.sh --prefix=/usr/local/Cellar/boost/1.52.0 --libdir=/usr/local/Cellar/boost/1.52.0/lib --with-icu=/usr/local/opt/icu4c
-n Building Boost.Build engine with toolset darwin... 
tools/build/v2/engine/bin.macosxx86_64/b2
-n Detecting Python version... 
2.7
-n Detecting Python root... 
/System/Library/Frameworks/Python.framework/Versions/2.7
-n Unicode/ICU support for Boost.Regex?... 
/usr/local/opt/icu4c
Generating Boost.Build configuration in project-config.jam...

Bootstrapping is done. To build, run:

    ./b2

To adjust configuration, edit 'project-config.jam'.
Further information:

   - Command line help:
     ./b2 --help

   - Getting started guide: 
     http://www.boost.org/more/getting_started/unix-variants.html

   - Boost.Build documentation:
     http://www.boost.org/boost-build2/doc/html/index.html

==> ./b2 --prefix=/usr/local/Cellar/boost/1.52.0 --libdir=/usr/local/Cellar/boost/1.52.0/lib -d2 -j4 --layout=tagged --user-config=user-config.jam threading=multi install
./b2 --prefix=/usr/local/Cellar/boost/1.52.0 --libdir=/usr/local/Cellar/boost/1.52.0/lib -d2 -j4 --layout=tagged --user-config=user-config.jam threading=multi install
Performing configuration checks

    - 32-bit                   : no
    - 64-bit                   : yes
    - x86                      : yes
    - has_icu builds           : yes
    - iconv (libc)             : no
    - iconv (separate)         : yes
    - icu                      : yes
    - gcc visibility           : yes
    - long double support      : yes

Component configuration:

    - chrono                   : building
    - context                  : building
    - date_time                : building
    - exception                : building
    - filesystem               : building
    - graph                    : building
    - graph_parallel           : building
    - iostreams                : building
    - locale                   : building
    - math                     : building
    - mpi                      : building
    - program_options          : building
    - python                   : building
    - random                   : building
    - regex                    : building
    - serialization            : building
    - signals                  : building
    - system                   : building
    - test                     : building
    - thread                   : building
    - timer                    : building
    - wave                     : building

...patience...
...patience...
...patience...
...patience...
...found 32191 targets...
...updating 10820 targets...

(skipped many lines here)

    "c++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -dynamic -gdwarf-2 -fexceptions -fPIC  -DBOOST_ALL_NO_LIB=1 -DBOOST_CHRONO_DYN_LINK=1 -DBOOST_LOCALE_DYN_LINK=1 -DBOOST_LOCALE_NO_WINAPI_BACKEND=1 -DBOOST_LOCALE_WITH_ICONV=1 -DBOOST_LOCALE_WITH_ICU=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_THREAD_BUILD_DLL=1 -DBOOST_THREAD_NO_LIB=1 -DBOOST_THREAD_POSIX -DBOOST_THREAD_THROW_IF_PRECONDITION_NOT_SATISFIED -DBOOST_THREAD_USE_DLL=1 -DNDEBUG  -I"." -c -o "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/std/numeric.o" "libs/locale/src/std/numeric.cpp"

darwin.link.dll /usr/local/Cellar/boost/1.52.0/lib/libboost_locale-mt.dylib

    "c++" -dynamiclib -Wl,-single_module -install_name "/usr/local/lib/libboost_locale-mt.dylib"  -o "/usr/local/Cellar/boost/1.52.0/lib/libboost_locale-mt.dylib" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/encoding/codepage.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/date_time.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/format.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/formatting.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/generator.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/ids.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/localization_backend.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/message.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/shared/mo_lambda.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/util/codecvt_converter.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/util/default_locale.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/util/info.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/util/locale_data.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/boundary.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/codecvt.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/collator.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/conversion.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/date_time.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/formatter.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/icu_backend.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/numeric.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/icu/time_zone.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/posix/codecvt.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/posix/collate.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/posix/converter.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/posix/numeric.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/posix/posix_backend.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/std/codecvt.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/std/collate.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/std/converter.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/std/numeric.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/std/std_backend.o" "bin.v2/libs/locale/build/darwin-4.2.1/release/threading-multi/util/gregorian.o" "bin.v2/libs/thread/build/darwin-4.2.1/release/threading-multi/libboost_thread-mt.dylib" "bin.v2/libs/chrono/build/darwin-4.2.1/release/threading-multi/libboost_chrono-mt.dylib" "bin.v2/libs/system/build/darwin-4.2.1/release/threading-multi/libboost_system-mt.dylib"  -licuuc -licui18n -licudata -liconv    -headerpad_max_install_names -Wl,-dead_strip -no_dead_strip_inits_and_terms 

Undefined symbols for architecture x86_64:
  "icu_50::UnicodeString::doReplace(int, int, unsigned short const*, int, int)", referenced from:
      boost::locale::impl_icu::strftime_to_icu(icu_50::UnicodeString const&, icu_50::Locale const&) in formatter.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
...failed darwin.link.dll /usr/local/Cellar/boost/1.52.0/lib/libboost_locale-mt.dylib...
darwin.compile.c++ bin.v2/libs/python/build/darwin-4.2.1/release/threading-multi/long.o

    "c++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -dynamic -gdwarf-2 -fexceptions -fPIC  -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG  -I"." -I"/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7" -c -o "bin.v2/libs/python/build/darwin-4.2.1/release/threading-multi/long.o" "libs/python/src/long.cpp"

darwin.compile.c++ bin.v2/libs/python/build/darwin-4.2.1/release/threading-multi/list.o

(skipped many lines)

@adamv

Ping @mikemcquaid were these bottles updated?

@mikemcquaid
Homebrew member

Yes.

@2bits

@dazzlezhang When that pull request lands, you should be able to do what you want after a brew update. There was an API change in icu4c, fortunately some fallout we were able to workaround. Thanks for the report.

@novocaine

boost built against libc++ is problematic for libraries trying to link to it that dont themselves build with libc++ (i.e. they use libstdc++) so this not really a good solution.

i have been having problems with v8 and pyv8 specifically.

is it possible for brew to patch libicu to avoid building with libc++ as it previously did, rather than changing up the boost package (i think this is what im gonna have to do locally anyway)

@2bits

Hmm I don't want to break things. So forcing c++11 would be bad. I'll have to change #16149. What I can tell you is that the current boost builds just fine with the previous icu4c-49.1.2 and no changes to the boost formula.

cd `brew --prefix`
brew rm -f icu4c boost
brew versions icu4c
git checkout c25fd2f `brew --prefix`/Library/Formula/icu4c.rb
brew install icu4c
brew install --with-icu boost

So at least you have a quick workaround for now. Thanks for the info.

@novocaine

ok, thanks.

i did succeed building latest icu4c 1.5 from source by hacking the configure files to not build with libc++. and then all my other packages are happy - brew install boost --build-from-source is happy, and i can build v8 and pyv8 just fine.

to be clear, building with std=c++11 isn't the problem, the problem is libc++ linkage.

i think the icu maintainers need to add a configure option..

@novocaine

this post illustrates the exact problem that libc++ causes:

http://stackoverflow.com/questions/8454329/why-cant-clang-with-libc-in-c0x-mode-link-this-boostprogram-options-examp

ie once you link boost against libc++, all its exported functions refer to types in libc++ so trying to link using types in libstdc++ fails

@2bits

Could you help us understand what changed from icu4c-49.1.2 to icu4c-50.1? I thought there was an API change, but you are saying that the latter is simply compiled against libc++? Would you please gist the patch you applied to icu4c-50.1 source to get it to build differently so that it would build into boost-1.52? Thanks. I'll read that link shortly.

@novocaine

this seemed to do the trick - total hack though.

*** icu/source/configure    2012-11-05 09:18:12.000000000 -0800
--- icu-hacked/source/configure 2012-11-19 09:40:33.000000000 -0800
***************
*** 7005,7011 ****
      if test "$GXX" = yes; then
          OLD_CXXFLAGS="${CXXFLAGS}"
          # -Wno-return-type-c-linkage is desired so that stable ICU API is not warned about.
-         CXXFLAGS="${CXXFLAGS} -std=c++11"
          ac_ext=cpp
  ac_cpp='$CXXCPP $CPPFLAGS'
  ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'
--- 7005,7010 ----

*** icu/source/configure.in 2012-11-05 09:18:12.000000000 -0800
--- icu-hacked/source/configure.in  2012-11-19 09:39:58.000000000 -0800
***************
*** 972,978 ****
      if test "$GXX" = yes; then
          OLD_CXXFLAGS="${CXXFLAGS}"
          # -Wno-return-type-c-linkage is desired so that stable ICU API is not warned about.
-         CXXFLAGS="${CXXFLAGS} -std=c++11"
          AC_LANG_PUSH([C++])
          AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
  static const char16_t test[] = u"This is a UTF16 literal string.";
--- 972,977 ----

if you look around the configure it aggressively seeks clang++ and sets std=c++11. notably it doesn't appear to set stdlib=libc++, however..

# Make sure that we try clang++ first, which provides C++11 support.
# The g++ compiler is less likely to support C++11.
ac_ext=cpp
ac_cpp='$CXXCPP $CPPFLAGS'
ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'
ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'
ac_compiler_gnu=$ac_cv_cxx_compiler_gnu
if test -z "$CXX"; then
  if test -n "$CCC"; then
    CXX=$CCC
  else
    if test -n "$ac_tool_prefix"; then
  for ac_prog in clang++ g++ c++ gpp xlC_r xlC aCC CC cxx cc++ cl.exe icc FCC KCC RCC
@novocaine

more clearly - the problem is not that icu is linked against libc++, its what i mentioned in my duplicate bugpost - it uses a new c++11 type.

This seems to be because libicuuc has been created with the c++11 type "char16_t":

~ nm /usr/local/Cellar/icu4c/50.1/lib/libicuuc.50.1.dylib | grep doReplace | c++filt
00000000000418bc T icu_50::UnicodeString::doReplace(int, int, char16_t const*, int, int)
0000000000043bd2 T icu_50::UnicodeString::doReplace(int, int, icu_50::UnicodeString const&, int, int)

which apparently doesn't match unsigned short const*.
@josegonzalez
Homebrew member

+1 Upgrading ICU4C broke intl support in PHP for homebrew-php - issues here

For the moment, I'm having people install php54-intl or php53-intl instead of including it with the binary, but since there aren't any post installation hooks for brews, it's kind of difficult...

@mikemcquaid
Homebrew member

Perhaps worth downgrading?

@adamv

Refs #16144.

Worth downgrading because things are broken.

@mikemcquaid
Homebrew member

@adamv Ok, go for it. I can't remember if we need to do anything special when downgrading?

@manphiz

Actually, according to ICU 50.1 release note (http://www.icu-project.org/repos/icu/icu/tags/release-50-1/readme.html#News), it seems this only happens when building with clang with c++11, and will fallback to non c++11 symbols when built with compiler other than clang. Building icu with --use-llvm I get:

$ nm `brew --prefix`/Cellar/icu4c/50.1/lib/libicuuc.50.1.dylib | grep doReplace | c++filt
0000000000045976 T icu_50::UnicodeString::doReplace(int, int, unsigned short const*, int, int)
0000000000045cf0 T icu_50::UnicodeString::doReplace(int, int, icu_50::UnicodeString const&, int, int)

I'm building boost against this version to see if things get better, will report back.

@manphiz

OK, looks like it works. I have opened a pull request at #16302.

@mikemcquaid
Homebrew member

Can/have you filed this upstream? I'm just debating whether the right solution is enabling C++11 for Boost without libc++ or this. I'd personally rather we lean on the side of using newer things rather than older ones as it encourages upstreams to fix things and means we get less problems as time goes on and Apple deprecate or remove things like llvm-gcc (which they will do eventually).

@manphiz

I found 2 relevant bugs in icu bug trac: http://bugs.icu-project.org/trac/ticket/9717 http://bugs.icu-project.org/trac/ticket/9728 , and this changeset http://bugs.icu-project.org/trac/changeset/32902 provides a way to workaround UChar problem, which will be included in icu 51.1. Meanwhile upstream expressed reluctant to disable C++11 support, and stating that there's no C++ compatibility between major releases.

However for the time being I'd suggest disable c++11 in icu, or there will be other subtle problem, in extreme case if linked to libc++ it will cause breakage when linked with other program built against libstdc++. There was a patch that add a switch to conditionally enable c++11 in icu ticket 9717 but got rejected. Falling back to non-C++11 compiler seems as a safe bet for now.

@manphiz

Also, IMHO, enabling -std=c++11 without -stdlib=libc++ is wrong because C++11 support is incomplete or broken without libc++ and will cause more problem when software assumes the current C++11 facilities from Clang are available.

On Linux, Clang depends on C++11 enabled libstdc++, which is consistent with GCC, so it's not a problem.

@manphiz

I have filed another ticket on icu trac for this Mac-specific issue: http://bugs.icu-project.org/trac/ticket/9772.

@novocaine

+1 for llvm-gcc not being a long term solution.

from the Xcode 4.6 release notes:

Xcode 4.6 is the last major Xcode release that will include the llvm-gcc compiler
and the GDB debugger.

@manphiz

This is an upstream problem per se, which is hopefully fixed for 51.1 and set for release next March. There's less can be done for downstreams, and workarounds will be inelegant. Also note that there are other formulas that depends on llvm-gcc or even apple-gcc to be built correctly.

Meanwhile, it is curious how libstdc++ vs. libc++ situation will be handled. IIRC (Correct me if I'm wrong), libc++ doesn't support C++98, and Clang cannot build libstdc++ with GCC 4.2.x. Whether libstdc++ will be dropped or not, the transition will be painful.

@mikemcquaid
Homebrew member

The formulae that depend on llvm-gcc or gcc will be dropped from core if patches aren't written after Xcode 4.7 is released.

@adamv

Since this is an upstream issue with a potential fix in March, closing it from Homebrew's issue tracker since there's nothing we can do in the meanwhile. (Unless someone wants to update the existing formulae to warn and quit when these options are given, or add something to the caveats.)

@adamv adamv closed this
@mbcoguno

Sorry, but http://bugs.icu-project.org/trac/ticket/9772 says milestone isn't this March, do we have solutions?

@manphiz

@mbcoguno Upstream have incorporated a patch that avoids using char16_t, so this problem can be considered fixed.

@mbcoguno

@manphiz Thanks, waiting for release and pull request.

This was referenced
@simeonwillbanks simeonwillbanks pushed a commit to simeonwillbanks/homebrew that referenced this issue
@manphiz manphiz icu4c: 51.1
* Update hashes and drop old bottles.
* Enable build with Clang as [1] shouldn't be happening again.

[1] Homebrew#16142

Closes #19354.

Signed-off-by: Mike McQuaid <mike@mikemcquaid.com>
341b60a
@finn finn pushed a commit that referenced this issue
@manphiz manphiz icu4c: 51.1
* Update hashes and drop old bottles.
* Enable build with Clang as [1] shouldn't be happening again.

[1] #16142

Closes #19354.

Signed-off-by: Mike McQuaid <mike@mikemcquaid.com>
b6c2b0b
@draftycode draftycode added a commit to draftycode/homebrew that referenced this issue
@manphiz manphiz icu4c: 51.1
* Update hashes and drop old bottles.
* Enable build with Clang as [1] shouldn't be happening again.

[1] Homebrew#16142

Closes #19354.

Signed-off-by: Mike McQuaid <mike@mikemcquaid.com>
d678b4c
@shelhamer shelhamer added a commit that referenced this issue
@manphiz manphiz icu4c: 51.1
* Update hashes and drop old bottles.
* Enable build with Clang as [1] shouldn't be happening again.

[1] #16142

Closes #19354.

Signed-off-by: Mike McQuaid <mike@mikemcquaid.com>
1e78f29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.