Boost linker warnings on OSX

[mcraveiro]

Hi vcpkg developers,

Now that I have my setup working reliably for OSX, Linux and Windows, I have a small nit to pick :-) I am seeing a lot of warnings on OSX like these [1]:

ld: warning: direct access in function 'void boost::throw_exception(boost::bad_weak_ptr const&)' from file '/tmp/vcpkg-export/installed/x64-osx/lib/libboost_log.a(text_file_backend.o)' to global weak symbol 'typeinfo for boost::exception_detail::clone_impl >' from file '/tmp/vcpkg-export/installed/x64-osx/lib/libboost_thread.a(thread.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
...
ld: warning: direct access in function 'boost::exception_detail::clone_impl >::rethrow() const' from file '/tmp/vcpkg-export/installed/x64-osx/lib/libboost_log.a(text_file_backend.o)' to global weak symbol 'typeinfo for boost::exception_detail::clone_impl >' from file '/tmp/vcpkg-export/installed/x64-osx/lib/libboost_thread.a(thread.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.

Apparently these are caused by differing visibility settings [2] between my build and the build of Boost [3]. As per those comments, I have tried setting my build to fvisibility=hidden and fvisibility=default, with neither helping. This ticket [4] suggests recompiling Boost with:

cxxflags=-fvisibility=hidden
cxxflags=-fvisibility-inlines-hidden

I will try that next when I have some down time; meanwhile, has anyone experienced/solved this issue by any chance? To be honest, its nothing major but I thought I'd ask.

Many thanks for your time.

[1] https://my.cdash.org/viewBuildError.php?type=1&buildid=1545212
[2] https://gcc.gnu.org/wiki/Visibility
[3] https://stackoverflow.com/questions/36567072/why-do-i-get-ld-warning-direct-access-in-main-to-global-weak-symbol-in-this
[4] https://stackoverflow.com/questions/8685045/xcode-with-boost-linkerid-warning-about-visibility-settings/8685088#8685088

[mcraveiro]

Quite remarkable, I did that change and I now have zero warnings on OSX! Can't quite believe it! :-) However, I'm not sure this is suitable for submission - not quite sure what the impact of this change is really. For what it's worth, here's the patch to my local tree:

diff --git a/ports/boost-modular-build-helper/CMakeLists.txt b/ports/boost-modular-build-helper/CMakeLists.txt
index dc4d13d..b0fe964 100644
--- a/ports/boost-modular-build-helper/CMakeLists.txt
+++ b/ports/boost-modular-build-helper/CMakeLists.txt
@@ -122,6 +122,9 @@ add_custom_target(boost ALL
         --hash
         -q

+        cxxstd=14
+        cxxflags=-fvisibility=hidden
+        cxxflags=-fvisibility-inlines-hidden
         architecture=x86
         threading=multi
         threadapi=pthread
diff --git a/toolsrc/CMakeLists.txt b/toolsrc/CMakeLists.txt
index a06cbae..834a459 100644
--- a/toolsrc/CMakeLists.txt
+++ b/toolsrc/CMakeLists.txt
@@ -2,7 +2,7 @@ cmake_minimum_required(VERSION 3.3)
 project(vcpkg C CXX)

 OPTION(DEFINE_DISABLE_METRICS "Option for disabling metrics" OFF)
 if(CMAKE_COMPILER_IS_GNUXX OR CMAKE_CXX_COMPILER_ID MATCHES "GNU")
     set(GCC 1)
 elseif(CMAKE_CXX_COMPILER_ID MATCHES "AppleClang")

I'm only doing this on OSX at present but I may look into doing it for Linux in the future when I fully understand the implications :-)

[mcraveiro]

Actually, its probably worth mentioning that my project's CMakeLists.txt has the following:

    <snip>
    set(other_flags "${other_flags} -frtti -fvisibility-inlines-hidden")
    set(other_flags "${other_flags} -fvisibility=hidden")
    <snip>
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${other_flags}")

And that's why it worked - basically, all I did was to make vcpkg's Boost match my project's visibility settings. Your setup may differ.

[mcraveiro]

One slight downside of the previous approach is that it required changes to vcpkg setup. I didn't want to have my own fork, so I ended up using a slightly different approach:

    if(APPLE)
        set(other_flags "${other_flags} -fvisibility=default")
    else()
        set(other_flags "${other_flags} -fvisibility-inlines-hidden")
        set(other_flags "${other_flags} -fvisibility=hidden")
    endif()

This resolved almost all of my OSX warnings, with the exception of one for my Debug build (manual indentation for clarity) [1]:

ld: warning: direct access in function 

'boost::archive::basic_text_oprimitive<
    std::__1::basic_ostream<char, std::__1::char_traits<char> >
>::~basic_text_oprimitive()' 

from file

'vcpkg_installed/x64-osx/debug/lib/libboost_serialization.a(basic_text_oprimitive.o)' 

to global weak symbol 

'std::__1::basic_ostream<char, std::__1::char_traits<char> >& 
std::__1::endl<char, std::__1::char_traits<char> >
(std::__1::basic_ostream<char, std::__1::char_traits<char> >&)' 

from file 'projects/dogen.utility/tests/CMakeFiles/dogen.utility.tests.dir/indenter_filter_tests.cpp.o'

means the weak symbol cannot be overridden at runtime. This was likely caused by different
translation units being compiled with different visibility settings.

I am finding this extremely puzzling because:

• it works fine for release builds.
• I have no other such warning. If I change back to visibility=hidden I get over 50 warnings, so its clearly doing something.
• there is nothing special about this project when compare to all other projects. However, I do have another build which also relies on boost serialisation and there I can also see a similar behaviour with a lot more warnings [1].

The only possible conclusion here is that boost is being compiled with different visibility settings? The plot thickens.

[1] https://my.cdash.org/viewBuildError.php?type=1&buildid=2221688

[mcraveiro]

Since every single warning on my debug builds is related to ~basic_text_oprimitive, I decided to investigate how this symbol is exported in boost. We start with macro BOOST_SYMBOL_VISIBLE which is defined as follows [1]:

Defines the syntax of a C++ language extension that indicates a symbol is to be globally visible. If the compiler has no such extension, the macro is defined with no replacement text. Needed for classes that are not otherwise exported, but are used by RTTI. Examples include class for objects that will be thrown as exceptions or used in dynamic_casts, across shared library boundaries.

This appears sensible enough. We can see basic_text_oprimitive making use of it [2]:

// class basic_text_oprimitive - output of prmitives to stream
template<class OStream>
class BOOST_SYMBOL_VISIBLE basic_text_oprimitive
{

In GCC this macro is defined as follows [3]:

#define BOOST_SYMBOL_VISIBLE __attribute__((__visibility__("default")))

In Clang too [4]:

#define BOOST_SYMBOL_VISIBLE __attribute__((__visibility__("default")))

The general conclusion is that by setting visibility to default we should match the symbols definition (as per previous comment [5]). We now turn our attention to the destructor [2]:

    BOOST_ARCHIVE_OR_WARCHIVE_DECL
    basic_text_oprimitive(OStream & os, bool no_codecvt);
    BOOST_ARCHIVE_OR_WARCHIVE_DECL
    ~basic_text_oprimitive();

The macro BOOST_ARCHIVE_OR_WARCHIVE_DECL is defined as follows:

    #if defined(BOOST_WARCHIVE_SOURCE) || defined(BOOST_ARCHIVE_SOURCE)
        #define BOOST_ARCHIVE_OR_WARCHIVE_DECL BOOST_SYMBOL_EXPORT
    #else
        #define BOOST_ARCHIVE_OR_WARCHIVE_DECL BOOST_SYMBOL_IMPORT
    #endif

The macros BOOST_SYMBOL_EXPORT and BOOST_SYMBOL_IMPORT are cousins of BOOST_SYMBOL_VISIBLE. Once more, clang and GCC are identical. GCC [3]:

#    define BOOST_SYMBOL_EXPORT __attribute__((__visibility__("default")))
#    define BOOST_SYMBOL_IMPORT

And Clang says [4]:

#  define BOOST_SYMBOL_EXPORT __attribute__((__visibility__("default")))
...
#  define BOOST_SYMBOL_IMPORT

This means when we are importing, visibility is not defined. We now need to find out if that is a good thing or bad.

[1] https://www.boost.org/doc/libs/master/libs/config/doc/html/boost_config/boost_macro_reference.html
[2] https://www.boost.org/doc/libs/1_80_0/boost/archive/basic_text_oprimitive.hpp
[3] https://www.boost.org/doc/libs/1_80_0/boost/config/compiler/gcc.hpp
[4] https://www.boost.org/doc/libs/1_80_0/boost/config/compiler/clang.hpp
[5] #4497 (comment))