Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use yaml-cpp with MSVC 2015 with BUILD_SHARED_LIBS #461

Open
Liosan opened this issue Jan 29, 2017 · 19 comments

Comments

Projects
None yet
9 participants
@Liosan
Copy link
Contributor

commented Jan 29, 2017

Hi,

I'm compiling yaml-cpp from source with MSVC 2015, in the RelWithDebInfo x64 configuration, with BUILD_SHARED_LIBS=ON specified. The code fails to link due to lack of YAML_CPP_API on classes in exceptions.h:

6>load_node_test.obj : error LNK2019: unresolved external symbol "public: virtual __cdecl YAML::Exception::~Exception(void)" (??1Exception@YAML@@UEAA@XZ) referenced in function "public: virtual void * __cdecl YAML::Exception::`scalar deleting destructor'(unsigned int)" (??_GException@YAML@@UEAAPEAXI@Z)
6>node_spec_test.obj : error LNK2001: unresolved external symbol "public: virtual __cdecl YAML::Exception::~Exception(void)" (??1Exception@YAML@@UEAA@XZ)

and several more. These are errors from the run-tests project, but my project gives the same. Now, I can solve this for all the non-template classes quite easily, but I also get a lot of "xxx needs to have dll-interface to be used by client of class yyy" warnings. And these warnings result in linkage erros in my project:

1>Parameter.obj : error LNK2001: unresolved external symbol "public: static class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > YAML::detail::node_data::empty_scalar" (?empty_scalar@node_data@detail@YAML@@2V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@A)
1>C:\Documents\src\Mars Initiative\mi-build\Output\entities.dll : fatal error LNK1120: 1 unresolved externals

empty_scalar is a static std::string in node_data, and the dll does not export std::string. I have 600+ "dll-interface" warnings, so I guess the problem is pretty broad. I guess you're familiar with the issue, but just to make sure we're on the same page, I'm talking about this problem: https://support.microsoft.com/en-us/help/168958/how-to-export-an-instantiation-of-a-standard-template-library-stl-class-and-a-class-that-contains-a-data-member-that-is-an-stl-object

How would you advise to proceed? :) Am I doing something wrong?

@jbeder

This comment has been minimized.

Copy link
Owner

commented Jan 29, 2017

We've run into this before, but I don't recall the full resolution.

What version of yaml-cpp are you using? Would you be willing to try some older (not too old) versions to see if this is a recent introduction? (Maybe binary search :) ) I could have sworn that it compiled and linked successfully on MSVC as a shared lib.

@Liosan

This comment has been minimized.

Copy link
Contributor Author

commented Jan 29, 2017

I'm using the newest version from master branch. Most of the necessary classes yaml_cpp have YAML_CPP_API, but the ones in Exceptions.h don't, and after looking through a few years worth of git log - never did.

I did a quick "binary search" on that header, and the commit that broke it is probably 0f20ddc from december, "Fix -Wweak-vtables warnings in exception classes.". Basically the exception destructors were converted from implicit to explicit and defined in the .cpp, changing exceptions.h classes from a header-only to requiring a .cpp file (and hence requiring YAML_CPP_API). Why does MS have to make it so complicated :)

However! Even with a commit that compiles, I am unable to link yaml-cpp.lib (the import lib for the dll) into my project. I still get the error about empty_scalar. Any ideas?

@jbeder

This comment has been minimized.

Copy link
Owner

commented Jan 30, 2017

Thanks for investigating. I don't know how to fix this, and I don't have access to MSVC at the moment.

If you have a patch that even fixes some of this issue, could you send a PR?

@alirezakazemi

This comment has been minimized.

Copy link

commented Feb 3, 2017

Hi, I have the same problem but compiling using MinGW32 (g++) version 4.9.1. The static build is done completely but the dynamic build complains with the similar errors (Compiling using command cmake -G "MinGW Makefiles" -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Debug/Release ..).

In the dynamic build, yaml-cpp.dll and yaml-cpp.dll.a compile successfully but after that during the run-test.exe linking stops with mentioned error.
I tried to continue compile of the sub-targets of Makefile which come after run-test.exe (i.e. the programs in util dir). Then mingw32-make parse and mingw32-make read result in success but mingw32-make sandbox, fails with errors similar to mingw32-make run-test.

About the yaml-cpp commit version, I'm not sure but I downloaded the zip file from master branch in recent one or two weeks.

Linking CXX executable sandbox.exe
CMakeFiles\sandbox.dir/objects.a(sandbox.cpp.obj):sandbox.cpp:(.text$_ZN4YAML11I
nvalidNodeC1Ev[__ZN4YAML11InvalidNodeC1Ev]+0x63): undefined reference to `vtable
 for YAML::Exception'
CMakeFiles\sandbox.dir/objects.a(sandbox.cpp.obj):sandbox.cpp:(.text$_ZN4YAML11I
nvalidNodeC1Ev[__ZN4YAML11InvalidNodeC1Ev]+0x8c): undefined reference to `vtable
 for YAML::RepresentationException'
E:/Qt/Qt5.4.1/Tools/mingw491_32/bin/../lib/gcc/i686-w64-mingw32/4.9.1/../../../.
./i686-w64-mingw32/bin/ld.exe: CMakeFiles\sandbox.dir/objects.a(sandbox.cpp.obj)
: bad reloc address 0x8c in section `.text$_ZN4YAML11InvalidNodeC1Ev[__ZN4YAML11
InvalidNodeC1Ev]'
collect2.exe: error: ld returned 1 exit status
util\CMakeFiles\sandbox.dir\build.make:97: recipe for target 'util/sandbox.exe'
failed
mingw32-make[3]: *** [util/sandbox.exe] Error 1
CMakeFiles\Makefile2:469: recipe for target 'util/CMakeFiles/sandbox.dir/all' fa
iled
mingw32-make[2]: *** [util/CMakeFiles/sandbox.dir/all] Error 2
CMakeFiles\Makefile2:481: recipe for target 'util/CMakeFiles/sandbox.dir/rule' f
ailed
mingw32-make[1]: *** [util/CMakeFiles/sandbox.dir/rule] Error 2
makefile:169: recipe for target 'util/CMakeFiles/sandbox.dir/rule' failed
mingw32-make: *** [util/CMakeFiles/sandbox.dir/rule] Error 2
@Liosan

This comment has been minimized.

Copy link
Contributor Author

commented Feb 5, 2017

@alirezakazemi As a workaround, I remember MinGW has a compilation switch to export ALL symbols from a DLL, which might solve this issue.

@jbeder

This comment has been minimized.

Copy link
Owner

commented Feb 5, 2017

@Liosan can you update this Issue with the failure you get at HEAD in MSVC 2015?

@Liosan

This comment has been minimized.

Copy link
Contributor Author

commented Feb 7, 2017

1>Parameter.obj : error LNK2001: unresolved external symbol "public: static class std::basic_string<char,struct std::char_traits,class std::allocator > YAML::detail::node_data::empty_scalar" (?empty_scalar@node_data@detail@YAML@@2v?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@A)
1>C:\Documents\src\Mars Initiative\mi-build\Output\entities.dll : fatal error LNK1120: 1 unresolved externals

From what I've been researching and experimenting in my own project:

  • anything that yields the warning "xxx needs to have dll-interface to be used by client of class yyy" has a potential to cause "undefined external symbol"
  • for general purpose DLLs, most Internet discussions suggest using std::string and other STL class in a DLL interface is a very bad idea, because STL implementation differ between MSVC versions (so you'd need a per-compiler version of the DLL) and a the DLL can have a separate heap that the main process, leading to mismatch alloc-dealloc pairs (I don't understand yet how this can happen)
  • the proposed solution in the aforementioned discussions is not using STL, either replacing it with custom containers, C-style code, dedicated Microsoft-provided container classes or PIMPL for hiding private implementation. All of this seems like throwing out the baby with the bathwater for a library like yaml-cpp, which doesn't target MSVC specifically, and allows static linkage
  • However, if we assume that the dll will be used with the exact same version of MSVC, we can presumably fix this using proper __declspec on the STL (template) forward declarations. Stuff like this (note it should also have "extern" in some cases, I'm still figuring it out):

struct YAML_CPP_API std::_Container_base12;
template class YAML_CPP_API std::_String_val<std::_Simple_types>;

We might need to do it per-MSVC version (I don't know how much the STL changes, but std::_String_val<std::_Simple_types> looks pretty internal). The list of classes affected is at least 30, but adding __declspec to some pulls in more.

Crazy :)

@Liosan

This comment has been minimized.

Copy link
Contributor Author

commented Feb 7, 2017

For completeness, here is the list of affected classes, here is where my speculation on the number of affected types comes from:

class 'std::basic_string<char,std::char_traits,std::allocator>'
class 'std::list<YAML::detail::node_data::kv_pair,std::allocator<_Ty>>'
class 'std::set<const char *,std::less<_Kty>,std::allocator<_Kty>>'
class 'std::set<YAML::detail::shared_node,std::less<_Kty>,std::allocator<_Kty>>'
class 'std::shared_ptrYAML::detail::memory_holder'
class 'std::shared_ptrYAML::detail::memory'
class 'std::unique_ptr<YAML::Directives,std::default_delete<_Ty>>'
class 'std::unique_ptr<YAML::EmitterState,std::default_delete<_Ty>>'
class 'std::unique_ptr<YAML::Scanner,std::default_delete<_Ty>>'
class 'std::vector<char,std::allocator>'
class 'std::vector<const testing::MatcherDescriberInterface *,std::allocator<_Ty>>'
class 'std::vector<const void *,std::allocator<_Ty>>'
class 'std::vector<int,std::allocator<_Ty>>'
class 'std::vector<std::pair<YAML::detail::node *,YAML::detail::node *>,std::allocator<_Ty>>'
class 'std::vector<testing::internal::linked_ptrtesting::internal::ExpectationBase,std::allocator<_Ty>>'
class 'std::vector<testing::TestInfo *,std::allocator<_Ty>>'
class 'std::vector<testing::TestPartResult,std::allocator<_Ty>>'
class 'std::vector<testing::TestProperty,std::allocator<_Ty>>'
class 'std::vector<unsigned char,std::allocator<_Ty>>'
class 'std::vector<YAML::detail::node *,std::allocator<_Ty>>'
class 'std::vector<YAML::RegEx,std::allocator<_Ty>>'
class 'testing::ExpectationSet'
class 'testing::internal::linked_ptr'
class 'testing::internal::linked_ptr<const testing::MatcherInterface>'
class 'testing::internal::linked_ptrtesting::Expectation'
class 'testing::internal::linked_ptrtesting::internal::ExpectationBase'
class 'testing::internal::Mutex'
class 'testing::internal::scoped_ptr'
class 'testing::internal::scoped_ptrstd::string'
class 'testing::internal::scoped_ptrstd::stringstream'

This is extracted from the list of warnings that MSVC 2015 generates for me, and is probably not complete. Some of them are actually from gtest, and might be a non-issue (the tests run fine in DLL mode if you copy the DLLs into the proper location).

@alirezakazemi

This comment has been minimized.

Copy link

commented Feb 8, 2017

Hi,
@Liosan, Thanks for suggesting the MinGW export all option. I searched more on this:

According to this articles guide:
https://blog.kitware.com/create-dlls-on-windows-without-declspec-using-new-cmake-export-all-feature/

I tried to build using following cmake command:

cmake.exe -G "MinGW Makefiles" -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release ..

As the article mentions, cmake has provided an option
CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS which automatically exports symbols
(except some special cases such as static class member or something so) and
lets most codes which compile to shared libs in linux be compiled as dll
for windows without need for explicitly adding dll export .

I tried that. Now sandbox is linked successfully
but run-test fails again.

In the build output log below I was running mingw32-make -j2 once before
and some targets were built and it failed on some. Now I run it again
(without cleaning to continue quickly) such that I can paste the output here.

Edit: seems that sandbox is also failed. Since I have used parallel compile using 2 processes (-j2) the output log is scrambled.

[  8%] Built target gmock
[ 56%] Built target yaml-cpp
[ 60%] Built target gtest
[ 66%] Built target gmock_main
[ 68%] Linking CXX executable sandbox.exe
[ 71%] Built target parse
[ 75%] Built target read
[ 76%] Linking CXX executable run-tests.exe
CMakeFiles\sandbox.dir/objects.a(sandbox.cpp.obj):sandbox.cpp:(.text$_ZN4YAML11I
nvalidNodeC1Ev[__ZN4YAML11InvalidNodeC1Ev]+0x63): undefined reference to
`vtable
 for YAML::Exception'
CMakeFiles\sandbox.dir/objects.a(sandbox.cpp.obj):sandbox.cpp:(.text$_ZN4YAML11I
nvalidNodeC1Ev[__ZN4YAML11InvalidNodeC1Ev]+0x8c): undefined reference to
`vtable
 for YAML::RepresentationException'
E:/Qt/Qt5.4.1/Tools/mingw491_32/bin/../lib/gcc/i686-w64-mingw32/4.9.1/../../../.
./i686-w64-mingw32/bin/ld.exe:
CMakeFiles\sandbox.dir/objects.a(sandbox.cpp.obj)
: bad reloc address 0x8c in section
`.text$_ZN4YAML11InvalidNodeC1Ev[__ZN4YAML11
InvalidNodeC1Ev]'
collect2.exe: error: ld returned 1 exit status
util\CMakeFiles\sandbox.dir\build.make:97: recipe for target
'util/sandbox.exe'
failed
mingw32-make[2]: *** [util/sandbox.exe] Error 1
CMakeFiles\Makefile2:469: recipe for target
'util/CMakeFiles/sandbox.dir/all' fa
iled
mingw32-make[1]: *** [util/CMakeFiles/sandbox.dir/all] Error 2
mingw32-make[1]: *** Waiting for unfinished jobs....
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xbb6): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xc16): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xc76): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xdac): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xf3a): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0x1126): more undefined references to
`YAML::InvalidNode::~InvalidNode()' f
ollow
E:/Qt/Qt5.4.1/Tools/mingw491_32/bin/../lib/gcc/i686-w64-mingw32/4.9.1/../../../.
./i686-w64-mingw32/bin/ld.exe:
CMakeFiles\run-tests.dir/objects.a(load_node_test
.cpp.obj): bad reloc address 0x11 in section `.text.unlikely'
collect2.exe: error: ld returned 1 exit status
test\CMakeFiles\run-tests.dir\build.make:368: recipe for target
'test/run-tests.exe' failed
mingw32-make[2]: *** [test/run-tests.exe] Error 1
CMakeFiles\Makefile2:220: recipe for target
'test/CMakeFiles/run-tests.dir/all' failed
mingw32-make[1]: *** [test/CMakeFiles/run-tests.dir/all] Error 2
makefile:137: recipe for target 'all' failed
mingw32-make: *** [all] Error 2
@alirezakazemi

This comment has been minimized.

Copy link

commented Feb 8, 2017

Here is my build output (continuing without clean) with mingw32-make -j1:
First target run-tests is built which fails then it would build the sandbox. But in parallel case both builds for targets sandbox and run-test were going together which both of them failed.

cmake command:
cmake.exe -G "MinGW Makefiles" -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release ..

[ 51%] Built target yaml-cpp
[ 56%] Built target gmock
[ 58%] Linking CXX executable run-tests.exe
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xbb6): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xc16): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xc76): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xdac): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0xf3a): undefined reference to `YAML::InvalidNode::~InvalidNode()'
CMakeFiles\run-tests.dir/objects.a(load_node_test.cpp.obj):load_node_test.cpp:(.
text+0x1126): more undefined references to `YAML::InvalidNode::~InvalidNode()' f
ollow
E:/Qt/Qt5.4.1/Tools/mingw491_32/bin/../lib/gcc/i686-w64-mingw32/4.9.1/../../../.
./i686-w64-mingw32/bin/ld.exe: CMakeFiles\run-tests.dir/objects.a(load_node_test
.cpp.obj): bad reloc address 0x11 in section `.text.unlikely'
collect2.exe: error: ld returned 1 exit status
test\CMakeFiles\run-tests.dir\build.make:368: recipe for target 'test/run-tests.
exe' failed
mingw32-make[2]: *** [test/run-tests.exe] Error 1
CMakeFiles\Makefile2:220: recipe for target 'test/CMakeFiles/run-tests.dir/all'
failed
mingw32-make[1]: *** [test/CMakeFiles/run-tests.dir/all] Error 2
makefile:137: recipe for target 'all' failed
mingw32-make: *** [all] Error 2
@cirquit

This comment has been minimized.

Copy link

commented Apr 17, 2017

Having the same problem with VS2017 and Windows 8.1.

  • Generated with
cmake.exe .. -G "Visual Studio 15 2017"
  • Opened in VS2017 with Admin rights, Build the INSTALL project
  • Added the generated include and lib directories to my own project in Project -> Properties -> VC++ and Linker

Current workaround is to blatantly copy the whole include and src folders to my own project. At least this works for the time being :(

EDIT: After some struggling I got it working. These were the steps (change the generator depending on your platform):

  • git clone https://github.com/jbeder/yaml-cpp
  • cd yaml-cpp
  • mkdir build
  • cd build
  • cmake -G "Visual Studio 15 2017 Win64" -DBUILD_SHARED_LIBS=OFF ..
  • Open the YAML_CPP.sln file located in the build folder with VS2017
  • Build the project ALL_BUILD
  • if you want to link against this library, go to your project:
    • Properties -> VC++ Directories -> Include Directories -> Add YOUR-PATH-TO-REPOSITORY\yaml-cpp\include
    • Properties -> VC++ Directories -> Library Directories -> Add YOUR-PATH-TO-REPOSITORY\yaml-cpp\build\Debug
    • Properties -> Linker -> Input -> Additional Dependencies -> Add libyaml-cppmdd.lib
    • use header <yaml-cpp/yaml.h>

This may be a good thing to add to the install.txt

@alirezakazemi

This comment has been minimized.

Copy link

commented Apr 27, 2017

@cynoxure

This comment has been minimized.

Copy link

commented Aug 2, 2017

Set MSVC_STHREADED_RT in CMakeLists (I used cmake-gui) and configure...worked great for me. Good Luck!

@sergiud

This comment has been minimized.

Copy link

commented Jan 22, 2018

I just hit this issue in a CMake superbuild. I was able to the trace the problem to the simple fact that YAML_CPP_API is never defined as __declspec(dllimport) in projects which use yaml-cpp. The reason for this is a missing definition of YAML_CPP_DLL in yaml-cpp's exported interface.

The issue can be resolved by adding

if (WIN32 AND BUILD_SHARED_LIBS)
  target_compile_definitions (yaml-cpp INTERFACE ${PROJECT_NAME}_DLL)
endif (WIN32 AND BUILD_SHARED_LIBS)

somewhere after the add_library call.

@jbeder

This comment has been minimized.

Copy link
Owner

commented Mar 6, 2018

@sergiud are you saying, to add that block in yaml-cpp's CMakeLists.txt or the dependent project's?

If the former, can you open a pull request?

@sergiud

This comment has been minimized.

Copy link

commented Mar 6, 2018

@jbeder It's the former. I'll create a PR.

@jianghe01

This comment has been minimized.

Copy link

commented Jul 10, 2018

i still cannot compile it on windows10 with vs2017, so can anyone send me a x64 libyaml.lib

@three0s

This comment has been minimized.

Copy link

commented Jul 25, 2018

in MSVC, try adding YAML_CPP_DLL to your project's Preprocessor Definitions

@thomasreiser

This comment has been minimized.

Copy link

commented Mar 14, 2019

Are there any news on that topic?
I am unable to build yaml-cpp on Windows 10 MSVC 17 as a shared library:

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2338	can't delete an incomplete type	yaml-cpp shared md	C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\memory	2082	
Error	C2027	use of undefined type 'YAML::Directives'	yaml-cpp shared md	C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\memory	2082	
Error	C2338	can't delete an incomplete type	yaml-cpp shared md	C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\memory	2082	
Error	C2027	use of undefined type 'YAML::Directives'	yaml-cpp shared md	C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\memory	2082	
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.