Linking with static expat

[henryborchers]

I don't know if this the place to ask this or not. I've been trying to build Exiv2 with static zlib and static expat in MSVC 2015 Win64.

I was able to build it with static zlib but if I tried to link it with static Expat, I'd get linker errors. exiv2 with static zlib and expat even possible?

[clanmills]

I believe it's possible. Are you building with the msvc/exiv2.sln file, or using CMake? Which Edition of Visual Studio 2015 are you using? Community, Express, or a Retail Edition?

The policy is "all static" or "all dynamic". I have successfully built x64/Debug (which is static) and x64/DebugDLL a few minutes ago:

c:\Raphael2015\exiv2-trunk\msvc\bin\x64\Debug>dir *.exe *.dll
 Volume in drive C has no label.
 Volume Serial Number is 0899-EF40
 Directory of c:\Raphael2015\exiv2-trunk\msvc\bin\x64\Debug
2017-06-29  08:55 PM         6,420,992 exiv2.exe


c:\Raphael2015\exiv2-trunk\msvc\bin\x64\Debug>cd ..\DebugDLL
c:\Raphael2015\exiv2-trunk\msvc\bin\x64\DebugDLL>dir *.exe *.dll
 Volume in drive C has no label.
 Volume Serial Number is 0899-EF40
 Directory of c:\Raphael2015\exiv2-trunk\msvc\bin\x64\DebugDLL
2017-06-29  08:58 PM           555,008 exiv2.exe
 Directory of c:\Raphael2015\exiv2-trunk\msvc\bin\x64\DebugDLL
2017-06-29  08:58 PM         5,425,152 libexiv2.dll
2017-06-29  08:55 PM           338,432 libexpat.dll
2017-06-29  08:55 PM           201,216 zlib1.dll
               4 File(s)      6,519,808 bytes
               0 Dir(s)   4,790,566,912 bytes free

c:\Raphael2015\exiv2-trunk\msvc\bin\x64\DebugDLL> 

However, I'm currently working on a build error for Visual Studio 2015 Community Edition x64 bit builds. I will submit the fix in the next couple of hours.

[henryborchers]

CMake, with VS2015 Community

[clanmills]

The CMake/Visual Studio build does not support static external libraries. The reason is historical. I found it almost impossible to get CMake to build anything with Visual Studio when I first worked with it around 2012. To simplify things, I could only get CMake to build and link DLLs.

Since then I've made progress and you will find scripts in contrib/cmake/msvc which "tame the beast". However the historical restriction of linking zlib and expat dynamically remains (although you may build a static exiv2.exe). You can build it with:

c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc>cmd.exe
c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc>vcvars 2015 64
c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc>cmakeBuild --exiv2 ..\..\.. --static --rebuild 2>&1

... shake rattle roll for 5 minutes ...

c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc>cd dist\2015\x64\static\Release\bin

c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc\dist\2015\x64\static\Release\bin>dir exiv2.exe *.dll
 Volume in drive C has no label.
 Volume Serial Number is 0899-EF40

 Directory of c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc\dist\2015\x64\static\Release\bin

2017-06-29  10:37 PM         4,209,152 exiv2.exe

 Directory of c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc\dist\2015\x64\static\Release\bin

2017-06-29  10:32 PM           138,752 expat.dll
2017-06-29  10:31 PM            82,432 zlib.dll
               3 File(s)      4,430,336 bytes
               0 Dir(s)   5,235,994,624 bytes free

c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc\dist\2015\x64\static\Release\bin> exit
c:\Users\rmills\GitHub\exiv2.git\contrib\cmake\msvc>

I'm willing to listen to any good argument for supporting static builds of the external libraries. However, as the pain and effort to achieve the current capability was huge, I'm unlikely to accept the challenge to support static builds of the external libraries.

You may think that this is an easy change, however the testing effort is considerable to support:
{2005 | 8 | 10 | 12 | 13 | 15} x { --static | --shared } x { x86 | x64 }

[piponazo]

Hi all,

I just want to comment that it is possible to link exiv2 with the static versions of zlib and expat. In my organisation we have spent some time creating conan recipes for all our dependencies, and we are supporting this case. The relevant python lines of the conan recipe looks like this:

            cmake_options.append("EXPAT_INCLUDE_DIR=%s" % expat_include_dir)
            cmake_options.append("EXPAT_LIBRARY=%s" % expat_library)
            cmake_options.append("ZLIB_INCLUDE_DIR=%s" % zlib_include_dir)
            cmake_options.append("ZLIB_LIBRARY=%s" % zlib_library)

Where zlib_library and expat_library are the paths to the .lib files.

Is not fully working yet but you can find the conan recipe here:

https://github.com/piponazo/conan-exiv2

(The support for zlib and expat has not been added here yet)

[clanmills]

I don't know what conan is. Is this something that will benefit the Exiv2 project?

There is no question that it's possible to link the static versions of expat and libz because the msvc/exiv2.sln file does it. However, it will require changes to CMake_msvc.txt and contrib/cmake/msvc/cmakeBuild.cmd, and probably changes to contrib/buildserver (which builds and tests the bundles using CMake).

A simpler solution is for you to execute devenv.exe commands to use msvc/exiv2.sln to build the libraries.

[piponazo]

[offtopic] Conan is a package manager that is becoming in the standard package manager for C++ :).

You can find more information in its page: https://www.conan.io/

The recipe is pointing now to this github repository, and it will make extremely easy to get a working package for someone interested in the usage of exiv2 (for linux, mac or windows).

[clanmills]

Conan sounds very useful and I'd like to work with you to ensure Exiv2 is published. I'll look at the effort to get our CMake static build to work with static external libraries.

[piponazo]

Sure. I am now working now in this recipe for Exiv2 and I will need to also contribute with some of the other dependencies that it has (expat). Other dependencies like zlib are already in the main conan-center repository.

If it is fine for you I will continue working on the conan-exiv2 recipe. Feel free to comment there and we can discuss more via email about the possible collaboration :)

[clanmills]

I'd expect zlib and expat to be "kind of core". They are so common. (And is it called libz or zlib? On the command-line, you use -lz to link libz.a. However it's everyday name is zlib. Groan ....... software).

I'm happy to work with you. All the different package systems: apt, brew, port (and probably others) intend to help on a single platform. I believe Microsoft are entering the packaging arena and will integrate theirs with Visual Studio. A cross-platform packaging systems helps everybody. Good Luck to you.

Please keep working on the recipe and I'm happy to help you. I'm going to leave this issue open to remind me to work on the original request to link static external libraries.

[henryborchers]

Not to get too far off topic but I'd like to second what piponazo mentioned. ,If exiv2 was on Conan, my life would be so much easier.

[clanmills]

@henryborchers You are not off topic. That's also what I'd like to achieve. I'm working with @piponazo to achieve that goal. I believe @piponazo owns the conan task and I'll make any changes required on Exiv2 to reach that objective.

[clanmills]

Progress update: I've done more (unsuccessful) work on this. My team-mate Ben has observed that expat is only used by xmpsdk (and samples/geotag.exe). I could always build the static version of expat and never build the DLLs. However, the architecture I would like is the behaviour of the msvc/exiv2.sln. Everything is static, or everything is a DLL. It can be done. msvc/exiv2.sln does it!

[clanmills]

Progress update: I've put another session into this without a successful result. @piponazo is giving the CMake support a makeover. Perhaps he'll even solve this as part of the makeover. If he doesn't, I take up this challenge after the makeover is complete.

I'm going to stop working on this for now to focus on others matters. In particular, I've discussed with @piponazo having a "dot" release of the code late August/early September. Ben and I have other work to undertake for that dot release and I'd like to complete that before going on vacation on July 21.

When I return in early August, we'll decide what's going into the 'dot' release and I will focus on testing and releasing v0.26.1.

[piponazo]

Good to know. I will keep this issue in my list of priorities during the CMake makeover ;)

[data-man]

Sorry for the question, why not use another library written in C++?
For example:

• tinyxml2 (XPath not supported)
• pugixml (very fast, XPath 1.0 and header-only mode supported)

[clanmills]

@piponazo I own the task to statically link Expat/Exiv2/CMake. To avoid "stepping on each others toes", I'm going to put it to the side for now and revisit it when you're done with your work. @draekko has a working solution in his branch cpp11-xmpsdk-split which always links statically with Expat (and never dynamically). We want to pull that into Exiv2 v0.26.1, so it's possible that this issue will disappear with no effort required by you at all! I will have to update xmpsdk/exiv2.sln to adopt the same approach. sample/geotag also requires Expat and I can either remove the sample or link statically.

@data-man Thank you for your suggestions. Our code builds and links a static version of Adobe's XMPsdk which uses Expat. I don't want to change anything in Adobe's code. Even if one of the suggested libraries interfaces using the Expat API, I still want to ship Exiv2 with libxmpsdk.a linked to Expat.

I believe the fix for this will be easy. I've had 3 sessions (of about 4 hours each) without resolving this. A break from the puzzle is probably the correct medicine. When I look at this in August, it's probable that the fix will be obvious in minutes. The difficulty is being caused by "Russian Dolls". CMake is a language for generating build environments and can generate Visual Studio Solution/Project Files which are an XML wrappers for running the Microsoft compiler and linker CL and LINK. CMake and Visual Studio are not happy partners and I know Thomas B (another Exiv2 Team Member) agrees.

[clanmills]

Delighted to report that I have a fix which I will submit this evening. The essential ingredient is to get Expat to link the static runtime library. So the following CMake options are required when building Expat:

-DBUILD_shared=0 -DCMAKE_C_FLAGS_RELEASE=/MT -DBUILD_examples=0 -DBUILD_tests=0

There are other "gotchas" concerning CMake/zlib which builds the static library as zlibstatic.lib I'm still considering alternative ways to deal with this. Success is certain after 4 sessions of 4 hours. Session 5 will fix this.

[clanmills]

I've submitted a fix to master. Please be aware that @draekko and I are merging a major change to master this week and the build is currently broken. However, you should be able to use the change above on the "old" master. In addition to the discovery above concerning Expat, you should link zlibstatic into exiv2. The appropriate change is around line 260 in src/CMakeLists.txt

    if ( EXIV2_ENABLE_STATIC )
        TARGET_LINK_LIBRARIES( exiv2lib zlibstatic ${ZLIB_LIBRARIES} )
    else()
        TARGET_LINK_LIBRARIES( exiv2lib ${PRIVATE_VAR} ${ZLIB_LIBRARIES} )
    endif()

Although it is (just) possible to use CMake from the command-line, I recommend that you build using the script in contrib/cmake/msvc as follows:

cd <exiv2dir>\contrib\cmake\msvc
cmd.exe
vcvars 2015 64
cmakeBuild --static 
exit

Usage: cmakeBuild --help
For more information please read <exiv2dir>\contrib\cmake\msvc\ReadMe.txt