How to build cyclonedds static?

[Kischy]

When I try to build static via
cmake -G "Visual Studio 15 2017 Win64" -DCMAKE_INSTALL_PREFIX=D:/EclipseCylconeDDS/cyclonedds/install -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF .. && cmake --build . --config Release -j --target install

I get these errors:

-- Configuring done
CMake Error: install(EXPORT "CycloneDDS" ...) includes target "ddsc" which requires target "security_core" that is not in any export set.
CMake Error:
  Error evaluating generator expression:

    $<TARGET_PDB_FILE:ddsc>

  TARGET_PDB_FILE is allowed only for targets with linker created artifacts.


CMake Error:
  Error evaluating generator expression:

    $<TARGET_PDB_FILE:ddsc>

  TARGET_PDB_FILE is allowed only for targets with linker created artifacts.


CMake Error:
  Error evaluating generator expression:

    $<TARGET_PDB_FILE:ddsc>

  TARGET_PDB_FILE is allowed only for targets with linker created artifacts.


CMake Error:
  Error evaluating generator expression:

    $<TARGET_PDB_FILE:ddsc>

  TARGET_PDB_FILE is allowed only for targets with linker created artifacts.


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

Any idea what I am doing wrong here? Might be releated to #317 .

[Kischy]

Does anyone have an idea how to fix this?

[k0ekk0ek]

Hi @Kischy. I think it's likely related to #764. Can you try disabling the installation of pdb files for things that rely on security_core, i.e. the security plugins? If that solves it the quick fix is to add a version check. Long term fix would be to cleanup CMake instructions for security_core. There's no reason that should be in a separate target to begin with (I think).

[Kischy]

First try

Ok, when I comment the following code in following files like this one:

 if (MSVC)
   install(FILES $<TARGET_PDB_FILE:dds_security_ac>
     DESTINATION "${CMAKE_INSTALL_BINDIR}"
     COMPONENT dev
     OPTIONAL
   )
 endif()

src/security/builtin_plugins/access_control/CMakeLists.txt: Commented lines 51-57
src/security/builtin_plugins/authentication/CMakeLists.txt: Commented lines 54-60
src/security/builtin_plugins/cryptographic/CMakeLists.txt: Commented lines 55-61

and
src/core/CMakeLists.txt: Commentes lines 62-68.

I get the following error still:

CMake Error: install(EXPORT "CycloneDDS" ...) includes target "ddsc" which requires target "security_core" that is not in any export set.

Second try

When I, in addition, comment line 29 of src/core/CMakeLists.txt #target_link_libraries(ddsc PRIVATE security_core) I get something like this

  time.c
  ddsconf.vcxproj -> D:\EclipseCylconeDDS\cyclonedds\build\bin\Release\ddsconf.exe
  Generating cyclonedds.rnc, cyclonedds.xsd, manual/options.md
  Generating defconfig.c

  ddsi_security_util.c
D:\EclipseCylconeDDS\cyclonedds\src\security\api\..\..\core\ddsi\include\dds/ddsi/ddsi_security_util.h(20): fatal error C1083: Cannot open include file: 'dds/security/core/dds_security_utils.h': No such file or directory [D:\EclipseCylconeDDS\cyclonedds\build\src\core\ddsc.vcxproj]
  ddsi_security_omg.c
D:\EclipseCylconeDDS\cyclonedds\src\security\api\..\..\core\ddsi\include\dds/ddsi/ddsi_security_util.h(20): fatal error C1083: Cannot open include file: 'dds/security/core/dds_security_utils.h': No such file or directory [D:\EclipseCylconeDDS\cyclonedds\build\src\core\ddsc.vcxproj]

  ddsi_handshake.c
D:\EclipseCylconeDDS\cyclonedds\src\core\ddsi\src\ddsi_handshake.c(27): fatal error C1083: Cannot open include file: 'dds/security/core/dds_security_fsm.h': No such file or directory [D:\EclipseCylconeDDS\cyclonedds\build\src\core\ddsc.vcxproj]

  ddsi_security_msg.c
D:\EclipseCylconeDDS\cyclonedds\src\core\ddsi\src\ddsi_security_msg.c(26): fatal error C1083: Cannot open include file: 'dds/security/core/dds_security_utils.h': No such file or directory [D:\EclipseCylconeDDS\cyclonedds\build\src\core\ddsc.vcxproj]
  ddsi_security_exchange.c

Did I forget to comment the installation of some pdb files?

[eboasson]

I suspect it is because DDS Security relies on dynamically loading plugins. Things could be reworked to make it possible to statically link in the required plugins, but that it is not currently possible.

I haven't verified on Windows, but on macOS with DBUILD_SHARED_LIBS=OFF, I also get

CMake Error: install(EXPORT "CycloneDDS" ...) includes target "ddsc" which requires target "security_core" that is not in any export set.

unless I also disable DDS Security using -DENABLE_SECURITY=NO.

[Kischy]

Also: For me the dynamic versions of the OpenSSL library were selected when using -DBUILD_SHARED_LIBS=OFF. Should't the static ones be used in this case? I changed that manually, but nothing changes.

[Kischy]

I suspect it is because DDS Security relies on dynamically loading plugins. Things could be reworked to make it possible to statically link in the required plugins, but that it is not currently possible.

@eboasson So in that case a static build of cyclonedds is not possible with security features enabled at the moment. And I suspect there is no workaround?

[eboasson]

@Kischy officially, I think that's the case. But if you don't mind getting your hands dirty, I think it wouldn't be very much work to avoid dynamically loading the plugins and replacing the dynamic symbol lookups in the libraries with the actual symbols.

That'd leave you with some problems, notably the possibility of symbol clashes, and CMake. The bit I'm pretty confident about is the source changes, the bit I have a hard time guessing how much work it'd be is the CMake part.

The problem is that we (Cyclone maintainers) don't have the time to do that sort of thing right now. But we'd definitely be happy if you were to contribute a fully static build with security.

[Kischy]

@eboasson Thank you. Unfortunately I do not have the time for this at the moment. If this changes I will have a look. As I do not know the source I have no idea how long this would take me.
What do you mean by "source changes" and "CMake" changes? I would expect that one "only" needs to change the CMake files.

[k0ekk0ek]

Quickfix might be to just don't build security as an intermediate step to see if you get the static linking to work. With -DENABLE_SSL=off and -DENABLE_SECURITY=off you should be able to do what you want. I think cyclonedds-python uses this and I've talked to some others that use a static library as well.

[Kischy]

So, I tested this. When I comment lines 62-68 from src/core/CMakeLists.txt and build via

cmake -G "Visual Studio 15 2017 Win64" -DCMAKE_INSTALL_PREFIX=D:/EclipseCylconeDDS/cyclonedds/install_static -DBUILD_EXAMPLES=OFF -DBUILD_IDLC=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF  -DENABLE_SECURITY=off -DENABLE_SSL=off .. && cmake --build . --config Release  -j --target install

which uses -DENABLE_SSL=off and -DENABLE_SECURITY=off everything works fine. What would be the next step to get a static security build?

[niclar]

@k0ekk0ek can we get static build support in time for the next release ?

[k0ekk0ek]

Hi @niclar, sorry for taking so long to respond. I'm not sure what the best approach is. If you want a true static build should security be included in that build? Personally, I don't see the value of a static build on a general purpose operating system like Windows because it has plenty of resources available, especially if the entirety of OpenSSL is to be included. That trade-off is different on embedded systems, but there you'll likely want to go with a different SSL/TLS implementation anyway (OpenSSL is simply too large). There's value in having it separate because if there's a flaw in OpenSSL it's likely that it'll have a big impact and you want to be able to swap .dll's (or .so's) quickly without rebuilding Cyclone DDS. Also, the master branch includes support for Eclipse Iceoryx, should that be included?

So, to answer your question (excuse the brevity), currently (at least for me) it doesn't have priority because it doesn't add features and it's likely different people want different things. However, if you have a use-case where a static build is a must-have, I'm happy to help out where I can. Please share your thoughts/shed some light on the why and we can go from there(?) Would that work for you?

[Kischy]

@k0ekk0ek I am using a dynamic build now and have no more need for a static build at the moment.

[niclar]

Hi @k0ekk0ek.
Re inclusion, I think security (OpenSSL and/or BoringSSL to begin with) should be made optional also in a static build and Icoryx baked in
(if there's no reason why the shared memory transport can't be included on all targets, in case it should be optional).

I don't see why there is a problem with re-building & -linking all if there's a serious error (and
I personally doubt the frequency of flaws in OpenSSL would be the source of that) (-& services still need to be re-tested etc.)

As for static linking goes, it's a requirement for us at least. (the static frankenbuild is running & plays nice with vcpkg.)
Api calls gets one guaranteed indirection less, dependency management is correct by construction and linktime optimization
apply. Unikernels without module support might then also become a target.

I think you could at least offer the bare minimum as given in the previous github comments (no security etc.) and we can take it from there
(incl. any code contributions, sponsoring etc.).

Looking forward to scale with cyclonedds.

[k0ekk0ek]

It's not my intention to dwell on this, but linking things like cryptographic libraries statically is/can be a real problem. Think of the impact on your system if every piece of software links it's own cryptographic library statically. It may not be that frequent, but think of the amount of work if something like heartbleed comes along and you have to update every software package on your system manually. https://www.openssl.org/news/vulnerabilities.html will show there have been a couple CVE's this year.

@niclar, if you're using vcpkg, am I right in assuming your on Windows? Iceoryx doesn't support Windows yet, so your problem might be that building a static version throws the error and is resolved if we make the CMake files behave a little better?

@Kischy, would that work for you too?

Instead of including OpenSSL/BoringSSL, I think for Windows the best/fanciest solution would be to actually use the crypto apis from Windows (same for macOS btw). It's on my whishlist, and I've discussed it a number of times, but unfortunately haven't found the time to do the work yet.

@niclar, @Kischy, if you can point me towards the projects in which it's used, that'd be really helpful to get a feel for how it's used.

[niclar]

@k0ekk0ek we're on linux (server-side) and windows (client-side, shadow compilation & additional standard compliance testing).

I'll reach out in private to you re the nature of our project

[Kischy]

It's not my intention to dwell on this, but linking things like cryptographic libraries statically is/can be a real problem.

I see your point. As I said, I use a dynamic build now and I will do so for the foreseeable future.

@niclar, if you're using vcpkg, am I right in assuming your on Windows? Iceoryx doesn't support Windows yet, so your problem might be that building a static version throws the error and is resolved if we make the CMake files behave a little better?

@Kischy, would that work for you too?

I think it is just a CMake problem, but I am not sure, since I did not have the time to look at the files properly.

[thijsmie]

Superseded by #1313. It is possible to build statically but it needs proper documentation.