Building both static and shared libraries at the same time #424

MaesterZ · 2022-06-10T08:22:39Z

Nomadic Labs is using libusb and hidapi for the Octez project at https://gitlab.com/tezos/tezos written in Ocaml.

Is there a clean way to compile and install both static and shared libraries at the same time?

@pietro submitted an MR to Alpine aports to have both static and shared libraries for package libusb-dev and is now available in Alpine 3.15+:

https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/23613

We did the same for package hidapi-dev but the switch to cmake removed the static libraries from Alpine 3.15 starting with 0.11.0:

https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/23615
https://gitlab.alpinelinux.org/alpine/aports/-/blob/master/community/hidapi/APKBUILD

At the moment we have no other solutions than building from source or use a custom Alpine package. We copied the current GitHub workflow for Ubuntu and run the configure/build/install twice:

https://gitlab.com/nomadic-labs/hidapi-apk/-/blob/main/APKBUILD#L21

# Shared libraries
cmake -B build/shared -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=lib -DBUILD_SHARED_LIBS=TRUE
cmake --build build/shared
cmake --install build/shared
# Static libraries
cmake -B build/static -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=lib -DBUILD_SHARED_LIBS=FALSE
cmake --build build/static
cmake --install build/static

I have no experience with cmake, my concern is that we overwrite the following files instlalling twice:

/usr/lib/cmake/hidapi/hidapi-config-version.cmake
/usr/lib/cmake/hidapi/hidapi-config.cmake
/usr/lib/cmake/hidapi/hidapi-release.cmake
/usr/lib/cmake/hidapi/hidapi.cmake

Note: Alpine aports MR https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/35251

The text was updated successfully, but these errors were encountered:

Youw · 2022-06-10T09:52:09Z

The way you're building it right now is the cleanest possible with CMake at the moment.

The only implication of the overwriting CMake files is that when a CMake user would try to use HIDAPI with
find_package(hidapi) it will only find target(s) (static or shared) that where installed the last.

In your example, CMake's find_package will find only targets for static HIDAPI.

This may be somethign that could be improved on HIDAPI side, but I need to understad one thing:

When a library is built and installed in the system, and there is both static and shared variants of the library + .pc pkg-config file(s).

Normally, if the user needs to use the library, the common approach is to use pkg-config, e.g.:
add to link flags: pkg-config --libs zlib
and usually that would be something like: -L/opt/local/lib -lz
which will link to a shared version of the library.
Having: pkg-config --libs zlib --static would give the same -L/opt/local/lib -lz and will link against shared library anyway, even though static version is installed and available too.

How exactly user gets to choose the static version of the library w/o manually hard-coding the path to it?

Youw · 2022-06-10T09:55:16Z

https://gitlab.com/nomadic-labs/hidapi-apk/-/blob/main/APKBUILD#L30

BTW: I recommend installing static version first, and then shared, so by default your CMake users would use a shared version of the library - to be consistent with pkgconfig users in general.

netfab · 2022-06-14T15:36:54Z

Hi,

How exactly user gets to choose the static version of the library w/o manually hard-coding the path to it?

In ¹ we can read :

Unfortunately, at least as of February 2013, there is no easy way to tell at once to an Autotools-based build system that you
intend to link statically; this means that the most common way to signal this to pkg-config becomes the command ./configure
LDFLAGS="-static" PKG_CONFIG="pkg-config --static".

However, if you want to link to a specific library statically (and others dynamically), you can use an option for the linker for that specific library :

PKG_CHECK_MODULES([LIBHIDAPI], [hidapi-libusb >= 0.10.0])
LIBHIDAPI_LIBS="-l:libhidapi-libusb.a"

See the following option in ² (but the documentation seems not very clear to me) :

-l namespec
--library=namespec

Tested here and it seems to work fine.

Youw · 2022-06-14T21:35:11Z

This may be somethign that could be improved on HIDAPI side

My analysis:

LIBHIDAPI_LIBS="-l:libhidapi-libusb.a"

I'd say this is practically a "manually hard-coding the path" (library name).

LDFLAGS="-static" PKG_CONFIG="pkg-config --static"

CMake doesn't have any similar machanism(s), so I don't think it worth making anything HIDAPI-specific in this matter on CMake level.

Basically, for now I'd say what is done at https://gitlab.com/nomadic-labs/hidapi-apk/-/blob/main/APKBUILD#L37 is fine (and can be used as a reference for others who need a similar thing).
https://gitlab.alpinelinux.org/alpine/aports/-/blob/master/community/hidapi/APKBUILD - uses the alternative approach, which technically is fine as well.
Both ways would give the same result.

The only true alternative, is to do something that, for instance, libjpeg-turbo does (here and here), but that would complicate CMake scripts for HIDAPI a lot, has complicated corner-cases (e.g. the Framework libraries on macOS), and I'm not sure it is really required.

netfab · 2022-06-15T09:52:47Z

LIBHIDAPI_LIBS="-l:libhidapi-libusb.a"

I'd say this is practically a "manually hard-coding the path" (library name).

Yes, but it is always possible to generate with cmake an additional *.pc file to link to the static archive.
In pc/hidapi-libusb-static.pc.in :

Libs: -L${libdir}                                                                                      
Libs.private: -L${libdir} -l:lib@OUTPUT_NAME@.a

And in cmake files :

index e5d7d51..d073545 100644
--- a/libusb/CMakeLists.txt
+++ b/libusb/CMakeLists.txt
@@ -65,3 +65,4 @@ if(HIDAPI_INSTALL_TARGETS)
 endif()
 
 hidapi_configure_pc("${PROJECT_ROOT}/pc/hidapi-libusb.pc.in")
+hidapi_configure_pc("${PROJECT_ROOT}/pc/hidapi-libusb-static.pc.in" "hidapi-libusb")
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 504b205..c8b0d4b 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -51,6 +51,9 @@ function(hidapi_configure_pc PC_IN_FILE)
     set(VERSION "${VERSION}${VERSION_SUFFIX}")
     set(prefix "${CMAKE_INSTALL_PREFIX}")
     set(exec_prefix "\${prefix}")
+    foreach(arg IN LISTS ARGN)
+        set(OUTPUT_NAME "${arg}")
+    endforeach()
     if(IS_ABSOLUTE "${CMAKE_INSTALL_LIBDIR}")
         set(libdir "${CMAKE_INSTALL_LIBDIR}")
     else()

Once all this stuff built and installed, the user may chose between static or dynamic linking by choosing a different *pc file :

$ pkg-config --libs hidapi-libusb
-lhidapi-libusb
$ pkg-config --libs hidapi-libusb-static --static
-l:libhidapi-libusb.a

I was just answering the question how a user can choose to link to the static library when static and dynamic are both installed. I do not know cmake more than that.

Youw · 2022-06-15T10:27:31Z

pkg-config --libs hidapi-libusb-static --static

Technically that is an entirely different library, which happens to have (almost) the same name. But thanks for suggestion.

I was just answering the question how a user can choose to link to the static library when static and dynamic are both installed. I do not know cmake more than that.

I guess the complete answer I was looking for:

to link all autotools dependencies statically: LDFLAGS="-static" PKG_CONFIG="pkg-config --static"
to link a specific library statically - almost certainly need to hard-code a static library name

netfab · 2022-06-17T08:21:53Z

pkg-config --libs hidapi-libusb-static --static

Technically that is an entirely different library, which happens to have (almost) the same name.

To be entirely complete on this subject, the additional pc file can be installed in a non-standard directory :

$ pwd
/usr/local/share/pkgconfig

$ ls
hidapi-libusb.pc

$ pkg-config --libs hidapi-libusb
-lhidapi-libusb

$ PKG_CONFIG_LIBDIR=/usr/local/share/pkgconfig pkg-config --libs hidapi-libusb --static
-l:libhidapi-libusb.a

$ pkg-config --list-all | grep hidapi-libusb
hidapi-libusb                  hidapi-libusb - C Library for USB HID device access from
                                      Linux, Mac OS X, FreeBSD, and Windows.
                                      This is the libusb implementation.

$ PKG_CONFIG_LIBDIR=/usr/local/share/pkgconfig pkg-config --list-all
hidapi-libusb                  hidapi-libusb - C Library for USB HID device access from
                                      Linux, Mac OS X, FreeBSD, and Windows.
                                      This is the libusb implementation, static version
                                      (you must use the pkg-config --static option).

But, yes, it seems that there is no other way.

mcuee · 2022-08-16T13:52:32Z

Interesting there are similar discussions here -- looks like a common CMake question.

Unable to build libavrdude dynamic and static library with CMake buildsystem avrdudes/avrdude#952

My comment here:

Add CMakeList for Windows Unix and Android platforms libusb#1134 (comment)

For the library developers (like libusb project), this is required if the developers need to release the binaries.
But I do not think this is an issue for CMake. libftdi project is using CMake and it provides both shared and static libraries in single build.
http://developer.intra2net.com/git/?p=libftdi

mcuee · 2022-08-16T13:56:22Z

For example, if I only build the default dynamic link version, then I will get wrong static link option using pkg-config.

mcuee@UbuntuSwift3:~/build$ pkg-config --libs hidapi-libusb --static
-L/usr/local/lib -lhidapi-libusb

Youw · 2022-08-16T14:02:54Z

So far I haven't seen a single pkg-config package, that would provide link options to link against static version of the library, when pass --static paramater to pkg-config.

Youw · 2022-08-16T14:06:16Z

a common CMake question

A common question of those who started using CMake after Autotools. Context/bias is important.

mcuee · 2022-08-16T14:09:40Z

So far I haven't seen a single pkg-config package, that would provide link options to link against static version of the library, when pass --static paramater to pkg-config.

I can see libusb-1.0 (autools) and libftdi-1.0 (CMake) try to differentiate between dynamic linking and static linking.

mcuee@UbuntuSwift3:~/build/hidapi/build$ pkg-config --libs libusb-1.0
-L/usr/local/lib -lusb-1.0
mcuee@UbuntuSwift3:~/build/hidapi/build$ pkg-config --static --libs libusb-1.0
-L/usr/local/lib -lusb-1.0 -ludev -latomic -lpthread
mcuee@UbuntuSwift3:~/build/hidapi/build$ pkg-config --libs libftdi1
-L/usr/local/lib -lftdi1 -lusb-1.0
mcuee@UbuntuSwift3:~/build/hidapi/build$ pkg-config --libs libftdi1 --static
-L/usr/local/lib -lftdi1 -lusb-1.0 -ludev -latomic -lpthread

Youw · 2022-08-16T14:12:28Z

mcuee@UbuntuSwift3:~/build/hidapi/build$ pkg-config --static --libs libusb-1.0
-L/usr/local/lib -lusb-1.0 -ludev -latomic -lpthread

True, but pass those to the linker of you own library/application - you're still going to be linked against shared/.so version of libusb, right?

mcuee · 2022-08-16T14:13:56Z

@Youw
Yes I agree they are not correct either -- use use libusb-1.0.a instead. I remember macOS is correct. I will check again.

mcuee · 2022-08-16T14:18:04Z

@Youw

avrdude-libhidapi is basically repackged hidapi (old version) adapted to MSVC static build. For MSVC, it makes sense to use static build. Is it possible to modify hidapi so that there is no need for avrdude to use a fork version of hidapi?

I tend to see that avrdude has legitimate reasons to use avrdude-libwinusb (at least when libusb-1.0.23/1.0.24 are broken under Windows for libusb0.sys) and avrdude-libftdi (to use FTDI vendor driver) for MSVC build (Static linking) but no good reasons to use a fork version of hidapi for MSVC build.

Ref: avrdudes/libhidapi@e3700e9

Youw · 2022-08-16T14:26:49Z

🤯
Agree, there is absolutely no reasons for that.

mcuee · 2022-11-06T12:13:46Z

It seems to me that we are not going to fix this one. Maybe this can be closed.

Youw · 2022-11-06T12:47:11Z

let me make a readme section as a summary of this one, and then we can close it

Document the recommended way to build both Shared and Static libraries of HIDAPI using CMake. Closes: #424

* doc: Shared+Static build for CMake Document the recommended way to build both Shared and Static libraries of HIDAPI using CMake. Closes: #424

MaesterZ · 2023-03-13T11:15:59Z

Thanks for the documentation and the help overall 🙏🏻

Youw added the question Information is requested label Jun 12, 2022

mcuee mentioned this issue Nov 1, 2022

libusb-config --libs is incorrect for static builds libusb/libusb-compat-0.1#15

Closed

Youw added a commit that referenced this issue Mar 12, 2023

doc: Shared+Static build for CMake

b8ba6d3

Document the recommended way to build both Shared and Static libraries of HIDAPI using CMake. Closes: #424

Youw mentioned this issue Mar 12, 2023

doc: Shared+Static build for CMake #517

Merged

Youw closed this as completed in #517 Mar 13, 2023

Youw added a commit that referenced this issue Mar 13, 2023

doc: Shared+Static build for CMake (#517)

7eedb61

* doc: Shared+Static build for CMake Document the recommended way to build both Shared and Static libraries of HIDAPI using CMake. Closes: #424

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building both static and shared libraries at the same time #424

Building both static and shared libraries at the same time #424

MaesterZ commented Jun 10, 2022 •

edited

Youw commented Jun 10, 2022 •

edited

Youw commented Jun 10, 2022 •

edited

netfab commented Jun 14, 2022 •

edited

Youw commented Jun 14, 2022 •

edited

netfab commented Jun 15, 2022

Youw commented Jun 15, 2022

netfab commented Jun 17, 2022 •

edited

mcuee commented Aug 16, 2022 •

edited

mcuee commented Aug 16, 2022

Youw commented Aug 16, 2022

Youw commented Aug 16, 2022

mcuee commented Aug 16, 2022

Youw commented Aug 16, 2022

mcuee commented Aug 16, 2022

mcuee commented Aug 16, 2022 •

edited

Youw commented Aug 16, 2022

mcuee commented Nov 6, 2022

Youw commented Nov 6, 2022

MaesterZ commented Mar 13, 2023

Building both static and shared libraries at the same time #424

Building both static and shared libraries at the same time #424

Comments

MaesterZ commented Jun 10, 2022 • edited

Youw commented Jun 10, 2022 • edited

Youw commented Jun 10, 2022 • edited

netfab commented Jun 14, 2022 • edited

Youw commented Jun 14, 2022 • edited

netfab commented Jun 15, 2022

Youw commented Jun 15, 2022

netfab commented Jun 17, 2022 • edited

mcuee commented Aug 16, 2022 • edited

mcuee commented Aug 16, 2022

Youw commented Aug 16, 2022

Youw commented Aug 16, 2022

mcuee commented Aug 16, 2022

Youw commented Aug 16, 2022

mcuee commented Aug 16, 2022

mcuee commented Aug 16, 2022 • edited

Youw commented Aug 16, 2022

mcuee commented Nov 6, 2022

Youw commented Nov 6, 2022

MaesterZ commented Mar 13, 2023

MaesterZ commented Jun 10, 2022 •

edited

Youw commented Jun 10, 2022 •

edited

Youw commented Jun 10, 2022 •

edited

netfab commented Jun 14, 2022 •

edited

Youw commented Jun 14, 2022 •

edited

netfab commented Jun 17, 2022 •

edited

mcuee commented Aug 16, 2022 •

edited

mcuee commented Aug 16, 2022 •

edited