Vendored HdrHistogram_c library build error on FreeBSD.

[tmcgilchrist]

Seems to be an issue with hdr_endian.h and not having the right defined(__FreeBSD__) or redefining what is present in <sys/endian.h>

Relevant log section and full logs

[ 69%] Linking C executable hdr_histogram_log_test
*** Error code 1

Stop.
make[2]: stopped in /src/_build/default/lib/libhdr_histogram/_build
*** Error code 1

Stop.
make[1]: stopped in /src/_build/default/lib/libhdr_histogram/_build
*** Error code 1

Stop.
make: stopped in /src/_build/default/lib/libhdr_histogram/_build
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:51:10: warning: 'be16toh' macro redefined [-Wmacro-redefined]
#       define be16toh(x) betoh16(x)
               ^
/usr/include/sys/_endian.h:111:9: note: previous definition is here
#define be16toh(x)      __bswap16((x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:52:10: warning: 'le16toh' macro redefined [-Wmacro-redefined]
#       define le16toh(x) letoh16(x)
               ^
/usr/include/sys/_endian.h:114:9: note: previous definition is here
#define le16toh(x)      ((uint16_t)(x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:10: warning: 'be32toh' macro redefined [-Wmacro-redefined]
#       define be32toh(x) betoh32(x)
               ^
/usr/include/sys/_endian.h:112:9: note: previous definition is here
#define be32toh(x)      __bswap32((x)

        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:55:10: warning: 'le32toh' macro redefined [-Wmacro-redefined]
#       define le32toh(x) letoh32(x)
               ^
/usr/include/sys/_endian.h:115:9: note: previous definition is here
#define le32toh(x)      ((uint32_t)(x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:10: warning: 'be64toh' macro redefined [-Wmacro-redefined]
#       define be64toh(x) betoh64(x)
               ^
/usr/include/sys/_endian.h:113:9: note: previous definition is here
#define be64toh(x)      __bswap64((x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:58:10: warning: 'le64toh' macro redefined [-Wmacro-redefined]
#       define le64toh(x) letoh64(x)
               ^
/usr/include/sys/_endian.h:116:9: note: previous definition is here
#define le64toh(x)      ((uint64_t)(x))
        ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:273:24: warning: implicit declaration of function 'betoh16' is invalid in C99 [-Wimplicit-function-declaration]
        h->counts[i] = be16toh(counts_data[i]);
                       ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:51:21: note: expanded from macro 'be16toh'
#       define be16toh(x) betoh16(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:282:24: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
        h->counts[i] = be32toh(counts_data[i]);
                       ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:291:24: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
        h->counts[i] = be64toh(counts_data[i]);
                       ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       define be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:380:22: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compressed_len = be32toh(compression_flyweight->length);
                     ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:404:32: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
    lowest_discernible_value = be64toh(encoding_flyweight.lowest_discernible_value);
                               ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       define be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:478:25: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compressed_length = be32toh(compression_flyweight->length);
                        ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:503:32: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
    lowest_discernible_value = be64toh(encoding_flyweight.lowest_discernible_value);
                               ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       define be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:580:25: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compressed_length = be32toh(compression_flyweight->length);
                        ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:604:32: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
    lowest_discernible_value = be64toh(encoding_flyweight.lowest_discernible_value);
                               ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       define be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:674:42: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compression_cookie = get_cookie_base(be32toh(compression_flyweight->cookie));
                                         ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
16 warnings generated.
/src/_build/default/lib/libhdr_histogram/src/hdr_time.c:80:2: warning: #warning is a language extension [-Wpedantic]
#warning "Platform not supported\n"
 ^
/src/_build/default/lib/libhdr_histogram/src/hdr_time.c:80:2: warning: "Platform not supported\n" [-W#warnings]
2 warnings generated.
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:51:10: warning: 'be16toh' macro redefined [-Wmacro-redefined]
#       define be16toh(x) betoh16(x)
               ^
/usr/include/sys/_endian.h:111:9: note: previous definition is here
#define be16toh(x)      __bswap16((x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:52:10: warning: 'le16toh' macro redefined [-Wmacro-redefined]
#       define le16toh(x) letoh16(x)
               ^
/usr/include/sys/_endian.h:114:9: note: previous definition is here
#define le16toh(x)      ((uint16_t)(x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:10: warning: 'be32toh' macro redefined [-Wmacro-redefined]
#       define be32toh(x) betoh32(x)
               ^
/usr/include/sys/_endian.h:112:9: note: previous definition is here
#define be32toh(x)      __bswap32((x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:55:10: warning: 'le32toh' macro redefined [-Wmacro-redefined]
#       define le32toh(x) letoh32(x)
               ^
/usr/include/sys/_endian.h:115:9: note: previous definition is here
#define le32toh(x)      ((uint32_t)(x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:10: warning: 'be64toh' macro redefined [-Wmacro-redefined]
#       define be64toh(x) betoh64(x)
               ^
/usr/include/sys/_endian.h:113:9: note: previous definition is here
#define be64toh(x)      __bswap64((x))
        ^
In file included from /src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:33:
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:58:10: warning: 'le64toh' macro redefined [-Wmacro-redefined]
#       define le64toh(x) letoh64(x)
               ^
/usr/include/sys/_endian.h:116:9: note: previous definition is here
#define le64toh(x)      ((uint64_t)(x))
        ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:273:24: warning: implicit declaration of function 'betoh16' is invalid in C99 [-Wimplicit-function-declaration]
        h->counts[i] = be16toh(counts_data[i]);
                       ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:51:21: note: expanded from macro 'be16toh'
#       define be16toh(x) betoh16(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:282:24: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
        h->counts[i] = be32toh(counts_data[i]);
                       ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:291:24: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
        h->counts[i] = be64toh(counts_data[i]);
                       ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       define be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:380:22: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compressed_len = be32toh(compression_flyweight->length);
                     ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:404:32: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
    lowest_discernible_value = be64toh(encoding_flyweight.lowest_discernible_value);
                               ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       d
fine be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:478:25: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compressed_length = be32toh(compression_flyweight->length);
                        ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:503:32: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
    lowest_discernible_value = be64toh(encoding_flyweight.lowest_discernible_value);
                               ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       define be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:580:25: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compressed_length = be32toh(compression_flyweight->length);
                        ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:604:32: warning: implicit declaration of function 'betoh64' is invalid in C99 [-Wimplicit-function-declaration]
    lowest_discernible_value = be64toh(encoding_flyweight.lowest_discernible_value);
                               ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:57:21: note: expanded from macro 'be64toh'
#       define be64toh(x) betoh64(x)
                          ^
/src/_build/default/lib/libhdr_histogram/src/hdr_histogram_log.c:674:42: warning: implicit declaration of function 'betoh32' is invalid in C99 [-Wimplicit-function-declaration]
    compression_cookie = get_cookie_base(be32toh(compression_flyweight->cookie));
                                         ^
/src/_build/default/lib/libhdr_histogram/src/hdr_endian.h:54:21: note: expanded from macro 'be32toh'
#       define be32toh(x) betoh32(x)
                          ^
16 warnings generated.
/src/_build/default/lib/libhdr_histogram/src/hdr_time.c:80:2: warning: #warning is a language extension [-Wpedantic]
#warning "Platform not supported\n"
 ^
/src/_build/default/lib/libhdr_histogram/src/hdr_time.c:80:2: warning: "Platform not supported\n" [-W#warnings]
2 warnings generated.
ld: error: undefined symbol: hdr_gettime
>>> referenced by hdr_histogram_log_test.c
>>>               CMakeFiles/hdr_histogram_log_test.dir/hdr_histogram_log_test.c.o:(writes_and_reads_log)
>>> referenced by hdr_histogram_log_test.c
>>>               CMakeFiles/hdr_histogram_log_test.dir/hdr_histogram_log_test.c.o:(log_reader_aggregates_into_single_histogram)

ld: error: undefined symbol: betoh32
>>> referenced by hdr_histogram_log.c
>>>               hdr_histogram_log.c.o:(hdr_decode_compressed) in archive ../src/libhdr_histogram_static.a
>>> referenced by hdr_histogram_log.c
>>>               hdr_histogram_log.c.o:(hdr_decode_compressed_v0) in archive ../src/libhdr_histogram_static.a
>>> referenced by hdr_histogram_log.c
>>>               hdr_histogram_log.c.o:(hdr_decode_compressed_v0) in archive ../src/libhdr_histogram_static.a
>>> referenced 14 more times

ld: error: undefined symbol: betoh64
>>> referenced by hdr_histogram_log.c
>>>               hdr_histogram_log.c.o:(hdr_decode_compressed_v0) in archive ../src/libhdr_histogram_static.a
>>> referenced by hdr_histogram_log.c
>>>               hdr_histogram_log.c.o:(hdr_decode_compressed_v0) in archive ../src/libhdr_histogram_static.a
>>> referenced by hdr_histogram_log.c
>>>               hdr_histogram_log.c.o:(hdr_decode_compressed_v1) in archive ../src/libhdr_histogram_static.a
>>> referenced 6 more times

ld: error: undefined symbol: betoh16
>>> referenced by hdr_histogram_log.c
>>>               hdr_histogram_log.c.o:(apply_to_counts_16) in archive ../src/libhdr_histogram_static.a
cc: error: linker command failed with exit code 1 (use -v to see invocation)
jail: /usr/bin/su -l opam -c cd '/src' && env OCAMLCI='true' CI='true' DEPS='base-bigarray.base base-domains.base base-nnp.base base-threads.base base-unix.base bigarray-compat.1.1.0 conf-cmake.1 conf-libffi.2.0.0 conf-pkg-config.3 conf-zlib.1 csexp.1.5.2 ctypes.0.21.1 ctypes-foreign.0.21.1 dune.3.12.1 dune-configurator.3.12.1 integers.0.7.0 ocaml.5.1.1 ocaml-base-compiler.5.1.1 ocaml-config.3 ocaml-options-vanilla.1 stdlib-shims.0.3.0' OPAMCOLOR='always' CLICOLOR_FORCE='1' '/usr/bin/env' 'bash' '-c' 'opam exec -- dune build @install @check @runtest && rm -rf _build': failed
"jail" "-c" "name=obuilder_42916_5323" "mount.devfs" "path=/obuilder/result/b9ebe90c59d7f9cf5474c58f73e70627d6f5f7c4138592dc1278c5391df10c1f/rootfs" "vnet" "exec.start=/sbin/ifconfig lo0 127.0.0.1/8" "command=/usr/bin/su" "-l" "opam" "-c" "cd '/src' && env OCAMLCI='true' CI='true' DEPS='base-bigarray.base base-domains.base base-nnp.base base-threads.base base-unix.base bigarray-compat.1.1.0 conf-cmake.1 conf-libffi.2.0.0 conf-pkg-config.3 conf-zlib.1 csexp.1.5.2 ctypes.0.21.1 ctypes-foreign.0.21.1 dune.3.12.1 dune-configurator.3.12.1 integers.0.7.0 ocaml.5.1.1 ocaml-base-compiler.5.1.1 ocaml-config.3 ocaml-options-vanilla.1 stdlib-shims.0.3.0' OPAMCOLOR='always' CLICOLOR_FORCE='1' '/usr/bin/env' 'bash' '-c' 'opam exec -- dune build @install @check @runtest && rm -rf _build'" failed with exit status 1
2023-12-20 05:42.05: Job failed: Failed: Build failed
2023-12-20 05:42.05: Log analysis:
2023-12-20 05:42.05: >>> : error: undefined symbol: hdr_gettime (score = 10)
2023-12-20 05:42.05: >>> : error: undefined symbol: betoh32 (score = 10)
2023-12-20 05:42.05: >>> : error: undefined symbol: betoh64 (score = 10)
2023-12-20 05:42.05: >>> : error: undefined symbol: betoh16 (score = 10)
2023-12-20 05:42.05: >>> : error: linker command failed with exit code 1 (use -v to see invocation) (score = 10)
2023-12-20 05:42.05: undefined symbol: hdr_gettime

[kayceesrk]

Ouch. I don't have a FreeBSD machine (or the FreeBSD foo) to work on a fix. Would @dustanddreams know what the fix should be here?

[dustanddreams]

Ouch. I don't have a FreeBSD machine (or the FreeBSD foo) to work on a fix. Would @dustanddreams know what the fix should be here?

DragonFlyBSD and FreeBSD can use the same construct as OpenBSD. The following diff ought to fix this:

diff --git lib/libhdr_histogram/src/hdr_endian.h lib/libhdr_histogram/src/hdr_endian.h
index 839fdb1..6276fab 100644
--- lib/libhdr_histogram/src/hdr_endian.h
+++ lib/libhdr_histogram/src/hdr_endian.h
@@ -38,15 +38,15 @@
 #      define __BYTE_ORDER    BYTE_ORDER
 #      define __BIG_ENDIAN    BIG_ENDIAN
 #      define __LITTLE_ENDIAN LITTLE_ENDIAN
 #      define __PDP_ENDIAN    PDP_ENDIAN
 
-#elif defined(__OpenBSD__)
+#elif defined(__DragonFly__) || defined(__FreeBSD__) || defined(__OpenBSD__)
 
 #      include <sys/endian.h>
 
-#elif defined(__NetBSD__) || defined(__FreeBSD__) || defined(__DragonFly__)
+#elif defined(__NetBSD__)
 
 #      include <sys/endian.h>
 
 #      define be16toh(x) betoh16(x)
 #      define le16toh(x) letoh16(x)

[kayceesrk]

@tmcgilchrist would you be able to check whether this fixes the builds? (Not sure this can be done easily on your end since this is a CI failure.

[tmcgilchrist]

Fixed.