TLS issues with cohttp 5.0.0

[arvidj]

I'm getting errors like:

[error] No SSL or TLS support compiled into Conduit

with recent versions of ocaml-gitlab. I have made sure the tls package is installed.

Here's my list of packages:

  ∗ install conf-pkg-config     2
  ∗ install conf-gmp            4
  ∗ install cmdliner            1.1.1
  ∗ install seq                 base
  ∗ install ocamlbuild          0.14.2
  ∗ install ocamlfind           1.9.5
  ∗ install dune                3.7.0
  ∗ install conf-gmp-powm-sec   3
  ∗ install uchar               0.0.2
  ∗ install zarith              1.12
  ∗ install topkg               1.0.7
  ∗ install num                 1.4
  ∗ install base-bytes          base
  ∗ install stringext           1.6.0
  ∗ install stdlib-shims        0.3.0
  ∗ install sexplib0            v0.15.1
  ∗ install result              1.5
  ∗ install re                  1.10.4
  ∗ install ppx_derivers        1.2.1
  ∗ install ocaml-syntax-shims  1.0.0
  ∗ install ocaml-compiler-libs v0.12.4
  ∗ install menhirSdk           20220210
  ∗ install menhirLib           20220210
  ∗ install magic-mime          1.3.0
  ∗ install macaddr             5.3.1
  ∗ install gmap                0.3.0
  ∗ install easy-format         1.3.4
  ∗ install duration            0.2.1
  ∗ install domain-name         0.4.0
  ∗ install csexp               1.5.1
  ∗ install cppo                1.6.9
  ∗ install camlp-streams       5.0.1
  ∗ install bigstringaf         0.9.0
  ∗ install base64              3.5.1
  ∗ install ISO8601             0.2.6
  ∗ install uutf                1.0.3
  ∗ install rresult             0.7.0
  ∗ install ptime               1.1.0
  ∗ install fmt                 0.9.0
  ∗ install astring             0.8.5
  ∗ install ppxlib              0.29.1
  ∗ install menhir              20220210
  ∗ install ipaddr              5.3.1
  ∗ install dune-configurator   3.7.0
  ∗ install yojson              2.0.2
  ∗ install ocplib-endian       1.2
  ∗ install biniou              1.2.2
  ∗ install angstrom            0.15.0
  ∗ install jsonm               1.0.1
  ∗ install cstruct             6.1.1
  ∗ install fpath               0.7.3
  ∗ install base                v0.15.1
  ∗ install atd                 2.11.0
  ∗ install lwt                 5.6.1
  ∗ install atdgen-runtime      2.11.0
  ∗ install uri                 4.2.0
  ∗ install hex                 1.5.0
  ∗ install eqaf                0.9
  ∗ install asn1-combinators    0.2.6
  ∗ install ppx_sexp_conv       v0.15.1
  ∗ install parsexp             v0.15.0
  ∗ install logs                0.7.0
  ∗ install atdgen              2.11.0
  ∗ install ezjsonm             1.3.0
  ∗ install mirage-crypto       0.11.0
  ∗ install uri-sexp            4.2.0
  ∗ install ipaddr-sexp         5.3.1
  ∗ install sexplib             v0.15.1
  ∗ install bos                 0.2.1
  ∗ install tezt                3.0.0
  ∗ install pbkdf               1.2.0
  ∗ install mirage-crypto-rng   0.11.0
  ∗ install hkdf                1.0.4
  ∗ install cohttp              5.0.0
  ∗ install ppx_cstruct         6.1.1
  ∗ install cstruct-sexp        6.1.1
  ∗ install conduit             6.2.0
  ∗ install mirage-crypto-pk    0.11.0
  ∗ install mirage-crypto-ec    0.11.0
  ∗ install cohttp-lwt          5.0.0
  ∗ install conduit-lwt         6.2.0
  ∗ install x509                0.16.4
  ∗ install gitlab              0.1.7
  ∗ install tls                 0.16.0
  ∗ install ca-certs            0.2.3
  ∗ install conduit-lwt-unix    6.2.0
  ∗ install cohttp-lwt-unix     5.0.0
  ∗ install gitlab-unix         0.1.7

The problems go away when I restrict cohttp to < 5.0.0. Here's the list of packages I get in this case:

# Name                # Installed # Synopsis
angstrom              0.15.0      Parser combinators built for speed and memory-efficiency
asn1-combinators      0.2.6       Embed typed ASN.1 grammars in OCaml
astring               0.8.5       Alternative String module for OCaml
atd                   2.11.0      Parser for the ATD data format description language
atdgen                2.11.0      Generates efficient JSON serializers, deserializers and validators
atdgen-runtime        2.11.0      Runtime library for code generated by atdgen
base                  v0.15.1     Full standard library replacement for OCaml
base-bigarray         base
base-bytes            base        Bytes library distributed with the OCaml compiler
base-threads          base
base-unix             base
base64                3.5.1       Base64 encoding for OCaml
bigstringaf           0.9.0       Bigstring intrinsics and fast blits based on memcpy/memmove
biniou                1.2.2       Binary data format designed for speed, safety, ease of use and backward compatibility as protocols evolve
bos                   0.2.1       Basic OS interaction for OCaml
ca-certs              0.2.3       Detect root CA certificates from the operating system
camlp-streams         5.0.1       The Stream and Genlex libraries for use with Camlp4 and Camlp5
cmdliner              1.1.1       Declarative definition of command line interfaces for OCaml
cohttp                4.0.0       An OCaml library for HTTP clients and servers
cohttp-lwt            4.0.0       CoHTTP implementation using the Lwt concurrency library
cohttp-lwt-unix       4.0.0       CoHTTP implementation for Unix and Windows using Lwt
conduit               4.0.2       A network connection establishment library
conduit-lwt           4.0.2       A portable network connection establishment library using Lwt
conduit-lwt-unix      4.0.2       A network connection establishment library for Lwt_unix
conf-gmp              4           Virtual package relying on a GMP lib system installation
conf-gmp-powm-sec     3           Virtual package relying on a GMP lib with constant-time modular exponentiation
conf-pkg-config       2           Check if pkg-config is installed and create an opam switch local pkgconfig folder
cppo                  1.6.9       Code preprocessor like cpp for OCaml
csexp                 1.5.1       Parsing and printing of S-expressions in Canonical form
cstruct               6.1.1       Access C-like structures directly from OCaml
cstruct-sexp          6.1.1       S-expression serialisers for C-like structures
domain-name           0.4.0       RFC 1035 Internet domain names
dune                  3.7.0       Fast, portable, and opinionated build system
dune-configurator     3.7.0       Helper library for gathering system configuration
duration              0.2.1       Conversions to various time units
easy-format           1.3.4       High-level and functional interface to the Format module of the OCaml standard library
eqaf                  0.9         Constant-time equal function on string
ezjsonm               1.3.0       Simple interface on top of the Jsonm JSON library
fmt                   0.9.0       OCaml Format pretty-printer combinators
fpath                 0.7.3       File system paths for OCaml
gitlab                0.1.7       GitLab APIv4 OCaml library
gitlab-unix           0.1.7       GitLab APIv4 OCaml library
gmap                  0.3.0       Heterogenous maps over a GADT
hex                   1.5.0       Library providing hexadecimal converters
hkdf                  1.0.4       HMAC-based Extract-and-Expand Key Derivation Function (RFC 5869)
ISO8601               0.2.6       ISO 8601 and RFC 3999 date parsing for OCaml
ipaddr                5.3.1       A library for manipulation of IP (and MAC) address representations
ipaddr-sexp           5.3.1       A library for manipulation of IP address representations using sexp
jsonm                 1.0.1       Non-blocking streaming JSON codec for OCaml
logs                  0.7.0       Logging infrastructure for OCaml
lwt                   5.6.1       Promises and event-driven I/O
macaddr               5.3.1       A library for manipulation of MAC address representations
magic-mime            1.3.0       Map filenames to common MIME types
menhir                20220210    An LR(1) parser generator
menhirLib             20220210    Runtime support library for parsers generated by Menhir
menhirSdk             20220210    Compile-time library for auxiliary tools related to Menhir
mirage-crypto         0.10.7      Simple symmetric cryptography for the modern age
mirage-crypto-ec      0.10.7      Elliptic Curve Cryptography with primitives taken from Fiat
mirage-crypto-pk      0.10.7      Simple public-key cryptography for the modern age
mirage-crypto-rng     0.10.7      A cryptographically secure PRNG
mirage-no-solo5       1           Virtual package conflicting with mirage-solo5
mirage-no-xen         1           Virtual package conflicting with mirage-xen
mtime                 2.0.0       Monotonic wall-clock time for OCaml
num                   1.4         The legacy Num library for arbitrary-precision integer and rational arithmetic
ocaml                 4.14.1      The OCaml compiler (virtual package)
ocaml-base-compiler   4.14.1      Official release 4.14.1
ocaml-compiler-libs   v0.12.4     OCaml compiler libraries repackaged
ocaml-config          2           OCaml Switch Configuration
ocaml-options-vanilla 1           Ensure that OCaml is compiled with no special options enabled
ocaml-syntax-shims    1.0.0       Backport new syntax to older OCaml versions
ocamlbuild            0.14.2      OCamlbuild is a build system with builtin rules to easily build most OCaml projects
ocamlfind             1.9.5       A library manager for OCaml
ocplib-endian         1.2         Optimised functions to read and write int16/32/64 from strings and bigarrays
parsexp               v0.15.0     S-expression parsing library
pbkdf                 1.2.0       Password based key derivation functions (PBKDF) from PKCS#5
ppx_cstruct           6.1.1       Access C-like structures directly from OCaml
ppx_derivers          1.2.1       Shared [@@deriving] plugin registry
ppx_sexp_conv         v0.15.1     [@@deriving] plugin to generate S-expression conversion functions
ppxlib                0.29.1      Standard library for ppx rewriters
ptime                 1.1.0       POSIX time for OCaml
re                    1.10.4      RE is a regular expression library for OCaml
result                1.5         Compatibility Result module
rresult               0.7.0       Result value combinators for OCaml
seq                   base        Compatibility package for OCaml's standard iterator type starting from 4.07.
sexplib               v0.15.1     Library for serializing OCaml values to and from S-expressions
sexplib0              v0.15.1     Library containing the definition of S-expressions and some base converters
stdlib-shims          0.3.0       Backport some of the new stdlib features to older compiler
stringext             1.6.0       Extra string functions for OCaml
tezt                  3.0.0       Test framework for unit tests, integration tests, and regression tests
tls                   0.15.4      Transport Layer Security purely in OCaml
topkg                 1.0.7       The transitory OCaml software packager
uchar                 0.0.2       Compatibility library for OCaml's Uchar module
uri                   4.2.0       An RFC3986 URI/URL parsing library
uri-sexp              4.2.0       An RFC3986 URI/URL parsing library
uutf                  1.0.3       Non-blocking streaming Unicode codec for OCaml
x509                  0.16.4      Public Key Infrastructure (RFC 5280, PKCS) purely in OCaml
yojson                2.0.2       Yojson is an optimized parsing and printing library for the JSON format
zarith                1.12        Implements arithmetic and logical operations over arbitrary-precision integers

[tmcgilchrist]

Thanks for the report @arvidj. That is strange cohttp doesn't seem to be pulling in the ssl or tls dependency correctly in 5.0.0. I'll do some investigation.

[arvidj]

Thanks, I'd be happy to provide more details if you there's something specific you need :)

[arvidj]

Got bit by this again. This time with the package set:

  - install conduit             6.2.0
  - install mirage-crypto-pk    0.10.7
  - install cohttp              4.1.2
  - install conduit-lwt         6.2.0
  - install x509                0.11.2
  - install cohttp-lwt          4.1.2
  - install tls                 0.12.8
  - install ca-certs            0.2.0
  - install gitlab              0.1.7
  - install conduit-lwt-unix    6.2.0
  - install cohttp-lwt-unix     4.1.2
  - install gitlab-unix         0.1.7

(from here).

I'm attempting to resolve it by forcing cohttp 4.0.0, that seems to work.

Any ideas on a more prinicipled approach to this problem? Shouldn't ocaml-gitlab also depend on ssl or tls? Or is the idea that the user decides whether they want ssl or tls, as discussed here?

In my dune-project, I have:

(depends
  ocaml
  dune
  tezt
  uri
  (cohttp-lwt-unix (< 4.1.2))
  (tls (< 0.13.0))
  ISO8601
  (gitlab (>= 0.1.7))
  gitlab-unix)

Perhaps I should have tls-lwt instead of tls?

[tmcgilchrist]

The idea is the user can choose which implementation they want. But it is a surprising error if you're just picking up this library to do something. A good solution would be to document the error message in the project README and show how to fix it plus a link to the discuss thread.

[tmcgilchrist]

In my dune-project, I have:

(depends
  ocaml
  dune
  tezt
  uri
  (cohttp-lwt-unix (< 4.1.2))
  (tls (< 0.13.0))
  ISO8601
  (gitlab (>= 0.1.7))
  gitlab-unix)

Perhaps I should have tls-lwt instead of tls?

I would have tls-lwt rather than tls in that setup.