Gettin TLS Alert "Handshake Failure" for many HTTPS sites

[orbitz]

I'm testing a bunch of random HTTPS URLs on the internet using ocaml-tls to see if I can use it for some production workloads. Around 10% of them are failing with a handshake failure.

I'm not entirely sure how to debug this but based on comparing a a tcpdump of a successful curl to my ocaml-tls test I can see that curl sends 71 cipher suites it supports and they negotiate to using ECDHE-RSA-AES256-GCM-SHA384. The ocaml-tls handshake sends 14 and there are no ECDHE suites in it. This seems pretty consistent among the URLs I've checked that fail. I'm guessing these sites are requiring ECDHE-RSA and ocaml-tls does not support it yet, based on the list in the Ciphersuites module.

Is ocaml-tls planning on supporting the ECDHE class of ciphers? Or expanding the ciphers it supports to the full range? Or have I misdiagnosed the error?

An example URL that I am experiencing this on is:

https://www.varnish-cache.org/content/varnish-cache-400

[hannesm]

Thanks for the analysis, and it is correct that ocaml-tls does not support elliptic curve ciphersuites right now, because nocrypto does not yet support ECC primitives.

[orbitz]

Is there a roadmap to implement these? I'd like to use ocaml-tls in production but this is a bit of a deal killer for my use-case. Do you have any suggestions for an alternative to ocaml-tls otherwise?

[annubiz]

Might be re-neogiation support not aligned between the two ends. On helpful tool is the QualsysSSLServerTest site : https://www.ssllabs.com/ssltest/ Where you can enter in the site your looking to test against. The SSLTest site attempts to connect at most conceivable combinations of ssl parameters (including cyrypto types) and displays a report when its done listing the types of support one would hope to see, and they types of depricated support that old sites still maintain. Might help you track down the combination of settings that are different between your non-functional sites and what ocaml-tls requires.

…

On 03/04/2017 04:08 PM, orbitz wrote: I'm testing a bunch of random HTTPS URLs on the internet using ocaml-tls to see if I can use it for some production workloads. Around 10% of them are failing with a handshake failure. I'm not entirely sure how to debug this but based on comparing a a tcpdump of a successful curl to my ocaml-tls test I can see that curl sends 71 cipher suites it supports and they negotiate to using `ECDHE-RSA-AES256-GCM-SHA384`. The ocaml-tls handshake sends 14 and there are no ECDHE suites in it. This seems pretty consistent among the URLs I've checked that fail. I'm guessing these sites are requiring ECDHE-RSA and ocaml-tls does not support it yet, based on the list in the `Ciphersuites` module. Is ocaml-tls planning on supporting the ECDHE class of ciphers? Or expanding the ciphers it supports to the full range? Or have I misdiagnosed the error? An example URL that I am experiencing this on is: `https://www.varnish-cache.org/content/varnish-cache-400`

[hannesm]

@orbitz I think @pqwy has some ECC code for nocrypto. Maybe he can outline a roadmap here.

[orbitz]

Any update on this? This week I'll probably need to either figure out if I can implement some ECC myself or switch to another set of bindings.

[hannesm]

@orbitz I don't have any ECC code. since @pqwy seems to be rather non-responsive, if you implement ECC suites, we can incorporate them into OCaml-TLS.

[orbitz]

Is it a bad idea to implement ECC suites myself as I'm not a crypto person? Any tips for how to do this? Can I cheat and call some C code for now?

[hannesm]

sure, you can use C code. I'm not sure how much time I have to look through PRs and help out in the near future (I do have a pile of other things on my plate atm).

[cfcs]

It is unclear to me at this point which curves are actually considered "nice," there seems to be different opinions on that. See this section in the RFC for how the selection is negotiated: https://tools.ietf.org/html/rfc4492#section-5.1

Depending on which curves your target server supports, you may be able to get away with implementing only curve25519 support and use libnacl/libsodium bindings such as the sodium module by David Sheets: https://github.com/dsheets/ocaml-sodium

Also found this by quick github search: https://github.com/nickgian/ECC-OCaml

[pqwy]

• Yes: if you consider yourself "not a crypto person", then it's an extremely bad idea to implement the ECC suites yourself, just as you would expect. ECC are the most error-prone crypto primitive group, both in their ground logic and the extremely finicky timing behavior. This is especially true for the weierstrass curves, like the Brainpool set, or the NIST-standardized ones. Curve25519 is a bit of an oddball in that it is given by a different equation. In the opinion of a certain celebrity cryptographer championing that formulation, it should be less error-prone to implement. But there is much less literature on that.

• One of the core tenets behind the Mirleft stuff was avoiding C, for well documented reasons, whenever this is practical. The practicality was largely determined by performance considerations. The compromises are things like offloading the bignums to GMP, and AES to the on-CPU implementation. And like with so many other algorithms covered, the performance of ECC comes from the underlying field, while the complexity and pitfalls of the implementation come from the code that builds a group out of that. Hence, doing ECC in C, especially plugging in pre-existing C implementations, especially especially if these are the weierstrass curves, runs directly counter to everything that this combined set of projects tried to do. It also runs directly against all of our published work on it.

By all means, feel free to try to add ECC in whichever way you see fit. But for these reasons, it's my position that the results of such work are not likely to get merged.

Now for my part, I'm on holiday and I'll see if I can devote some time to to finishing the old ECC stuff, but right now I can't promise anything.

Also, @orbitz -- you never mentioned which production you'd like to use this in?

[orbitz]

@pqwy I'm implementing a tool which performs HTTP/S queries on client-provided URLs, so I don't have control over how they setup their webservers.

For me there is a bigger question:

Is the goal of ocaml-tls to be a production-quality TLS implementation and keep up with the whole spec or not? If it is not meant to be production-quality, then I unfortunately will have to switch to a different underlying TLS implementation that will get more attention.

It's perfectly fine if the goal of ocaml-tls is not to be production-quality, I just need to know if I should base production tools on it. I just can't be in a situation where I depend it for making money and find a feature is missing. Normally this is OK but since I don't know anything about crypto I don't think I can implement missing features myself :(

As a user of ocaml-tls is there anything I can do to move it toward production? I unfortunately don't have any money to donate to it yet.

[cfcs]

First a general comment: One problem with having a "production quality" implementation, apart from writing quality code, is that the combined selection of options specified in the whole TLS specification_s_ (am I wrong to assume your site implements the deprecated TLS v1.2?) is nowhere near "production quality" -- in fact you have to walk a very thin line as to not accidentally hurt yourself with obscure cipher modes, weak keying algorithms, and outright flawed (and/or broken) cryptography.

I think in this scenario you would probably want to use openssl or gnutls bindings, if you just want to talk to as many TLS servers as possible and don't care about security implications.

I have projects that require making SSLv3 connections (pre-TLSv1.0), but I realize that this is a niche thing, and I believe that there should be a strong distinction in library interfaces between "best practice" crypto for new applications vs "toy crypto" for talking to arcane systems.

ECC is not exactly toy crypto, but as @pqwy pointed out it is still a subject of (hot) debate, so I think it's fair to say its use is debatable. Even the cryptographers can't agree on what comprises a secure ECC cryptosystem, or how to safely implement it. Personally I think the arguments put forward by Lange/Bernstein sound sensible, but I am not a mathematician and have no way to evaluate them.

[orbitz]

am I wrong to assume your site implements the deprecated TLS v1.2

Is TLS v1.2 deprecated? As far as I am aware v1.3 is still in draft.

Here is the SSLLabs report on the domain I'm using for testing:

https://www.ssllabs.com/ssltest/analyze.html?d=www.varnish-cache.org&s=176.58.90.154&latest

The host gets an A grade from ssllabs.

I guess I'm a bit confused now. When I opened this issue my belief was that there either wasn't enough user-driven motivation to implement ECC or that it was on the roadmap but just hadn't been gotten to yet. Is the case that ECC is not in ocaml-tls because it is being debated?

Maybe an alternative is that the project could have a goal of supporting any site that has an A grade from SSLLabs, that way it is easy for someone like me to understand if a TLS connection should be expected to work or not?

[talex5]

BTW, if this does get implemented, I'd be interested in using ocaml-tls in 0install. I tried using the current version, but it failed on the first site I tried (gitlab.io) because (I think) they didn't have any ciphers in common.

[hannesm]

now that tls 0.12.0 is released which includes TLS 1.3 support, and the mandated P-256 and X25519 ECDHE, I'm closing this issue. if you encounter problems connecting to servers, please report a new issue with the specific server name.

[hannesm]

A report with a server using TLS 1.2 and ECDHE was done in #413, support for ECDHE and TLS < 1.3 has been developed in #414.