handshake failure - malformed_handshake_data on Erlang 19.1, Mac OS Sierra, Homebrew Openssl

[fp]

The explicit specification of the cipher list in hackney_ssl.erl causes TLS handshake error with AWS S3 endpoint

8> Url = <<"https://s3-us-west-2.amazonaws.com/...">>,
9> hackney:get(Url, Headers, [] ).                     
Nov 10 10:17:40 CSV00002W SSL: certify: tls_connection.erl:688:Fatal error: handshake failure - malformed_handshake_data {error,{tls_alert,"handshake failure"}}

which ultimately comes down to the following in hackney_ssl:

10> f(BO), BO = [binary, {active, false}, {packet, raw}, {versions,['tlsv1.2', 'tlsv1.1', tlsv1, sslv3]},{secure_renegotiate, true},{reuse_sessions, true},{honor_cipher_order, true},{ciphers,DC}], ssl:connect( "s3-us-west-2.amazonaws.com", 443, BO ).

=ERROR REPORT==== 10-Nov-2016::10:30:04 ===
SSL: certify: tls_connection.erl:688:Fatal error: handshake failure - malformed_handshake_data
{error,{tls_alert,"handshake failure"}}

but if we remove explicitly setting the {ciphers, ?DEFAULT_CIPHERS} option:

11> f(BO), BO = [binary, {active, false}, {packet, raw}, {versions,['tlsv1.2', 'tlsv1.1', tlsv1, sslv3]},{secure_renegotiate, true},{reuse_sessions, true},{honor_cipher_order, true}], ssl:connect( "s3-us-west-2.amazonaws.com", 443, BO ).   {ok,{sslsocket,{gen_tcp,#Port<0.12682>,tls_connection,
                        undefined},
               <0.372.0>}}

This occurs on Mac OS Sierra w/homebrew latest openssl, but it seems like it would not be limited to just this platform, although I have not had time to validate that yet.

[fp]

This appears to be related to Erlang being compiled with -DOPENSSL_NO_EC=1:

12> ssl:cipher_suites().
[{dhe_rsa,aes_256_gcm,null,sha384},
 {dhe_dss,aes_256_gcm,null,sha384},
 {dhe_rsa,aes_256_cbc,sha256},
 {dhe_dss,aes_256_cbc,sha256},
 {rsa,aes_256_gcm,null,sha384},
 {rsa,aes_256_cbc,sha256},
 {dhe_rsa,aes_128_gcm,null,sha256},
 {dhe_dss,aes_128_gcm,null,sha256},
 {dhe_rsa,aes_128_cbc,sha256},
 {dhe_dss,aes_128_cbc,sha256},
 {rsa,aes_128_gcm,null,sha256},
 {rsa,aes_128_cbc,sha256},
 {dhe_rsa,aes_256_cbc,sha},
 {dhe_dss,aes_256_cbc,sha},
 {rsa,aes_256_cbc,sha},
 {dhe_rsa,'3des_ede_cbc',sha},
 {dhe_dss,'3des_ede_cbc',sha},
 {rsa,'3des_ede_cbc',sha},
 {dhe_rsa,aes_128_cbc,sha},
 {dhe_dss,aes_128_cbc,sha},
 {rsa,aes_128_cbc,sha}]

and if I explicitly take out all of the EC ciphers from the explicit list, then it works:

13> DCnoEC = ["DHE-DSS-AES256-GCM-SHA384","DHE-DSS-AES256-SHA256",
13>  "AES256-GCM-SHA384","AES256-SHA256",
13>  "DHE-DSS-AES128-GCM-SHA256","DHE-DSS-AES128-SHA256",
13>  "AES128-GCM-SHA256","AES128-SHA256",
13>  "DHE-DSS-AES256-SHA","AES256-SHA","DHE-DSS-AES128-SHA","AES128-SHA"].
["DHE-DSS-AES256-GCM-SHA384","DHE-DSS-AES256-SHA256",
 "AES256-GCM-SHA384","AES256-SHA256",
 "DHE-DSS-AES128-GCM-SHA256","DHE-DSS-AES128-SHA256",
 "AES128-GCM-SHA256","AES128-SHA256","DHE-DSS-AES256-SHA",
 "AES256-SHA","DHE-DSS-AES128-SHA","AES128-SHA"]
14> f(BO), BO = [binary, {active, false}, {packet, raw}, {versions,['tlsv1.2', 'tlsv1.1', tlsv1, sslv3]},{secure_renegotiate, true},{reuse_sessions, true},{honor_cipher_order, true},{ciphers,DCnoEC}], ssl:connect( "s3-us-west-2.amazonaws.com", 443, BO ).
{ok,{sslsocket,{gen_tcp,#Port<0.12683>,tls_connection,
                        undefined},
               <0.381.0>}}

[benoitc]

mmm so I probably need to check if these ciphers are available first. Thanks for the info. Patch is coming :)

[dustinsmith1024]

I think I am having a similar issue. I started a new project and it was using 1.6.3. I rolled it back to 1.6.1 and things seem to work now.

[benoitc]

It seems we are too strict for some servers. Will go for a less strict mode
asap
On Sat, 19 Nov 2016 at 06:40, Dustin Smith notifications@github.com wrote:

I think I am having a similar issue. I started a new project and it was
using 1.6.3. I rolled it back to 1.6.1 and things seem to work now.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#362 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAA4op0bW1_DUD-vFOJXqUIdtzhJmp0-ks5q_ou8gaJpZM4Ku34d
.

[dustinsmith1024]

Had any luck with the patch? Anything we can do to help?

[michalmuskala]

Any updates on this? It generally makes the 1.6.2 and 1.6.3 versions unusable.

[benoitc]

i don't reproduce it on macosx sierra with latest openssl from homebrew. but a fix is coming.

[michalmuskala]

Thank you.

[gVolop]

any updates?

use hackney 1.6.6 but still receive same error:
[error] SSL: :certify: tls_connection.erl:704:Fatal error: handshake failure - malformed_handshake_data

     {:hackney, "1.6.6", override: true},

macosx sierra
OpenSSL 0.9.8zh 14 Jan 2016
Erlang/OTP 19
Elixir 1.4.2

[benoitc]

can you try the latest version of hackney?

…

On Sun, 26 Feb 2017 at 12:56, Gittel Volpo ***@***.***> wrote: any updates? use hackney 1.6.6 but still receive same error: [error] SSL: :certify: tls_connection.erl:704:Fatal error: handshake failure - malformed_handshake_data {:hackney, "1.6.6", override: true}, macosx sierra OpenSSL 0.9.8zh 14 Jan 2016 Erlang/OTP 19 Elixir 1.4.2 — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#362 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA4okfdOvRR73OZPC32q78RYA-3vmttks5rgWh0gaJpZM4Ku34d> .

[gVolop]

i'm use it. 1.6.6 (tagged 2 hours ago..)
https://github.com/benoitc/hackney/releases/tag/1.6.6

[gVolop]

      {:httpoison, "~> 0.11.0"},
      {:hackney, "1.6.6", override: true},

----> [error] SSL: :certify: tls_connection.erl:704:Fatal error: handshake failure - malformed_handshake_data

---

      {:httpoison, "~> 0.11.0"},
      {:hackney, "1.6.1", override: true},

-----> all fine

[benoitc]

@gVolop did you change the version of Erlang? Which version of openssl is installed on your system?

So far I need a way to reproduce this issue. The only change in the current version is that hackney is now more strict and use latest certificates from the Mozila project. Is this a public server? Is there any link I can test? If not what is the SSL configuration of your server?

Also can you enable the tracing in hackney and provide me a full trace?

[benoitc]

@gVolop also this ticket is closed. It may be not the same issue. I would appreciate a new ticket with all the info above if you can that would help a lot :)

[gVolop]

@benoitc here is the new ticket
hope all is there. if you have need additional details i would be happy to help. =)