Ssl certificate verification failing for a _valid_ cert #240

Closed
smeevil opened this Issue Sep 9, 2015 · 19 comments

Comments

Projects
@smeevil

smeevil commented Sep 9, 2015

Hi there,

Using :

➜  erl -pa ebin -pa deps/*/ebin
Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Eshell V6.4  (abort with ^G)
1> hackney:start().
ok
2> Method = post,
2> URL = <<"https://rest-api.pay.nl/v5/Transaction/start/json">>,
2> Headers = [],
2> Payload = <<>>,
2> Options = [],
2> {ok, StatusCode, RespHeaders, ClientRef} = hackney:request(Method, URL,
2>                                                         Headers, Payload,
2>                                                         Options).

Results into an error :

=ERROR REPORT==== 9-Sep-2015::15:41:47 ===
SSL: certify: ssl_handshake.erl:1389:Fatal error: bad certificate
** exception error: no match of right hand side value {error,{tls_alert,"bad certificate"}}

While curl verification shows the cert is actually valid :

curl https://rest-api.pay.nl/v5/Transaction/start/json -v
* Hostname was NOT found in DNS cache
*   Trying 37.46.137.138...
* Connected to rest-api.pay.nl (37.46.137.138) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES256-SHA256
* Server certificate:
*    subject: C=NL; postalCode=3201BB; ST=Zuid-Holland; L=Spijkenisse; street=Voorstraat 2; O=Tintel B.V.; OU=Hosted by Cyso; OU=Wildcard SSL; CN=*.pay.nl
*    start date: 2015-08-26 00:00:00 GMT
*    expire date: 2016-09-11 23:59:59 GMT
*    subjectAltName: rest-api.pay.nl matched
*    issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Organization Validation Secure Server CA
*    SSL certificate verify ok.
> GET /v5/Transaction/start/json HTTP/1.1
@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Sep 9, 2015

Owner

i have the same issue on Erlang 18. I checked and one of the certificate at least is in the list of supported certificate so not sure what is happening for now. Working on it anyway.

Owner

benoitc commented Sep 9, 2015

i have the same issue on Erlang 18. I checked and one of the certificate at least is in the list of supported certificate so not sure what is happening for now. Working on it anyway.

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Sep 10, 2015

Owner

It seems related to Erlang or maybe to our certificate file (but that one should be the same than the one in curl... :

15> ssl:connect( "rest-api.pay.nl", 443, [{verify,verify_peer},{server_name_indication,"rest-api.pay.nl"},{depth,2},{cacertfile, "priv/ca-bundle.crt"}] ).

=ERROR REPORT==== 10-Sep-2015::11:01:31 ===
SSL: certify: ssl_handshake.erl:1476:Fatal error: bad certificate
{error,{tls_alert,"bad certificate"}}
Owner

benoitc commented Sep 10, 2015

It seems related to Erlang or maybe to our certificate file (but that one should be the same than the one in curl... :

15> ssl:connect( "rest-api.pay.nl", 443, [{verify,verify_peer},{server_name_indication,"rest-api.pay.nl"},{depth,2},{cacertfile, "priv/ca-bundle.crt"}] ).

=ERROR REPORT==== 10-Sep-2015::11:01:31 ===
SSL: certify: ssl_handshake.erl:1476:Fatal error: bad certificate
{error,{tls_alert,"bad certificate"}}

benoitc added a commit that referenced this issue Sep 10, 2015

benoitc added a commit that referenced this issue Sep 10, 2015

@shurkus

This comment has been minimized.

Show comment
Hide comment
@shurkus

shurkus Sep 18, 2015

I had a similar problem.
Sometimes sertifikate had ^L^K instead of ^S^N.

The solution was to change in public_key
(/usr/local/lib/erlang/lib/public_key/asn1/OTP-PUB-KEY.erl)
lines
%%================================
%% CommonName
%%================================
'enc_CommonName'(Val) ->
'enc_CommonName'(Val, [<<19>>]).

to

%%================================
%% CommonName
%%================================
'enc_CommonName'(Val) ->
'enc_CommonName'(Val, [<<12>>]).
and rebuild (recompile).

Method = post,URL = <<"https://rest-api.pay.nl/v5/Transaction/start/json">>,Headers = [],Payload = <<>>,Options = [],{ok, StatusCode, RespHeaders, ClientRef} = hackney:request(Method, URL,Headers, Payload,Options).

=INFO REPORT==== 18-Sep-2015::14:45:15 ===
SSL WARNING: Ignoring a CA cert as it could not be correctly decoded.

=INFO REPORT==== 18-Sep-2015::14:45:15 ===
SSL WARNING: Ignoring a CA cert as it could not be correctly decoded.

{ok,400,
[{<<"Date">>,<<"Fri, 18 Sep 2015 11:45:15 GMT">>},
{<<"Server">>,<<"Apache">>},
{<<"Set-Cookie">>,
<<"device_id=1213e89a5844c7d4481ee7c51a55291f452b8880; expires=Sat, 17-Sep-2016 11:45:1"...>>},
{<<"Set-Cookie">>,
<<"language_id=1; expires=Sat, 17-Sep-2016 11:45:15 GMT; path=/; httponly">>},
{<<"Set-Cookie">>,
<<"width=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; httponly">>},
{<<"Set-Cookie">>,
<<"ci_session_idefix=9a340f6f35361a6099922a76a18ef95b101c904b; expires=Fri,"...>>},
{<<"Expires">>,<<"Mon, 26 Jul 1990 05:00:00 GMT">>},
{<<"Last-Modified">>,<<"Fri, 18 Sep 2015 11:45:15 GMT">>},
{<<"Cache-Control">>,<<"post-check=0, pre-check=0">>},
{<<"Pragma">>,<<"no-cache">>},
{<<"Content-Length">>,<<"127">>},
{<<"Connection">>,<<"close">>},
{<<"Content-Type">>,<<"application/json">>}],
#Ref<0.0.1.138>}

Sorry for my English :)

shurkus commented Sep 18, 2015

I had a similar problem.
Sometimes sertifikate had ^L^K instead of ^S^N.

The solution was to change in public_key
(/usr/local/lib/erlang/lib/public_key/asn1/OTP-PUB-KEY.erl)
lines
%%================================
%% CommonName
%%================================
'enc_CommonName'(Val) ->
'enc_CommonName'(Val, [<<19>>]).

to

%%================================
%% CommonName
%%================================
'enc_CommonName'(Val) ->
'enc_CommonName'(Val, [<<12>>]).
and rebuild (recompile).

Method = post,URL = <<"https://rest-api.pay.nl/v5/Transaction/start/json">>,Headers = [],Payload = <<>>,Options = [],{ok, StatusCode, RespHeaders, ClientRef} = hackney:request(Method, URL,Headers, Payload,Options).

=INFO REPORT==== 18-Sep-2015::14:45:15 ===
SSL WARNING: Ignoring a CA cert as it could not be correctly decoded.

=INFO REPORT==== 18-Sep-2015::14:45:15 ===
SSL WARNING: Ignoring a CA cert as it could not be correctly decoded.

{ok,400,
[{<<"Date">>,<<"Fri, 18 Sep 2015 11:45:15 GMT">>},
{<<"Server">>,<<"Apache">>},
{<<"Set-Cookie">>,
<<"device_id=1213e89a5844c7d4481ee7c51a55291f452b8880; expires=Sat, 17-Sep-2016 11:45:1"...>>},
{<<"Set-Cookie">>,
<<"language_id=1; expires=Sat, 17-Sep-2016 11:45:15 GMT; path=/; httponly">>},
{<<"Set-Cookie">>,
<<"width=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; httponly">>},
{<<"Set-Cookie">>,
<<"ci_session_idefix=9a340f6f35361a6099922a76a18ef95b101c904b; expires=Fri,"...>>},
{<<"Expires">>,<<"Mon, 26 Jul 1990 05:00:00 GMT">>},
{<<"Last-Modified">>,<<"Fri, 18 Sep 2015 11:45:15 GMT">>},
{<<"Cache-Control">>,<<"post-check=0, pre-check=0">>},
{<<"Pragma">>,<<"no-cache">>},
{<<"Content-Length">>,<<"127">>},
{<<"Connection">>,<<"close">>},
{<<"Content-Type">>,<<"application/json">>}],
#Ref<0.0.1.138>}

Sorry for my English :)

@jamilabreu jamilabreu referenced this issue in edgurgel/httpoison Nov 10, 2015

Open

Fatal error: bad certificate #93

@terinjokes

This comment has been minimized.

Show comment
Hide comment
@terinjokes

terinjokes Dec 30, 2015

Contributor

@benoitc I'm also running into the aforementioned issue, this time with the cert from https://registry.npmjs.com

Contributor

terinjokes commented Dec 30, 2015

@benoitc I'm also running into the aforementioned issue, this time with the cert from https://registry.npmjs.com

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Jan 24, 2016

Owner

@terinjokes afaik one of the root certificate is expired which cause the issue. For no there is no hope in the current released version of Erlang. Hopefully it will be solved with the latest patch in Erlang that landed on the ml.

Owner

benoitc commented Jan 24, 2016

@terinjokes afaik one of the root certificate is expired which cause the issue. For no there is no hope in the current released version of Erlang. Hopefully it will be solved with the latest patch in Erlang that landed on the ml.

@suprafly

This comment has been minimized.

Show comment
Hide comment
@suprafly

suprafly Apr 15, 2016

I am running into this issue, albeit through Elixir's HTTPoison library.
For me, I am attempting to POST to an SSL and providing the :insecure option. The issue is intermittent.
Erlang/OTP 18

Error: ** (HTTPoison.Error) {:tls_alert, 'bad certificate'}

suprafly commented Apr 15, 2016

I am running into this issue, albeit through Elixir's HTTPoison library.
For me, I am attempting to POST to an SSL and providing the :insecure option. The issue is intermittent.
Erlang/OTP 18

Error: ** (HTTPoison.Error) {:tls_alert, 'bad certificate'}

@uri

This comment has been minimized.

Show comment
Hide comment
@uri

uri Jun 27, 2016

Is there a anyway to fix this besides passing the insecure option? Can I update the certs that erlang provides or something? I'm on 18.3

uri commented Jun 27, 2016

Is there a anyway to fix this besides passing the insecure option? Can I update the certs that erlang provides or something? I'm on 18.3

@jaimeiniesta

This comment has been minimized.

Show comment
Hide comment
@jaimeiniesta

jaimeiniesta Aug 4, 2016

I'm having the same issue, here's another example, the server cert should be valid:

erl -pa ebin -pa deps/*/ebin
Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Eshell V7.3  (abort with ^G)
1> hackney:start().
ok
2> Method = get,
2> URL = <<"https://qoolife.com">>,
2> Headers = [],
2> Payload = <<>>,
2> Options = [],
2> {ok, StatusCode, RespHeaders, ClientRef} = hackney:request(Method, URL, Headers, Payload, Options).

=ERROR REPORT==== 4-Aug-2016::19:29:33 ===
SSL: certify: ssl_handshake.erl:1476:Fatal error: bad certificate
** exception error: no match of right hand side value {error,{tls_alert,"bad certificate"}}

I'm having the same issue, here's another example, the server cert should be valid:

erl -pa ebin -pa deps/*/ebin
Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Eshell V7.3  (abort with ^G)
1> hackney:start().
ok
2> Method = get,
2> URL = <<"https://qoolife.com">>,
2> Headers = [],
2> Payload = <<>>,
2> Options = [],
2> {ok, StatusCode, RespHeaders, ClientRef} = hackney:request(Method, URL, Headers, Payload, Options).

=ERROR REPORT==== 4-Aug-2016::19:29:33 ===
SSL: certify: ssl_handshake.erl:1476:Fatal error: bad certificate
** exception error: no match of right hand side value {error,{tls_alert,"bad certificate"}}
@aeons

This comment has been minimized.

Show comment
Hide comment
@aeons

aeons Sep 12, 2016

Any update on this?

I am running into this as well.

iex(1)> :hackney.start()
:ok
iex(2)> :hackney.request(:get, "https://oauth.withings.com", [], "", [])
{:error, {:tls_alert, 'bad certificate'}}
[error] SSL: :certify: ssl_handshake.erl:1509:Fatal error: bad certificate
iex(3)> :hackney.request(:get, "https://oauth.withings.com", [], "", [insecure: true])
{:ok, 302,
 [{"Server", "nginx"}, {"Date", "Mon, 12 Sep 2016 07:52:27 GMT"},
  {"Content-Type", "text/html"}, {"Content-Length", "0"},
  {"Connection", "keep-alive"}, {"X-Powered-By", "Banana Pi"},
  {"Set-Cookie",
   "lang_select_language=en; expires=Wed, 12-Sep-2018 07:52:27 GMT; path=/"},
  {"Location", "http://oauth.withings.com/en/account/dashboard"},
  {"Vary", "Accept-Encoding"}], #Reference<0.0.4.182>}

aeons commented Sep 12, 2016

Any update on this?

I am running into this as well.

iex(1)> :hackney.start()
:ok
iex(2)> :hackney.request(:get, "https://oauth.withings.com", [], "", [])
{:error, {:tls_alert, 'bad certificate'}}
[error] SSL: :certify: ssl_handshake.erl:1509:Fatal error: bad certificate
iex(3)> :hackney.request(:get, "https://oauth.withings.com", [], "", [insecure: true])
{:ok, 302,
 [{"Server", "nginx"}, {"Date", "Mon, 12 Sep 2016 07:52:27 GMT"},
  {"Content-Type", "text/html"}, {"Content-Length", "0"},
  {"Connection", "keep-alive"}, {"X-Powered-By", "Banana Pi"},
  {"Set-Cookie",
   "lang_select_language=en; expires=Wed, 12-Sep-2018 07:52:27 GMT; path=/"},
  {"Location", "http://oauth.withings.com/en/account/dashboard"},
  {"Vary", "Accept-Encoding"}], #Reference<0.0.4.182>}
@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Sep 12, 2016

Owner

@aeons your issue is not related. In your case hackney is connecting via https to https://oauth.withings.com and then get in return an absolute url using http instead of https (which should be, the behaviour of withings is quite insecure here). ON which hackney is trying to connect via SSL. I will fix it asap.

@jaimeiniesta well the handshakingof the server is not good:
https://www.ssllabs.com/ssltest/analyze.html?d=qoolife.com

Actually this is the order of one of the certificates that is preventing the handcheck to complete. As far as I know thre is no way to reorder the chain right now. Maybe it changed in Erlang 19. I will check.

Owner

benoitc commented Sep 12, 2016

@aeons your issue is not related. In your case hackney is connecting via https to https://oauth.withings.com and then get in return an absolute url using http instead of https (which should be, the behaviour of withings is quite insecure here). ON which hackney is trying to connect via SSL. I will fix it asap.

@jaimeiniesta well the handshakingof the server is not good:
https://www.ssllabs.com/ssltest/analyze.html?d=qoolife.com

Actually this is the order of one of the certificates that is preventing the handcheck to complete. As far as I know thre is no way to reorder the chain right now. Maybe it changed in Erlang 19. I will check.

@aeons

This comment has been minimized.

Show comment
Hide comment
@aeons

aeons Sep 12, 2016

Oh man, I didn't even notice that. That is horrible.

Thanks for looking into it.

EDIT: Actually, that redirect is only because I shortened the url. With the full url it looks like this:

iex(1)> :hackney.start()
:ok
iex(2)> :hackney.request(:get, "https://oauth.withings.com/account/request_token", [], "", [])
{:error, {:tls_alert, 'bad certificate'}}
[error] SSL: :certify: ssl_handshake.erl:1509:Fatal error: bad certificate
iex(3)> :hackney.request(:get, "https://oauth.withings.com/account/request_token", [], "", [insecure: true])
{:ok, 200,
 [{"Server", "nginx"}, {"Date", "Mon, 12 Sep 2016 11:40:34 GMT"},
  {"Content-Type", "text/html"}, {"Content-Length", "48"},
  {"Connection", "keep-alive"}, {"X-Powered-By", "Banana Pi"},
  {"Set-Cookie",
   "lang_select_language=en; expires=Wed, 12-Sep-2018 11:40:34 GMT; path=/"},
  {"Vary", "Accept-Encoding"}], #Reference<0.0.1.125>}

aeons commented Sep 12, 2016

Oh man, I didn't even notice that. That is horrible.

Thanks for looking into it.

EDIT: Actually, that redirect is only because I shortened the url. With the full url it looks like this:

iex(1)> :hackney.start()
:ok
iex(2)> :hackney.request(:get, "https://oauth.withings.com/account/request_token", [], "", [])
{:error, {:tls_alert, 'bad certificate'}}
[error] SSL: :certify: ssl_handshake.erl:1509:Fatal error: bad certificate
iex(3)> :hackney.request(:get, "https://oauth.withings.com/account/request_token", [], "", [insecure: true])
{:ok, 200,
 [{"Server", "nginx"}, {"Date", "Mon, 12 Sep 2016 11:40:34 GMT"},
  {"Content-Type", "text/html"}, {"Content-Length", "48"},
  {"Connection", "keep-alive"}, {"X-Powered-By", "Banana Pi"},
  {"Set-Cookie",
   "lang_select_language=en; expires=Wed, 12-Sep-2018 11:40:34 GMT; path=/"},
  {"Vary", "Accept-Encoding"}], #Reference<0.0.1.125>}
@aeons

This comment has been minimized.

Show comment
Hide comment
@aeons

aeons Sep 12, 2016

However, https://www.ssllabs.com/ssltest/analyze.html?d=oauth.withings.com , shows Chain issues Incorrect order.

Could that be the cause?

aeons commented Sep 12, 2016

However, https://www.ssllabs.com/ssltest/analyze.html?d=oauth.withings.com , shows Chain issues Incorrect order.

Could that be the cause?

@barttenbrinke

This comment has been minimized.

Show comment
Hide comment
@barttenbrinke

barttenbrinke Jan 31, 2017

Erlang master has fixed some of the above SSL issues. https://qoolife.com and https://rest-api.pay.nl work fine now. oauth.withings still gives me errors, it will probably work when the chain is corrected.

barttenbrinke commented Jan 31, 2017

Erlang master has fixed some of the above SSL issues. https://qoolife.com and https://rest-api.pay.nl work fine now. oauth.withings still gives me errors, it will probably work when the chain is corrected.

@benoitc benoitc added rfc ssl and removed rfc labels May 19, 2017

@benoitc benoitc added this to SSL in Development May 19, 2017

@jimdigriz

This comment has been minimized.

Show comment
Hide comment
@jimdigriz

jimdigriz May 20, 2017

Contributor

MaxCDN have the incorrect chain order problem when you try to pull https://www.maxcdn.com/one/assets/ips.txt (SSL Labs)

Contributor

jimdigriz commented May 20, 2017

MaxCDN have the incorrect chain order problem when you try to pull https://www.maxcdn.com/one/assets/ips.txt (SSL Labs)

@benoitc benoitc moved this from SSL to Metrics in Development May 21, 2017

@benoitc benoitc moved this from Metrics to SSL in Development May 21, 2017

@mustafaturan

This comment has been minimized.

Show comment
Hide comment
@mustafaturan

mustafaturan Jul 17, 2017

Not recommended but:
You can disable it with the disable option: {server_name_indication, disable}

Not recommended but:
You can disable it with the disable option: {server_name_indication, disable}

@lpil

This comment has been minimized.

Show comment
Hide comment
@lpil

lpil Apr 3, 2018

Hello! We're having a similar problem on Erlang 20 with Ubuntu 16.04, current OSX and current Alpine Linux.

 hackney:request(get, <<"https://airbrake.io/">>, [], <<"">>, []).
% =>  {error, {tls_alert, "bad certificate"}}

The cert seems to be OK https://www.ssllabs.com/ssltest/analyze.html?d=airbrake.io

Is there a know problem or fix here? We're not sure how to proceed.

Thanks,
Louis

lpil commented Apr 3, 2018

Hello! We're having a similar problem on Erlang 20 with Ubuntu 16.04, current OSX and current Alpine Linux.

 hackney:request(get, <<"https://airbrake.io/">>, [], <<"">>, []).
% =>  {error, {tls_alert, "bad certificate"}}

The cert seems to be OK https://www.ssllabs.com/ssltest/analyze.html?d=airbrake.io

Is there a know problem or fix here? We're not sure how to proceed.

Thanks,
Louis

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Apr 3, 2018

Owner

@ipil as reported above the chain order is not OK which is an issue with the Erlang implementation.
We could probably trick the order as it has been reported on the ml. This code posted on the ml could help: https://github.com/Vagabond/erl_crl_example/blob/master/src/client.erl

However it would be better if airbrake could fix their certificate in the mean time , as fixing the order is somehow hackish. Using {insecure, true} which stops the SSL validation is another option.

Owner

benoitc commented Apr 3, 2018

@ipil as reported above the chain order is not OK which is an issue with the Erlang implementation.
We could probably trick the order as it has been reported on the ml. This code posted on the ml could help: https://github.com/Vagabond/erl_crl_example/blob/master/src/client.erl

However it would be better if airbrake could fix their certificate in the mean time , as fixing the order is somehow hackish. Using {insecure, true} which stops the SSL validation is another option.

@benoitc

This comment has been minimized.

Show comment
Hide comment
@benoitc

benoitc Apr 3, 2018

Owner

Sorry as reported in #490 . I'm closing this issue in favour of the last one since it's the same.

Owner

benoitc commented Apr 3, 2018

Sorry as reported in #490 . I'm closing this issue in favour of the last one since it's the same.

@benoitc benoitc closed this Apr 3, 2018

@lpil

This comment has been minimized.

Show comment
Hide comment
@lpil

lpil Apr 3, 2018

Ah! I misread the report. Thank you for your help and your super speedy reply :)

lpil commented Apr 3, 2018

Ah! I misread the report. Thank you for your help and your super speedy reply :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment