Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.11.3.rc1 Broke HTTPS ciphers allowed #1149

Closed
efajardo opened this issue Mar 5, 2020 · 8 comments · Fixed by #1151
Closed

4.11.3.rc1 Broke HTTPS ciphers allowed #1149

efajardo opened this issue Mar 5, 2020 · 8 comments · Fixed by #1151

Comments

@efajardo
Copy link
Contributor

efajardo commented Mar 5, 2020

It looks like 4.11.3.rc1 broke the HTTPS ciphers negotiation. This is what I see in the xrootd logs:

200305 18:39:54 586 ?:211@ldas-nat.ligo.caltech.edu XrdPoll: FD 211 attached to poller 0; num=1
200305 18:39:54 586 ?:211@ldas-nat.ligo.caltech.edu sysXrdHttp:  Process. lp:0x7f6eb80012b8 reqstate: 0
200305 18:39:54 586 ?:211@ldas-nat.ligo.caltech.edu sysXrdHttp:  Setting host: [::ffff:131.215.113.203]
200305 18:39:54 586 ?:211@ldas-nat.ligo.caltech.edu sysXrdHttp:  Entering SSL_accept...
200305 18:39:54 586 ?:211@ldas-nat.ligo.caltech.edu sysXrdHttp:  SSL_accept returned :-1
140114551641856:error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher:s3_srvr.c:1435:
200305 18:39:54 586 sysXrdHttp:  Cleanup
200305 18:39:54 586 sysXrdHttp:  Reset
200305 18:39:54 586 sysXrdHttp:  XrdHttpReq request ended.
200305 18:39:54 586 ?:211@ldas-nat.ligo.caltech.edu XrdPoll: FD 211 detached from poller 0; num=

@djw8605 points me that this might be the commit culprit

I found online this helpfull script to test against a different ciphers.

This is a test against Xrootd 4.11.2:

./test_ciphers osg-chicago-stashcache.nrp.internet2.edu:8444
Obtaining cipher list from OpenSSL 1.0.1e-fips 11 Feb 2013.
Testing ECDHE-RSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES256-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES256-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-RSA-CAMELLIA256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-CAMELLIA256-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ADH-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ADH-AES256-SHA256...NO (sslv3 alert handshake failure)
Testing ADH-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ADH-CAMELLIA256-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing AES256-GCM-SHA384...YES
Testing AES256-SHA256...YES
Testing AES256-SHA...YES
Testing CAMELLIA256-SHA...YES
Testing PSK-AES256-CBC-SHA...NO (no ciphers available)
Testing ECDHE-RSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-RSA-SEED-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-SEED-SHA...NO (sslv3 alert handshake failure)
Testing DHE-RSA-CAMELLIA128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-CAMELLIA128-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ADH-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ADH-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ADH-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ADH-SEED-SHA...NO (sslv3 alert handshake failure)
Testing ADH-CAMELLIA128-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing AES128-GCM-SHA256...YES
Testing AES128-SHA256...YES
Testing AES128-SHA...YES
Testing SEED-SHA...YES
Testing CAMELLIA128-SHA...YES
Testing PSK-AES128-CBC-SHA...NO (no ciphers available)
Testing ECDHE-RSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing EDH-RSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing EDH-DSS-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ADH-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing DES-CBC3-SHA...YES
Testing IDEA-CBC-SHA...YES
Testing DES-CBC3-MD5...NO (no ciphers available)
Testing IDEA-CBC-MD5...NO (no ciphers available)
Testing RC2-CBC-MD5...NO (no ciphers available)
Testing PSK-3DES-EDE-CBC-SHA...NO (no ciphers available)
Testing KRB5-IDEA-CBC-SHA...NO (no ciphers available)
Testing KRB5-DES-CBC3-SHA...NO (no ciphers available)
Testing KRB5-IDEA-CBC-MD5...NO (no ciphers available)
Testing KRB5-DES-CBC3-MD5...NO (no ciphers available)
Testing ECDHE-RSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-RC4-SHA...NO (sslv3 alert handshake failure)
Testing ADH-RC4-MD5...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing RC4-SHA...YES
Testing RC4-MD5...NO (sslv3 alert handshake failure)
Testing RC4-MD5...NO (sslv3 alert handshake failure)
Testing PSK-RC4-SHA...NO (no ciphers available)
Testing KRB5-RC4-SHA...NO (no ciphers available)
Testing KRB5-RC4-MD5...NO (no ciphers available)
Testing ECDHE-RSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-NULL-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing NULL-SHA256...NO (sslv3 alert handshake failure)
Testing NULL-SHA...NO (sslv3 alert handshake failure)
Testing NULL-MD5...NO (sslv3 alert handshake failure)

And this is against a 4.11.3

 ./test_ciphers stashcache.t2.ucsd.edu:8444 
Obtaining cipher list from OpenSSL 1.0.1e-fips 11 Feb 2013.
Testing ECDHE-RSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES256-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES256-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-RSA-CAMELLIA256-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-CAMELLIA256-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ADH-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ADH-AES256-SHA256...NO (sslv3 alert handshake failure)
Testing ADH-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ADH-CAMELLIA256-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES256-SHA384...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES256-SHA...NO (sslv3 alert handshake failure)
Testing AES256-GCM-SHA384...NO (sslv3 alert handshake failure)
Testing AES256-SHA256...NO (sslv3 alert handshake failure)
Testing AES256-SHA...NO (sslv3 alert handshake failure)
Testing CAMELLIA256-SHA...NO (sslv3 alert handshake failure)
Testing PSK-AES256-CBC-SHA...NO (no ciphers available)
Testing ECDHE-RSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDHE-RSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing DHE-RSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-AES128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-RSA-SEED-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-SEED-SHA...NO (sslv3 alert handshake failure)
Testing DHE-RSA-CAMELLIA128-SHA...NO (sslv3 alert handshake failure)
Testing DHE-DSS-CAMELLIA128-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ADH-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ADH-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ADH-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ADH-SEED-SHA...NO (sslv3 alert handshake failure)
Testing ADH-CAMELLIA128-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES128-SHA256...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-AES128-SHA...NO (sslv3 alert handshake failure)
Testing AES128-GCM-SHA256...NO (sslv3 alert handshake failure)
Testing AES128-SHA256...NO (sslv3 alert handshake failure)
Testing AES128-SHA...NO (sslv3 alert handshake failure)
Testing SEED-SHA...NO (sslv3 alert handshake failure)
Testing CAMELLIA128-SHA...NO (sslv3 alert handshake failure)
Testing PSK-AES128-CBC-SHA...NO (no ciphers available)
Testing ECDHE-RSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing EDH-RSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing EDH-DSS-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ADH-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing DES-CBC3-SHA...NO (sslv3 alert handshake failure)
Testing IDEA-CBC-SHA...NO (sslv3 alert handshake failure)
Testing DES-CBC3-MD5...NO (no ciphers available)
Testing IDEA-CBC-MD5...NO (no ciphers available)
Testing RC2-CBC-MD5...NO (no ciphers available)
Testing PSK-3DES-EDE-CBC-SHA...NO (no ciphers available)
Testing KRB5-IDEA-CBC-SHA...NO (no ciphers available)
Testing KRB5-DES-CBC3-SHA...NO (no ciphers available)
Testing KRB5-IDEA-CBC-MD5...NO (no ciphers available)
Testing KRB5-DES-CBC3-MD5...NO (no ciphers available)
Testing ECDHE-RSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-RC4-SHA...NO (sslv3 alert handshake failure)
Testing ADH-RC4-MD5...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-RC4-SHA...NO (sslv3 alert handshake failure)
Testing RC4-SHA...NO (sslv3 alert handshake failure)
Testing RC4-MD5...NO (sslv3 alert handshake failure)
Testing RC4-MD5...NO (sslv3 alert handshake failure)
Testing PSK-RC4-SHA...NO (no ciphers available)
Testing KRB5-RC4-SHA...NO (no ciphers available)
Testing KRB5-RC4-MD5...NO (no ciphers available)
Testing ECDHE-RSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing ECDHE-ECDSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing AECDH-NULL-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-RSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing ECDH-ECDSA-NULL-SHA...NO (sslv3 alert handshake failure)
Testing NULL-SHA256...NO (sslv3 alert handshake failure)
Testing NULL-SHA...NO (sslv3 alert handshake failure)
Testing NULL-MD5...NO (sslv3 alert handshake failure)

Notice the latter one all the ciphers fail. So I believe there seems something wrong with the way the code was written. If I try it manually:

openssl s_client -connect stashcache.t2.ucsd.edu:8444
CONNECTED(00000003)
140164355618632:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:744:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 247 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

I hope this helps to narrow it down

@brianhlin
Copy link

@abh3 you mentioned that the change in the cipher list was made as a request by @bbockelm ? Got a link to the GitHub issue/PR?

@bbockelm
Copy link
Contributor

bbockelm commented Mar 5, 2020

Yes, I had requested this - basically limit the cipher list to things considered secure. I think @ffurano did it with this PR:

https://github.com/xrootd/xrootd/pull/1137/files

I have not had the time to test this.

@jthiltges
Copy link
Contributor

Server needs to include a call to SSL_CTX_set_ecdh_auto() ?

@abh3
Copy link
Member

abh3 commented Mar 5, 2020

Thanks for testing this (where did you get test_ciphers?). First, the undocumented directive to set the cipher list is:
http.cipherfilter colon-separated-list
The code assumes the default of:
ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

I don't see anything in the code that would not set the cipher list via SSL_CTX_set_cipher_list() so it's not apparent what is wrong here. However, the previous version specified
ALL:!LOW:!EXP:!MD5:!MD2

The same as for xroots protocol.

Now, I do know that TLS 1.3 is no longer backward compatible with previous versions of TLS should you specify certain ciphers. Not that we are using TLS 1.3 but may be, see
openssl/openssl#8838

The other issue is if you are using OpenSSL 1.1.1 and give it an unsupported cipher it does absolutely nothing and appears to drop all ciphers with no error indication. The developers claim this is the best comprmise they could reach to support TLS 1.3, see issue
openssl/openssl#6296

Ir would appear that until all of this get straightened out, specifying explicit ciphers is not recommended. Anyway, use the cipherfilter directive to specify the old string and see what happens. You may even want to check which of those ciphers in the new string is causing the problem. In any case, OpenSSL strkes again!

@abh3
Copy link
Member

abh3 commented Mar 5, 2020 via email

@abh3
Copy link
Member

abh3 commented Mar 5, 2020 via email

@efajardo
Copy link
Contributor Author

efajardo commented Mar 6, 2020

@abh3 I linked the script to in the first comment in the issue, but it is also here:

#!/usr/bin/env bash

# OpenSSL requires the port number.
SERVER=$1
DELAY=1
ciphers=$(openssl ciphers 'ALL:eNULL' | sed -e 's/:/ /g')

echo Obtaining cipher list from $(openssl version).

for cipher in ${ciphers[@]}
do
echo -n Testing $cipher...
result=$(echo -n | openssl s_client -cipher "$cipher" -connect $SERVER 2>&1)
if [[ "$result" =~ ":error:" ]] ; then
  error=$(echo -n $result | cut -d':' -f6)
  echo NO \($error\)
else
  if [[ "$result" =~ "Cipher is ${cipher}" || "$result" =~ "Cipher    :" ]] ; then
    echo YES
  else
    echo UNKNOWN RESPONSE
    echo $result
  fi
fi
sleep $DELAY
done

@jthiltges
Copy link
Contributor

I'm not sure of the source for the cipher list, but it matches the current recommendations from Mozilla: https://ssl-config.mozilla.org/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants