Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG / possible BUG] test suite stuck at the very same test in 3.0.4 #1824

Open
anthraxx opened this issue Jan 13, 2021 · 8 comments
Open

[BUG / possible BUG] test suite stuck at the very same test in 3.0.4 #1824

anthraxx opened this issue Jan 13, 2021 · 8 comments

Comments

@anthraxx
Copy link

**Please don't remove this template. We would like to reproduce the bug and need concise information. **

Command line / docker command to reproduce

TESTSSL_INSTALL_DIR="$(pwd)" prove -v

Expected behavior
test suite should pass

Your system (please complete the following information):

  • OS: Arch Linux
  • Platform: Linux 5.9.14-arch1-1 x86_64
  • Version: 3.0.4
  • OpenSSL: 1.1.1.i

Additional context
Test suite is stuck forever at the very same place which has passed in former versions. I tried getting it pass on various different days and running for multipel hours, always the same. I would like to be able to pass test suites as a process of distro package release:

....
t/20_baseline_ipv4_http.t ..........

Baseline unit test IPv4 via sockets --> google.com ...
ok 1 -

Baseline unit test IPv4 via OpenSSL --> google.com ...
ok 2 -
1..2
ok
t/23_client_simulation.t ...........

Client simulations unit test via sockets --> google.com ...
ok 1 -

Client simulations unit test via OpenSSL --> google.com ...
ok 2 -

STARTTLS: Client simulations unit test via sockets --> smtp-relay.gmail.com:587 ...
ok 3 -

STARTTLS: Client simulations unit test via OpenSSL --> smtp-relay.gmail.com:587 ...
ok 4 -
1..4
ok

At this point the test suite will be stuck for all eternity.
@drwetter
Copy link
Owner

drwetter commented Jan 13, 2021 •
edited

Hmm, works for me. What's your platform?

t/23_client_simulation.t ........... 

Client simulations unit test via sockets --> google.com ...
ok 1 - 

Client simulations unit test via OpenSSL --> google.com ...
ok 2 - 

STARTTLS: Client simulations unit test via sockets --> smtp-relay.gmail.com:587 ...
ok 3 - 

STARTTLS: Client simulations unit test via OpenSSL --> smtp-relay.gmail.com:587 ...
ok 4 - 
1..4
ok
t/25_baseline_starttls.t ........... 

STARTTLS SMTP unit test via sockets --> smtp-relay.gmail.com:587 ...
ok 1 - 

STARTTLS SMTP unit tests via OpenSSL --> smtp-relay.gmail.com:587 ...
ok 2 - 

STARTTLS POP3 unit tests via sockets --> pop.gmx.net:110 ...
ok 3 - 

[..]

However due to your complaint I realized a problem, see #1825

@drwetter
Copy link
Owner

What does ./testssl.sh -t smtp --protocols --standard --pfs --server-preference --headers --vulnerable --each-cipher -q --ip=one --color 0 smtp-relay.gmail.com:587 return?

@drwetter
Copy link
Owner

https://travis-ci.org/github/drwetter/testssl.sh/builds/754361070

@anthraxx
Copy link
Author

Hey @drwetter thanks for your fast answer.
We have tried running this on multiple different machines leading to the same hang.

The output of your command (with replaced --pfs with --fs as i assumed thats what you meant?

./testssl.sh -t smtp --protocols --standard --fs --server-preference --headers --vulnerable --each-cipher -q --ip=one --color 0 smtp-relay.gmail.com:587

 Start 2021-01-14 02:38:11        -->> 173.194.69.28:587 (smtp-relay.gmail.com) <<--

 Further IP addresses:   2a00:1450:4013:c04::1c
 A record via:           supplied IP "173.194.69.28"
 rDNS (173.194.69.28):   ef-in-f28.1e100.net.
 Service set:            STARTTLS via SMTP

 Testing protocols via sockets

 SSLv2      not offered (OK)
 SSLv3      not offered (OK)
 TLS 1      offered (deprecated)
 TLS 1.1    offered (deprecated)
 TLS 1.2    offered (OK)
 TLS 1.3    offered (OK): final

 Testing cipher categories

 NULL ciphers (no encryption)                      not offered (OK)
 Anonymous NULL Ciphers (no authentication)        not offered (OK)
 Export ciphers (w/o ADH+NULL)                     not offered (OK)
 LOW: 64 Bit + DES, RC[2,4], MD5 (w/o export)      not offered (OK)
 Triple DES Ciphers / IDEA                         offered
 Obsoleted CBC ciphers (AES, ARIA etc.)            offered
 Strong encryption (AEAD ciphers) with no FS       offered (OK)
 Forward Secrecy strong encryption (AEAD ciphers)  offered (OK)


 Testing server's cipher preferences

 Has server cipher order?     yes (OK) -- only for < TLS 1.3
 Negotiated protocol          TLSv1.3
 Negotiated cipher            TLS_AES_256_GCM_SHA384, 253 bit ECDH (X25519)
 Cipher per protocol

Hexcode  Cipher Suite Name (OpenSSL)       KeyExch.   Encryption  Bits     Cipher Suite Name (IANA/RFC)
-----------------------------------------------------------------------------------------------------------------------------
SSLv2
 -
SSLv3
 -
TLSv1 (server order)
 xc009   ECDHE-ECDSA-AES128-SHA            ECDH 256   AES         128      TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
 xc00a   ECDHE-ECDSA-AES256-SHA            ECDH 256   AES         256      TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
 xc013   ECDHE-RSA-AES128-SHA              ECDH 256   AES         128      TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
 xc014   ECDHE-RSA-AES256-SHA              ECDH 256   AES         256      TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
 x2f     AES128-SHA                        RSA        AES         128      TLS_RSA_WITH_AES_128_CBC_SHA
 x35     AES256-SHA                        RSA        AES         256      TLS_RSA_WITH_AES_256_CBC_SHA
 x0a     DES-CBC3-SHA                      RSA        3DES        168      TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLSv1.1 (server order)
 xc009   ECDHE-ECDSA-AES128-SHA            ECDH 256   AES         128      TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
 xc00a   ECDHE-ECDSA-AES256-SHA            ECDH 256   AES         256      TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
 xc013   ECDHE-RSA-AES128-SHA              ECDH 256   AES         128      TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
 xc014   ECDHE-RSA-AES256-SHA              ECDH 256   AES         256      TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
 x2f     AES128-SHA                        RSA        AES         128      TLS_RSA_WITH_AES_128_CBC_SHA
 x35     AES256-SHA                        RSA        AES         256      TLS_RSA_WITH_AES_256_CBC_SHA
 x0a     DES-CBC3-SHA                      RSA        3DES        168      TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLSv1.2 (server order)
 xcca9   ECDHE-ECDSA-CHACHA20-POLY1305     ECDH 253   ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
 xc02b   ECDHE-ECDSA-AES128-GCM-SHA256     ECDH 253   AESGCM      128      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 xc02c   ECDHE-ECDSA-AES256-GCM-SHA384     ECDH 253   AESGCM      256      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
 xc009   ECDHE-ECDSA-AES128-SHA            ECDH 253   AES         128      TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
 xc00a   ECDHE-ECDSA-AES256-SHA            ECDH 253   AES         256      TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
 xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH 253   ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
 xc02f   ECDHE-RSA-AES128-GCM-SHA256       ECDH 253   AESGCM      128      TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 xc030   ECDHE-RSA-AES256-GCM-SHA384       ECDH 253   AESGCM      256      TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
 xc013   ECDHE-RSA-AES128-SHA              ECDH 253   AES         128      TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
 xc014   ECDHE-RSA-AES256-SHA              ECDH 253   AES         256      TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
 x9c     AES128-GCM-SHA256                 RSA        AESGCM      128      TLS_RSA_WITH_AES_128_GCM_SHA256
 x9d     AES256-GCM-SHA384                 RSA        AESGCM      256      TLS_RSA_WITH_AES_256_GCM_SHA384
 x2f     AES128-SHA                        RSA        AES         128      TLS_RSA_WITH_AES_128_CBC_SHA
 x35     AES256-SHA                        RSA        AES         256      TLS_RSA_WITH_AES_256_CBC_SHA
 x0a     DES-CBC3-SHA                      RSA        3DES        168      TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLSv1.3 (no server order, thus listed by strength)
 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256      TLS_AES_256_GCM_SHA384
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256      TLS_CHACHA20_POLY1305_SHA256
 x1301   TLS_AES_128_GCM_SHA256            ECDH 253   AESGCM      128      TLS_AES_128_GCM_SHA256


 Testing robust forward secrecy (FS) -- omitting Null Authentication/Encryption, 3DES, RC4

 FS is offered (OK)           TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 ECDHE-RSA-AES256-GCM-SHA384
                              ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA ECDHE-ECDSA-AES256-SHA
                              ECDHE-ECDSA-CHACHA20-POLY1305 ECDHE-RSA-CHACHA20-POLY1305 TLS_AES_128_GCM_SHA256
                              ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA
                              ECDHE-ECDSA-AES128-SHA
 Elliptic curves offered:     prime256v1 X25519


 Testing vulnerabilities

 Heartbleed (CVE-2014-0160)                not vulnerable (OK), no heartbeat extension
 CCS (CVE-2014-0224)                       not vulnerable (OK)
 ROBOT                                     not vulnerable (OK)
 Secure Renegotiation (RFC 5746)           supported (OK)
 Secure Client-Initiated Renegotiation     not vulnerable (OK)
 CRIME, TLS (CVE-2012-4929)                not vulnerable (OK) (not using HTTP anyway)
 POODLE, SSL (CVE-2014-3566)               not vulnerable (OK), no SSLv3 support
 TLS_FALLBACK_SCSV (RFC 7507)              Downgrade attack prevention supported (OK)
 SWEET32 (CVE-2016-2183, CVE-2016-6329)    VULNERABLE, uses 64 bit block ciphers
 FREAK (CVE-2015-0204)                     not vulnerable (OK)
 DROWN (CVE-2016-0800, CVE-2016-0703)      not vulnerable on this host and port (OK)
                                           make sure you don't use this certificate elsewhere with SSLv2 enabled services
                                           https://censys.io/ipv4?q=5617D2864E8359DF0F260D03554E5BEB76F64F00008196B81F2329D4E2E7333C could help you to find out
 LOGJAM (CVE-2015-4000), experimental      not vulnerable (OK): no DH EXPORT ciphers, no DH key detected with <= TLS 1.2
 BEAST (CVE-2011-3389)                     TLS1: ECDHE-ECDSA-AES128-SHA ECDHE-ECDSA-AES256-SHA
                                                 ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA AES128-SHA
                                                 AES256-SHA DES-CBC3-SHA
                                           VULNERABLE -- but also supports higher protocols  TLSv1.1 TLSv1.2 (likely mitigated)
 LUCKY13 (CVE-2013-0169), experimental     potentially VULNERABLE, uses cipher block chaining (CBC) ciphers with TLS. Check patches
 Winshock (CVE-2014-6321), experimental    not vulnerable (OK)
 RC4 (CVE-2013-2566, CVE-2015-2808)        no RC4 ciphers detected (OK)
 STARTTLS injection (experimental)         not vulnerable (OK)


 Testing 370 ciphers via OpenSSL plus sockets against the server, ordered by encryption strength

Hexcode  Cipher Suite Name (OpenSSL)       KeyExch.   Encryption  Bits     Cipher Suite Name (IANA/RFC)
-----------------------------------------------------------------------------------------------------------------------------
 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256      TLS_AES_256_GCM_SHA384
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256      TLS_CHACHA20_POLY1305_SHA256
 xc030   ECDHE-RSA-AES256-GCM-SHA384       ECDH 256   AESGCM      256      TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
 xc02c   ECDHE-ECDSA-AES256-GCM-SHA384     ECDH 256   AESGCM      256      TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
 xc014   ECDHE-RSA-AES256-SHA              ECDH 256   AES         256      TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
 xc00a   ECDHE-ECDSA-AES256-SHA            ECDH 256   AES         256      TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
 xcca9   ECDHE-ECDSA-CHACHA20-POLY1305     ECDH 253   ChaCha20    256      TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
 xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH 253   ChaCha20    256      TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
 x9d     AES256-GCM-SHA384                 RSA        AESGCM      256      TLS_RSA_WITH_AES_256_GCM_SHA384
 x35     AES256-SHA                        RSA        AES         256      TLS_RSA_WITH_AES_256_CBC_SHA
 x1301   TLS_AES_128_GCM_SHA256            ECDH 253   AESGCM      128      TLS_AES_128_GCM_SHA256
 xc02f   ECDHE-RSA-AES128-GCM-SHA256       ECDH 256   AESGCM      128      TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 xc02b   ECDHE-ECDSA-AES128-GCM-SHA256     ECDH 256   AESGCM      128      TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 xc013   ECDHE-RSA-AES128-SHA              ECDH 256   AES         128      TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
 xc009   ECDHE-ECDSA-AES128-SHA            ECDH 256   AES         128      TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
 x9c     AES128-GCM-SHA256                 RSA        AESGCM      128      TLS_RSA_WITH_AES_128_GCM_SHA256
 x2f     AES128-SHA                        RSA        AES         128      TLS_RSA_WITH_AES_128_CBC_SHA
 x0a     DES-CBC3-SHA                      RSA        3DES        168      TLS_RSA_WITH_3DES_EDE_CBC_SHA


 Done 2021-01-14 02:40:00 [ 114s] -->> 173.194.69.28:587 (smtp-relay.gmail.com) <<--

@anthraxx
Copy link
Author

what i find interesting is that it seems to be stuck right after printing success of the last test from 23_client_simulation.t but before any other start message from the following test 51_badssl.com.t:

ok 4 -
1..4
ok

@anthraxx
Copy link
Author

looking at the process tree we were inspecting the wrong place. This is from one of the hangs:

└─ perl /usr/bin/core_perl/prove -v
   └─ perl t/25_baseline_starttls.t
      └─ bash ./testssl.sh -q --ip=one --color 0 -t xmpp jabber.org:5222

digging further into it, the bundled openssl.Linux.x86_64 seems to be to blame as it seems to segfault:

./testssl.sh -q --ip=one --color 0 -t xmpp jabber.org:5222

 Start 2021-01-14 03:11:01        -->> 208.68.163.218:5222 (jabber.org) <<--

 A record via:           supplied IP "208.68.163.218"
 rDNS (208.68.163.218):  xmpp.org.
./testssl.sh: line 18364: 863659 Segmentation fault      (core dumped) $OPENSSL s_client $(s_client_options "$STARTTLS_OPTIMAL_PROTO $BUGS -connect "$NODEIP:$PORT" $PROXY -msg $STARTTLS $SNI") < /dev/null > $TMPFILE 2>> $ERRFILE
./testssl.sh: line 18364: 863676 Segmentation fault      (core dumped) $OPENSSL s_client $(s_client_options "$STARTTLS_OPTIMAL_PROTO $BUGS -connect "$NODEIP:$PORT" $PROXY -msg $STARTTLS $SNI") < /dev/null > $TMPFILE 2>> $ERRFILE
./testssl.sh: line 18364: 863689 Segmentation fault      (core dumped) $OPENSSL s_client $(s_client_options "$STARTTLS_OPTIMAL_PROTO $BUGS -connect "$NODEIP:$PORT" $PROXY -msg $STARTTLS $SNI") < /dev/null > $TMPFILE 2>> $ERRFILE
./testssl.sh: line 18364: 863699 Segmentation fault      (core dumped) $OPENSSL s_client $(s_client_options "$STARTTLS_OPTIMAL_PROTO $BUGS -connect "$NODEIP:$PORT" $PROXY -msg $STARTTLS $SNI") < /dev/null > $TMPFILE 2>> $ERRFILE
./testssl.sh: line 18364: 863712 Segmentation fault      (core dumped) $OPENSSL s_client $(s_client_options "$STARTTLS_OPTIMAL_PROTO $BUGS -connect "$NODEIP:$PORT" $PROXY -msg $STARTTLS $SNI") < /dev/null > $TMPFILE 2>> $ERRFILE
 Your OpenSSL cannot connect to jabber.org:5222
 The results might look ok but they could be nonsense. Really proceed ? ("yes" to continue) -->

I guess the yes promt is to blame for the hang here but the Segmentation fault is worrying:

       Message: Process 863712 (openssl.Linux.x) of user 1000 dumped core.

                Stack trace of thread 863712:
                #0  0x00006975da246e84 __nss_readline (/usr/lib/libc-2.32.so + 0x124e84)
                #1  0x00006975da2ef4cd n/a (/usr/lib/libnss_files-2.32.so + 0x44cd)
                #2  0x00006975da2f06f4 _nss_files_gethostbyname4_r (/usr/lib/libnss_files-2.32.so + 0x56f4)
                #3  0x0000000000654e46 n/a (testssl.sh/bin/openssl.Linux.x86_64 + 0x254e46)
                #4  0x000000000065861a n/a (testssl.sh/bin/openssl.Linux.x86_64 + 0x25861a)
                #5  0x000000000043ef72 n/a (testssl.sh/bin/openssl.Linux.x86_64 + 0x3ef72)

If i call ./testssl.sh with --openssl /usr/bin/openssl then i do not get any segmentation faults. Something seems off with that binary?

@drwetter
Copy link
Owner

Thanks for the input.

The openssl binaries were compiled statically under old platforms to achieve as much compatibility as possible. The error you see is probably bc the world moved on since then and __nss_readline in the glibc changed :-( -- see https://fossies.org/linux/glibc/nss/Versions . What is strange that one one my machines I am using also glibc 2.32. @ic0ns reported similar segfault @ #1275 . Don't know whether there's anything special about Arch Linux?

Can you try

systemctl stop systemd-resolved
systemctl disable systemd-resolved

?

At some certain point we need to decide whether we're better off with a recent binary from the system or provide a different one (see #1589, #1275 and probably more).

While the segfault maybe painful for you (just curious; why do you use at all prove -v?) it is not causing as much pain for joe average user. But that's certainly coming sooner or later.

@drwetter
Copy link
Owner

If you're into debugging input from gdb would help us better to understand the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@anthraxx @drwetter