Possible problem with TLS and SMTP causing timeout in curl 8.12

[MonkeybreadSoftware]

I did this

So I connect with CURL 8.12 and I see this output:

curl -k --ssl -v "smtp://sslout.xxx.eu:25"

...

• Connected to sslout.xxx.eu (xxx) port 25
• server response timeout
• closing connection #0
  curl: (28) server response timeout

So we get a timeout as EHLO is not sent.

Doing the same with

curl -k -v "smtps://sslout.df.eu:465"

Shows this:

...

• Connected to sslout.xxx.eu (xxx) port 465
  < 220 smtprelay06.xxx.xx ESMTP dfex
  EHLO MyMac
  < 250-smtprelay06.xxx.xx Hello MyMacM3 [xxx]

And using the first variant with command line tool using older curl version, I see

• SSL certificate verify ok.
  EHLO MyMac
  < 250-smtprelay08.xxx.xx Hello MyMacM3 [xxx]

So for some reason I don't see yet, we don't get to the state when to send EHLO.

Has anyone seen this?

I expected the following

SMTP with STARTTLS should work.

curl/libcurl version

curl 8.12.0

operating system

macOS. Also on happens on Windows.

[MonkeybreadSoftware]

the smtp.c file didn't change, but probably something with connection filters?

[MonkeybreadSoftware]

So in smtp_perform_upgrade_tls, the function calls Curl_conn_connect and expects ssldone to be set to true eventually. This doesn't happen with the current version as I see.

But in smtp_multi_statemach, I noticed that Curl_conn_connect sets the ssldone variable there.

I can add a check there to call smtp_perform_ehlo as a workaround to fix the issue for me.

    if(smtpc->state == SMTP_UPGRADETLS && smtpc->ssldone) {
      smtp_to_smtps(conn);
      result = smtp_perform_ehlo(data);
    }

Then it seems to work here. But I would prefer if the underlaying problem could be fixed.

[icing]

@MonkeybreadSoftware I cannot connect to the url you posted. When I use my own mail server, everything works fine. Can you produce a log with -vvvv, so I can see what is going on? Thanks.

[MonkeybreadSoftware]

it's sslout.df.eu but I thought I should xxx that out.

I build it again without my patch:

curl 8.12.0 (aarch64-apple-darwin21.6.0) libcurl/8.12.0 OpenSSL/3.3.2 (SecureTransport) zlib/1.2.13 zstd/1.5.2 AppleIDN libssh2/1.11.1
Release-Date: 2025-02-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS HSTS HTTPS-proxy IDN IPv6 Largefile libz MultiSSL NTLM SSL threadsafe TLS-SRP UnixSockets zstd

So I run it:

cs@MyMacM3 ~ % /tmp/macARM/src/.libs/curl  -k --ssl -v "smtp://sslout.xxx.eu:25"
Warning: --ssl is an insecure option, consider --ssl-reqd instead
* Host sslout.xxx.eu:25 was resolved.
* IPv6: (none)
* IPv4: 64.190.63.222
*   Trying 64.190.63.222:25...
* connect to 64.190.63.222 port 25 from 192.168.188.75 port 53398 failed: Connection refused
* Failed to connect to sslout.xxx.eu port 25 after 71 ms: Could not connect to server
* closing connection #0
curl: (7) Failed to connect to sslout.xxx.eu port 25 after 71 ms: Could not connect to server
cs@MyMacM3 ~ % /tmp/macARM/src/.libs/curl  -k --ssl -v "smtp://sslout.df.eu:25" 
Warning: --ssl is an insecure option, consider --ssl-reqd instead
* Host sslout.df.eu:25 was resolved.
* IPv6: (none)
* IPv4: 134.119.18.24
*   Trying 134.119.18.24:25...
* Connected to sslout.df.eu (134.119.18.24) port 25
< 220 smtprelay08.ispgateway.de ESMTP dfex
> EHLO MyMacM3
< 250-smtprelay08.ispgateway.de Hello MyMacM3 [185.231.252.69]
< 250-SIZE 104857600
< 250-LIMITS MAILMAX=1000 RCPTMAX=50000
< 250-8BITMIME
< 250-PIPELINING
< 250-PIPECONNECT
< 250-AUTH PLAIN LOGIN
< 250-STARTTLS
< 250 HELP
> STARTTLS
< 220 TLS go ahead
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 / secp256r1 / rsaEncryption
* Server certificate:
*  subject: CN=sslout.df.eu
*  start date: Jan 29 09:43:13 2024 GMT
*  expire date: Mar  1 09:43:13 2025 GMT
*  issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
*  SSL certificate verify result: self-signed certificate in certificate chain (19), continuing anyway.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* Connected to sslout.df.eu (134.119.18.24) port 25
* server response timeout
* closing connection #0
curl: (28) server response timeout
cs@MyMacM3 ~ % 

[icing]

Thanks, reproducing here locally. Analysing...

[icing]

Ok, found it. Proposing #16206 as fix. Could you verify? Thanks.

[MonkeybreadSoftware]

Would it be useful to make a few tests, which try to connect to a few smtp servers in different combinations and try to see if we come to the authentication part?
If no internet is available, the test should not return an error.

[bagder]

Adding tests for this would be awesome of course. However, I don't think our test suite should use any remotely accessed servers, only things we run locally. Because that's how our test suite runs (and a enough number of users run the test suite without access to the internet), because we don't run those other servers and don't want our test suite to become a DDOS attack on them and because using "random" remote servers tend to be fragile and flaky.