Can not send emails after updating from .NET Core 3.1 to .NET 5

[mihail-brinza]

Hello,
I recently updated a mailing microservice from .net core 3.1 to .net 5, which runs in a linux docker image, and I keep getting the exception "An error occurred while attempting to establish an SSL or TLS connection."

First I've tried every solution from your FAQ and it still did not work. When I downgrade the service from .NET 5 to .NET core 3.1 it works fine. So it is not the code, and furthermore I am able to connect to google's smtp server with the .NET 5 version.

Having said that, the only options left for me is this breaking change from microsoft Default TLS cipher suites for .NET on Linux.
The server I'm trying to connect to is using TLS1.2 with the cipher AES256-GCM-SHA384, which is not in the list for .NET 5 for Linux from the link above.

I've tried to add the cipher to openssl.cnf in the docker image but without success.

Also, I've tried to set the CipherSuitePolicy for Kestrel and it still did not work.

This is the Smtp.log with .NET 5

Connected to smtp://smtp.mngt.local:587/?starttls=when-available
S: 220 smtp.company-name.pt ESMTP Haraka/2.8.23 ready
C: EHLO [10.42.20.59]
S: 250-smtp.company-name.pt Hello [10.172.16.38]Haraka is at your service.
S: 250-PIPELINING
S: 250-8BITMIME
S: 250-SMTPUTF8
S: 250-SIZE 0
S: 250 STARTTLS
C: STARTTLS
S: 220 Go ahead.

So I'd like to know where is MailKit getting the available ciphers from? Is it something internal? system defaults?

Thank you very much!

[jstedfast]

MailKit just uses the System.Net.Security.SslStream class without any overrides (other than allowing you to specify the allowed SSL/TLS protocol versions). By default, though, TLS v1.2 is allowed.

As far as the crypto algorithms allowed and changing the values in openssl.cnf, I'm not at all experienced with that so I doubt I'd be of much help.

Interestingly, when I use openssl to connect to smtp.gmail.com, it uses the algorithm you want:

openssl s_client -connect smtp.gmail.com:465
CONNECTED(00000198)
depth=1 C = US, O = Google Trust Services, CN = GTS CA 1O1
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google LLC, CN = smtp.gmail.com
verify return:1
---
Certificate chain
 0 s:C = US, ST = California, L = Mountain View, O = Google LLC, CN = smtp.gmail.com
   i:C = US, O = Google Trust Services, CN = GTS CA 1O1
 1 s:C = US, O = Google Trust Services, CN = GTS CA 1O1
   i:OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEyDCCA7CgAwIBAgIRAOREAfxFihxYBQAAAACFkX0wDQYJKoZIhvcNAQELBQAw
QjELMAkGA1UEBhMCVVMxHjAcBgNVBAoTFUdvb2dsZSBUcnVzdCBTZXJ2aWNlczET
MBEGA1UEAxMKR1RTIENBIDFPMTAeFw0yMTAxMDUxMjExMjRaFw0yMTAzMzAxMjEx
MjNaMGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQH
Ew1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKEwpHb29nbGUgTExDMRcwFQYDVQQDEw5z
bXRwLmdtYWlsLmNvbTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABJmcQYOgDFpO
/f0JiEr/AwwITxLkeNqIkfq5Q9rXZiy/eOnHzPc93SgFIdwoUWU8uII+6azHIJhu
iV2EiZx+YYCjggJcMIICWDAOBgNVHQ8BAf8EBAMCB4AwEwYDVR0lBAwwCgYIKwYB
BQUHAwEwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUIC4tK7Qz3kLVPfu59cAfoPym
3kgwHwYDVR0jBBgwFoAUmNH4bhDrz5vsYJ8YkBug630J/SswaAYIKwYBBQUHAQEE
XDBaMCsGCCsGAQUFBzABhh9odHRwOi8vb2NzcC5wa2kuZ29vZy9ndHMxbzFjb3Jl
MCsGCCsGAQUFBzAChh9odHRwOi8vcGtpLmdvb2cvZ3NyMi9HVFMxTzEuY3J0MBkG
A1UdEQQSMBCCDnNtdHAuZ21haWwuY29tMCEGA1UdIAQaMBgwCAYGZ4EMAQICMAwG
CisGAQQB1nkCBQMwMwYDVR0fBCwwKjAooCagJIYiaHR0cDovL2NybC5wa2kuZ29v
Zy9HVFMxTzFjb3JlLmNybDCCAQQGCisGAQQB1nkCBAIEgfUEgfIA8AB1APZclC/R
dzAiFFQYCDCUVo7jTRMZM7/fDC8gC8xO8WTjAAABdtKsbl8AAAQDAEYwRAIgcMv6
CkpOR3Q97hv7REx2brduub2gYEkstHUNpWiTh74CICcCX0RrvlBcLC4B2QxUVTku
o0ch03kT9qjqTFUqrLMZAHcAXNxDkv7mq0VEsV6a1FbmEDf71fpH3KFzlLJe5vbH
DsoAAAF20qxuggAABAMASDBGAiEA4z3SYto9GHbrUnDWSL0OMwDwdYWCZhPRsiww
jFONPOQCIQCxYAfGRN97PgmYqz0hIih1iQX6QqQbPTQmWbY2SVJznjANBgkqhkiG
9w0BAQsFAAOCAQEAzJm7C9iJad/1qp6C0hhL9lVGTNsXm3TPAvQ0rG6w+k1PI86I
GFdbcLrYGrbyE8exFCaL7MLTk/YrVxdTdh28hGMVuSz6efZy83podPNHSpU5MvXs
QTlE6o1+635d2CJw8UR3gddHcw9T0+teAohA7NU4AWjnr5K9ql0SEWoOkZVoJIUo
+EVeHbv7+tnC3p2eJ6AeuPiVZdmahAUr/PXhApiw8R3iObNk8Cnnad98JyiyPtJ6
ighZkgxfmmITuWq3ktEMXM2wlXzmEaa5fLhTC4kg9ucYcKr8vbVZw6qoZmo/u/+P
of+bp7KxL1RXZMLr6BjHIYRZrxGt1WTvLPVs7g==
-----END CERTIFICATE-----
subject=C = US, ST = California, L = Mountain View, O = Google LLC, CN = smtp.gmail.com

issuer=C = US, O = Google Trust Services, CN = GTS CA 1O1

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2640 bytes and written 396 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 7DAFE17EEC1F576503D3DABD02445E2C5C5926AAA0CEADC1F1966C698C037B5C
    Session-ID-ctx:
    Resumption PSK: 5FDCEDA75C7C53D0D076B15590B35693BA4C446F58BDBCB7F2F27828175921CFC0CEF0C3D8826E1844633FE83ECAE73F
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 172800 (seconds)
    TLS session ticket:
    0000 - 01 ed 2a 78 ae e2 dd bf-5f 3a 1d 92 7c 1e b5 5a   ..*x...._:..|..Z
    0010 - 7f 48 2c c0 07 a7 9b e0-61 a2 1d b5 74 c2 b0 01   .H,.....a...t...
    0020 - 73 c2 81 f6 97 67 db 3a-37 8d e1 46 6f c2 84 6b   s....g.:7..Fo..k
    0030 - 5f 31 4e 5c 1d e3 c9 48-72 f4 38 5d 45 d3 75 08   _1N\...Hr.8]E.u.
    0040 - 0b e3 8c 21 d6 50 b6 0d-7a 09 9c 62 93 74 24 c7   ...!.P..z..b.t$.
    0050 - 40 ed c1 d2 c4 40 e5 c9-56 0f 5a 80 c9 7e 9e e3   @....@..V.Z..~..
    0060 - 1c af 07 a3 f8 87 f2 24-bc 2a d5 3c 92 ef 65 a2   .......$.*.<..e.
    0070 - e3 d2 a4 7e 7c 82 ff 7d-c6 77 a0 30 3b 6a c9 7b   ...~|..}.w.0;j.{
    0080 - a7 a5 f9 7d eb c2 f7 4a-70 ea 97 23 c2 9b 0c 53   ...}...Jp..#...S
    0090 - f9 2a 05 80 4e 1f 51 9c-aa 54 47 2f e9 63 3e 20   .*..N.Q..TG/.c>
    00a0 - 02 94 bc 61 1a fe a6 9c-fc 87 09 d9 54 43 84 59   ...a........TC.Y
    00b0 - 56 2f b3 17 71 af 94 df-5c 36 92 f1 89 ae ba 06   V/..q...\6......
    00c0 - 21 86 d1 1e 9b e9 46 5f-d7 7d eb eb c6 fa 5a de   !.....F_.}....Z.
    00d0 - b0 a5 5e bc 76 47 e6 6c-8c 7b f8 c8 63 31 3a 2f   ..^.vG.l.{..c1:/
    00e0 - 84 b0 46 95 b9 8b e8 eb-db c6 e0 f3               ..F.........

    Start Time: 1611244044
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 3EF884BC758A65FE1B1D67A6585604C13F8C5FE9546B02070C6943C59F543C97
    Session-ID-ctx:
    Resumption PSK: 38E3B86F40A51C74653F349D341939B5EA7E4C098AE0AA59BD04A794B63D811DE069AE4A5A9E64FFC0BE1B3B9F782352
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 172800 (seconds)
    TLS session ticket:
    0000 - 01 ed 2a 78 ae e2 dd bf-5f 3a 1d 92 7c 1e b5 5a   ..*x...._:..|..Z
    0010 - 88 c9 f7 1e 56 35 f4 dd-4a b5 d6 c8 a2 a6 20 cf   ....V5..J..... .
    0020 - cd 8d ad db 9d b4 9f 65-f4 5b 7d 4d 66 55 5f 49   .......e.[}MfU_I
    0030 - 7c 8a c0 0d f6 21 c7 b7-48 c1 38 d5 24 f0 35 c1   |....!..H.8.$.5.
    0040 - 64 4b f2 c7 95 39 b5 d4-44 d2 b3 04 76 2e 3a 43   dK...9..D...v.:C
    0050 - ae 75 90 c6 79 cb df 92-4b 87 8c 59 93 6c f6 21   .u..y...K..Y.l.!
    0060 - 5f 7c 61 ce c7 97 60 55-5f 67 6e b6 2c 54 a0 47   _|a...`U_gn.,T.G
    0070 - 72 1e 9f a4 31 de 55 fe-69 ed 7d f9 a5 db e1 92   r...1.U.i.}.....
    0080 - 1d 06 7e c3 b2 7d c7 cf-f8 f0 29 c0 c8 37 b5 e9   ..~..}....)..7..
    0090 - 5e cb 62 94 65 4e fb be-87 fd e9 7a 20 9f f6 3c   ^.b.eN.....z ..<
    00a0 - da e0 9a d0 ad e2 03 36-71 72 aa 3a 10 87 b1 86   .......6qr.:....
    00b0 - 79 60 9c 66 2e c5 f1 8f-8c c1 f4 eb 7d a7 6b db   y`.f........}.k.
    00c0 - 8d ae 21 ff 3f a8 c1 81-29 07 9d fe 26 d1 25 e2   ..!.?...)...&.%.
    00d0 - 34 fa 0b 9c 1a 87 10 ed-ef 97 09 0f 92 7d e9 19   4............}..
    00e0 - 3a 65 6c 31 b2 8b f3 c7-fb a4 75 0a               :el1......u.

    Start Time: 1611244044
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
220 smtp.gmail.com ESMTP c7sm3879522qkm.99 - gsmtp

Is it possible that the openssl.cnf file expects the format of the algorithm name to be TLS_AES_256_GCM_SHA384 instead of AES256-GCM-SHA384 and that's all you need to do?

[mihail-brinza]

Thank you for your answer!

From my research AES256-GCM-SHA384 and TLS_AES_256_GCM_SHA384 are different ciphers, their authentication is different.

When I use OpenSsl to connect to the server I get this:

openssl s_client -connect smtp.mngt.local:465
CONNECTED(00000003)
depth=0 C = PT, ST = Campo Grande, L = Lisboa, O = NOS DC, CN = smtp.mngt.local, emailAddress = admin@dc.nos.pt
verify error:num=18:self signed certificate
verify return:1
depth=0 C = PT, ST = Campo Grande, L = Lisboa, O = NOS DC, CN = smtp.mngt.local, emailAddress = admin@dc.nos.pt
verify return:1
---
Certificate chain
 0 s:C = PT, ST = Campo Grande, L = Lisboa, O = NOS DC, CN = smtp.mngt.local, emailAddress = admin@dc.nos.pt
   i:C = PT, ST = Campo Grande, L = Lisboa, O = NOS DC, CN = smtp.mngt.local, emailAddress = admin@dc.nos.pt
---
Server certificate
-----BEGIN CERTIFICATE-----
MIID1TCCAr2gAwIBAgIJANUClQo1V4djMA0GCSqGSIb3DQEBCwUAMIGAMQswCQYD
VQQGEwJQVDEVMBMGA1UECAwMQ2FtcG8gR3JhbmRlMQ8wDQYDVQQHDAZMaXNib2Ex
DzANBgNVBAoMBk5PUyBEQzEYMBYGA1UEAwwPc210cC5tbmd0LmxvY2FsMR4wHAYJ
KoZIhvcNAQkBFg9hZG1pbkBkYy5ub3MucHQwHhcNMTkwMTIxMTQ0ODA3WhcNMjUw
MTE5MTQ0ODA3WjCBgDELMAkGA1UEBhMCUFQxFTATBgNVBAgMDENhbXBvIEdyYW5k
ZTEPMA0GA1UEBwwGTGlzYm9hMQ8wDQYDVQQKDAZOT1MgREMxGDAWBgNVBAMMD3Nt
dHAubW5ndC5sb2NhbDEeMBwGCSqGSIb3DQEJARYPYWRtaW5AZGMubm9zLnB0MIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAshe3Pne/fegAfT7KSNgeFh9Z
3eOLnSCQ5x/U/vRNygsdH5Zhx35rzX7ru5UaobzAwC80FQksK1GVnUaOBMEqGmQ6
srs6zsy/1a7jC4tjNZh/LMWprS8BX/UMf+70gVRsyW5iLTzBVuPxo/UDmp1Hs/Er
Msva5UkHjBvdRsjze+KXQ5SBupAZ0g/5u5+j/FqeUpckpNjPSri7ITj4FRJzCsFj
PuJ0zHHrhpIfoPWtUc13sECquFukPs5Sp3pyp2sa/zDfRWaBgCqnNx9rI4DiOczO
wUWEJcFz9oGaRkM7vO8lra3UM6PmV72ItD95I9z79Zcty8zhBEHp4SGmgQxfLQID
AQABo1AwTjAdBgNVHQ4EFgQUrKzpacIm6baoxQoPnDqQpSMAD9kwHwYDVR0jBBgw
FoAUrKzpacIm6baoxQoPnDqQpSMAD9kwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0B
AQsFAAOCAQEAUrQZa0TQRpcOoR3AnY/nkNPTCuiOh24MqE+0l11e2TxhTDhwcwHK
EpWdL+f9MQMI5ncn3d9gyYYtj1yuQNQEv25o9R7DEmF1tMw4qximOZHkm91Gvwny
sktf1fCEd1J0LGoE0BxTjx67lyY+nTNP0kV1gbjY//iEkbTZ1pMuQrjHe+y+uPEe
05KW3UNKPOQ0pWWaWTvVKxWTnvLhGUqypcT+Jv0yXpgS1m04eAO0DRBPMuZz3UER
Ea1JB2SPJnYznCsx1aCkj5w4EcT4w1umNWopx1IioTEbT/SaHO4FNS40yx+S7Gsv
Zlr7sP3MdkANkn9F5qKFXbAQhvwu1tyF7Q==
-----END CERTIFICATE-----
subject=C = PT, ST = Campo Grande, L = Lisboa, O = NOS DC, CN = smtp.mngt.local, emailAddress = admin@dc.nos.pt

issuer=C = PT, ST = Campo Grande, L = Lisboa, O = NOS DC, CN = smtp.mngt.local, emailAddress = admin@dc.nos.pt

---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224
---
SSL handshake has read 1351 bytes and written 637 bytes
Verification error: self signed certificate
---
New, TLSv1.2, Cipher is AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Session-ID: 13951D0A92FFF79CD904858A2948B8B43CF50E6ABC518370DD9AE66DC8ED4D5C
    Session-ID-ctx: 
    Master-Key: C484F7ADC846A31F7268AE9D150489D2136496377BD3257FC48ADF8CE60F4ABCE45E497CD6B441A145F14A38B3142E0A
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 6c 5c a0 30 f1 a3 e0 09-31 2e 0b 32 69 1b 21 56   l\.0....1..2i.!V
    0010 - 57 81 ae 71 18 90 8c 11-fa 9d 4e 4e a4 00 da 48   W..q......NN...H
    0020 - 2f d7 ed 73 05 6e 93 08-35 c3 89 b1 d5 9f f6 e8   /..s.n..5.......
    0030 - 23 8a 8a 49 0d 99 e7 d9-13 0b 64 c2 22 96 f5 e3   #..I......d."...
    0040 - 6e a7 2b 47 e3 24 a0 b8-3a 9f a5 4b 6c d4 b7 ff   n.+G.$..:..Kl...
    0050 - f7 bc 2d 9f f1 a1 14 53-c0 99 01 b2 98 4d e6 ef   ..-....S.....M..
    0060 - e3 61 af d8 6f 9a a8 c1-ad e4 ce e3 2b 74 ef 9c   .a..o.......+t..
    0070 - 39 81 9c 26 4a 76 9d f6-32 01 b8 93 1f 25 9b 0e   9..&Jv..2....%..
    0080 - da ca cb c2 3c ab 9b 71-dd ff 7a b4 d6 23 99 9b   ....<..q..z..#..
    0090 - a8 80 b2 96 6d ac 49 8c-1c fd cf c9 7e c1 4c 06   ....m.I.....~.L.
    00a0 - 74 9c c6 79 fb 14 3d cb-71 ab 5a 01 9f 26 85 55   t..y..=.q.Z..&.U

    Start Time: 1611247663
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
    Extended master secret: no
---
220 smtp.dc.nos.pt ESMTP Haraka/2.8.23 ready

From this we can see that it is using a self signed certificate, however I am using

client.ServerCertificateValidationCallback = (s,c,h,e) => true;
client.CheckCertificateRevocation = false;

And furthermore, it doesn't make sense to work perfectly on .NET Core 3.1 and not in .NET 5

I have tried with Ubuntu and Alpine ditros, and I appended these lines to /etc/ssl/openssl.cnf

openssl_conf = default_conf

[default_conf]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
CipherString = AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256

[jstedfast]

From my research AES256-GCM-SHA384 and TLS_AES_256_GCM_SHA384 are different ciphers, their authentication is different.

Ah, good to know! Looks like, based on your openssl output, that the algorithm name that you used in the config file would be correct.

I'm curious, what if you set client.ServerCertificateValidationCallback to a method with the proper signature that returns true but also prints something to the console so that you can see if it even gets called?

That might at least tell us if it's getting that far or if it's failing before that check.

[mihail-brinza]

I'm curious, what if you set client.ServerCertificateValidationCallback to a method with the proper signature that returns true but also prints something to the console so that you can see if it even gets called?

I did that and I can not see the log! It must fail before that

[jstedfast]

That suggests that the SslStream hasn't gotten around to validating certificates, yet, then, which supports your theory that it has something to do with negotiating the supported algorithms.

[jstedfast]

Looks like .NET 5.0 added AuthenticateAsClientAsync(SslClientAuthenticationOptions, CancellationToken) and SslClientAuthenticationOptions has a CipherSuitesPolicy property that allows setting which cipher algorithms to allow.

Maybe I can extend MailKit's IMailService API to allow setting a CipherSuitePolicy as well for NET50.

[mihail-brinza]

Yes that is what I was going to suggest, a way of setting the CipherSuitesPolicy. Setting it at "system default" does not seem to be easy, and also, a developer may want to only allow weak ciphers to a specific mail server, and not to the whole system.
I can even help with the code if you need me to :)

Thank you for your time!

[jstedfast]

The above patch should fix this issue. I've got to get back to work, but hopefully some myget packages will be pushed by the automated build system to https://www.myget.org/feed/mimekit/package/nuget/MailKit

It'll be v2.10.1.8, I think.

There's a new property on each of the Smtp/Pop3/ImapClients called SslCipherSuitesPolicy that you can set.

[mihail-brinza]

Thank you! I will test again when this change is out and I will let you know if it worked.

[mihail-brinza]

Hi,
I tried to test this but I don't seem to find the property SslCipherSuitesPolicy . I'm using .NET 5.0 and Mailkit 2.11.1

I was looking at your commit and I noticed you have #if NET50, and I looked up the documentation and it says that we should use NET5_0 instead, so this could be the problem.

Thank you

[mihail-brinza]

I can confirm that this change fixed the issue.