JAMES-3680 SMTP enable OAUTHBEARER authentication #810

quantranhong1999 · 2021-12-27T10:41:00Z

WIP: reconsider logic and improve code quality (reduce some if else if it can be,...), documentation

As the code stands now, when client does not using TLS connect (server requireSSL) the sever does not advertise any AUTH types:

250-hp-quanth Hello localhost [192.168.42.103])
250-PIPELINING
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 STARTTLS

And only return available auth when client executeTLS:

250-hp-quanth Hello localhost [192.168.42.103])
250-AUTH OAUTHBEARER
250-AUTH=OAUTHBEARER
250-PIPELINING
250-ENHANCEDSTATUSCODES
250 8BITMIME

Is this a correct behavior? Should we return Auth even when client does not use TLS?
RFC4954:
Modern implementations SHOULD NOT advertise mechanisms that are not permitted due to lack of encryption, unless an encryption layer of sufficient strength is currently being employed => I suppose this is correct behavior.

quantranhong1999 · 2021-12-29T04:35:44Z

openssl client test:

Success case:

 ✘ hp@hp-quanth  ~  openssl s_client -starttls smtp -crlf -connect localhost:39983 
CONNECTED(00000003)
Can't use SSL_get_servername
depth=0 C = FR, ST = Unknown, L = Puteaux, O = James, OU = Linagora, CN = Benoit Tellier
verify error:num=18:self signed certificate
verify return:1
depth=0 C = FR, ST = Unknown, L = Puteaux, O = James, OU = Linagora, CN = Benoit Tellier
verify error:num=10:certificate has expired
notAfter=Nov 24 07:32:55 2015 GMT
verify return:1
depth=0 C = FR, ST = Unknown, L = Puteaux, O = James, OU = Linagora, CN = Benoit Tellier
notAfter=Nov 24 07:32:55 2015 GMT
verify return:1
---
Certificate chain
 0 s:C = FR, ST = Unknown, L = Puteaux, O = James, OU = Linagora, CN = Benoit Tellier
   i:C = FR, ST = Unknown, L = Puteaux, O = James, OU = Linagora, CN = Benoit Tellier
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDeTCCAmGgAwIBAgIEXaLC/zANBgkqhkiG9w0BAQsFADBtMQswCQYDVQQGEwJG
UjEQMA4GA1UECBMHVW5rbm93bjEQMA4GA1UEBxMHUHV0ZWF1eDEOMAwGA1UEChMF
SmFtZXMxETAPBgNVBAsTCExpbmFnb3JhMRcwFQYDVQQDEw5CZW5vaXQgVGVsbGll
cjAeFw0xNTA4MjYwNzMyNTVaFw0xNTExMjQwNzMyNTVaMG0xCzAJBgNVBAYTAkZS
MRAwDgYDVQQIEwdVbmtub3duMRAwDgYDVQQHEwdQdXRlYXV4MQ4wDAYDVQQKEwVK
YW1lczERMA8GA1UECxMITGluYWdvcmExFzAVBgNVBAMTDkJlbm9pdCBUZWxsaWVy
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAhW0GWSgkK44XQpwLn8KX
q9kJ3zgDYllEX7W8p+3sBYMAP0JC+lzh42mX/8XHti0vkmv/vjk0paAsB9s5uYhv
/W94sdqjexi3L213+OD5Kcy+2tTgXN1ucF65d/dDeqGHAyBs1rm6LgyRwQ5ye8Cl
OtXbSkz5qujmnFDDzrDIgzefxFsTHHIBTsdyttq2Atqzgdg2LHaFbIntwr6lfl9v
puXr8p+CY6PfehX8mdmaJ7J/gfAll3zzdgeNUoAW0eDvcsphJ06elsDahieo0n/o
XVhy+TGWtAQRDZQDjeK2BX/vVSPAlGOW7hVTv7WycAolzKlXZyiJMnwTWJl6YoUP
vQIDAQABoyEwHzAdBgNVHQ4EFgQU7tDLswRlOJqoX4aVgrXRQbmOYbIwDQYJKoZI
hvcNAQELBQADggEBAGmzK2i2H5D6xITyN1iNQhXbABQ/3rz9K5dEvy0ZLgYOUqyw
1WD43Do/A3eZuEZbuuYAlbvDEUNgS7rJ5HCrwIKFbPHWmfRX2j59UX+R8fI6G9wB
qqcRYWpert347rGT/7MFLHk55LL6Tf//fwoWu6GWRj3wTvosVuunUiiC6zTS4MN9
moJp+IF03Q6JOPWu7/tfaKfXQHxG/hK492oV2vBG2r29UUJW6YO1S0DK+/cU0cCu
+jqbY1ZOIAk906onRUFoPGuypOm3vmbE6mo5o49rNtp+VmZagZ7GsjJ4KWJB1c6d
SFNIFlH2VlS8Lywr2tesClWO5tqtMswRsoA9GeU=
-----END CERTIFICATE-----
subject=C = FR, ST = Unknown, L = Puteaux, O = James, OU = Linagora, CN = Benoit Tellier

issuer=C = FR, ST = Unknown, L = Puteaux, O = James, OU = Linagora, CN = Benoit Tellier

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1613 bytes and written 396 bytes
Verification error: certificate has expired
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 10 (certificate has expired)
---
250 STARTTLS
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: F9B28D6A7E1BD26A6F5781FC63A042C884917BF323202D126A332CEF94820310
    Session-ID-ctx: 
    Resumption PSK: 6BBC0ACE586E5DF5328C23F7E5FFE602FEB4B0AD9175A1A80E670A5F24001A354676D2B275BC1874541F2AF0199F60C3
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 86400 (seconds)
    TLS session ticket:
    0000 - 30 8d f3 6d b9 e8 72 3a-56 f1 6b bf 0b 2f 6d e6   0..m..r:V.k../m.
    0010 - 8d 3a 27 1d c8 06 8f d8-79 1b a1 bf 49 90 d0 f7   .:'.....y...I...

    Start Time: 1640752217
    Timeout   : 7200 (sec)
    Verify return code: 10 (certificate has expired)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
EHLO localhost
250-hp-quanth Hello localhost [127.0.0.1])
250-AUTH OAUTHBEARER
250-AUTH=OAUTHBEARER
250-PIPELINING
250-ENHANCEDSTATUSCODES
250 8BITMIME
AUTH OAUTHBEARER bixhPXVzZXJAZG9tYWluLm9yZwFhdXRoPWV5SmhiR2NpT2lKU1V6STFOaUlzSW5SNWNDSTZJa3BYVkNJc0ltdHBaQ0k2SW5jNE1GQnpOVWxoYzI0dFlVZFhiWGN5VkhKNFJHbE9ZMkZvY0VneWMxaDZOWEJ4WkdoQmJEbElXR01pZlEuZXlKbGVIQWlPak01TXprMU1EWXhOamNzSW1saGRDSTZNVFl6T1RVd05UZzJOeXdpWVhWMGFGOTBhVzFsSWpvek5qTTVOVEExT0RReExDSnFkR2tpT2lKak1qUTVaVEJrTmkxalkySmlMVFJtWkRBdE9ESTVZaTA0T1RNMU1qY3pOMll6WkdJaUxDSnBjM01pT2lKb2RIUndPaTh2Ykc5allXeG9iM04wT2pnd09EQXZZWFYwYUM5eVpXRnNiWE12Y21WaGJHMHhJaXdpWVhWa0lqb2lZV05qYjNWdWRDSXNJbk4xWWlJNklqSXdORFV5TnpGaUxXTXhZbUl0TkRKaU9DMWhNVGt3TFRobFlXSTFNbVl6WW1Fd09TSXNJblI1Y0NJNklrSmxZWEpsY2lJc0ltRjZjQ0k2SW1GalkyOTFiblF0WTI5dWMyOXNaU0lzSW01dmJtTmxJam9pTldVeU9HSmpOVEF0T0RFNU5TMDBOak0zTFRobU1XRXRZV1V6TldGbFlUazBOVGMxSWl3aWMyVnpjMmx2Ymw5emRHRjBaU0k2SW1NeFl6STNNbVl3TFdNd01qQXROR1ptTUMxaE16WXdMVFEzTUdKbFlXVmxOV1V3TUNJc0ltRmpjaUk2SWpBaUxDSnlaWE52ZFhKalpWOWhZMk5sYzNNaU9uc2lZV05qYjNWdWRDSTZleUp5YjJ4bGN5STZXeUp0WVc1aFoyVXRZV05qYjNWdWRDSXNJbTFoYm1GblpTMWhZMk52ZFc1MExXeHBibXR6SWwxOWZTd2ljMk52Y0dVaU9pSnZjR1Z1YVdRZ1pXMWhhV3dnY0hKdlptbHNaU0lzSW5OcFpDSTZJbU14WXpJM01tWXdMV013TWpBdE5HWm1NQzFoTXpZd0xUUTNNR0psWVdWbE5XVXdNQ0lzSW1WdFlXbHNYM1psY21sbWFXVmtJanBtWVd4elpTd2ljSEpsWm1WeWNtVmtYM1Z6WlhKdVlXMWxJam9pYW1GdFpYTWlMQ0psYldGcGJGOWhaR1J5WlhOeklqb2lkWE5sY2tCa2IyMWhhVzR1YjNKbkluMC5icUhzWDN5bmdYd1h5Vlc3TGVuS3pIYmRxWnkxQW1DakUzUVdycDdZMXNkX3pjUUV1NVdBQndMSU9BenJYaU5GZUd3eXd3OHRhR0pCZFlhMEtUQkNZNk1Za0FIQUVhMXZ5eU8xTGZKZ3IzY0lmUVQ2V0NmM2cyQkpxSFJqVXNxTmdUX1NpdDlkcnVNUmtlMDFtMVYwRW16cUlkTExIcDhWbC11NFIzSlNEeDFic1ExdzNXQ1JsY2dyX2szRUo3ak5pdU5ua2xDSDhfbzU5eTRjN1J6ZHBsLVk4dGNBMDduR2plSl83cVBnTlpYNmxnd3ZyMEVocFFwYlZESFh3UWxwMk5EemtXd0JMSlIwLVY1MFEwYS1MMFFENjl3cWVFYXFpMXhhUkFmeDJHd24yRmdDZ01VV3pLZVdfcWtFQlAwdG5OLXB6bDdqMzFFT25tS2hzaGxPdHcBAQ==
235 Authentication successful.

Fail case:

 hp@hp-quanth  ~  openssl s_client -starttls smtp -crlf -connect localhost:39983
CONNECTED(00000003)
...
EHLO localhost               
250-hp-quanth Hello localhost [127.0.0.1])
250-AUTH OAUTHBEARER
250-AUTH=OAUTHBEARER
250-PIPELINING
250-ENHANCEDSTATUSCODES
250 8BITMIME
AUTH OAUTHBEARER invalidtoken
334 eyJzdGF0dXMiOiJpbnZhbGlkX3Rva2VuIiwic2NvcGUiOiJzY29wZSIsInNjaGVtZXMiOiJodHRwczovL2V4YW1wbGUuY29tL2p3a3MifQ==
AQ==
535 Authentication Failed

Telnet try to do OAUTHBEARER when not STARTTLS should fail:

 hp@hp-quanth  ~  telnet localhost 39983
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 Apache JAMES awesome SMTP Server
EHLO localhost
250-hp-quanth Hello localhost [127.0.0.1])
250-PIPELINING
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 STARTTLS
AUTH OAUTHBEARER bixhPXVzZXJAZG9tYWluLm9yZwFhdXRoPWV5SmhiR2NpT2lKU1V6STFOaUlzSW5SNWNDSTZJa3BYVkNJc0ltdHBaQ0k2SW5jNE1GQnpOVWxoYzI0dFlVZFhiWGN5VkhKNFJHbE9ZMkZvY0VneWMxaDZOWEJ4WkdoQmJEbElXR01pZlEuZXlKbGVIQWlPak01TXprMU1EWXhOamNzSW1saGRDSTZNVFl6T1RVd05UZzJOeXdpWVhWMGFGOTBhVzFsSWpvek5qTTVOVEExT0RReExDSnFkR2tpT2lKak1qUTVaVEJrTmkxalkySmlMVFJtWkRBdE9ESTVZaTA0T1RNMU1qY3pOMll6WkdJaUxDSnBjM01pT2lKb2RIUndPaTh2Ykc5allXeG9iM04wT2pnd09EQXZZWFYwYUM5eVpXRnNiWE12Y21WaGJHMHhJaXdpWVhWa0lqb2lZV05qYjNWdWRDSXNJbk4xWWlJNklqSXdORFV5TnpGaUxXTXhZbUl0TkRKaU9DMWhNVGt3TFRobFlXSTFNbVl6WW1Fd09TSXNJblI1Y0NJNklrSmxZWEpsY2lJc0ltRjZjQ0k2SW1GalkyOTFiblF0WTI5dWMyOXNaU0lzSW01dmJtTmxJam9pTldVeU9HSmpOVEF0T0RFNU5TMDBOak0zTFRobU1XRXRZV1V6TldGbFlUazBOVGMxSWl3aWMyVnpjMmx2Ymw5emRHRjBaU0k2SW1NeFl6STNNbVl3TFdNd01qQXROR1ptTUMxaE16WXdMVFEzTUdKbFlXVmxOV1V3TUNJc0ltRmpjaUk2SWpBaUxDSnlaWE52ZFhKalpWOWhZMk5sYzNNaU9uc2lZV05qYjNWdWRDSTZleUp5YjJ4bGN5STZXeUp0WVc1aFoyVXRZV05qYjNWdWRDSXNJbTFoYm1GblpTMWhZMk52ZFc1MExXeHBibXR6SWwxOWZTd2ljMk52Y0dVaU9pSnZjR1Z1YVdRZ1pXMWhhV3dnY0hKdlptbHNaU0lzSW5OcFpDSTZJbU14WXpJM01tWXdMV013TWpBdE5HWm1NQzFoTXpZd0xUUTNNR0psWVdWbE5XVXdNQ0lzSW1WdFlXbHNYM1psY21sbWFXVmtJanBtWVd4elpTd2ljSEpsWm1WeWNtVmtYM1Z6WlhKdVlXMWxJam9pYW1GdFpYTWlMQ0psYldGcGJGOWhaR1J5WlhOeklqb2lkWE5sY2tCa2IyMWhhVzR1YjNKbkluMC5icUhzWDN5bmdYd1h5Vlc3TGVuS3pIYmRxWnkxQW1DakUzUVdycDdZMXNkX3pjUUV1NVdBQndMSU9BenJYaU5GZUd3eXd3OHRhR0pCZFlhMEtUQkNZNk1Za0FIQUVhMXZ5eU8xTGZKZ3IzY0lmUVQ2V0NmM2cyQkpxSFJqVXNxTmdUX1NpdDlkcnVNUmtlMDFtMVYwRW16cUlkTExIcDhWbC11NFIzSlNEeDFic1ExdzNXQ1JsY2dyX2szRUo3ak5pdU5ua2xDSDhfbzU5eTRjN1J6ZHBsLVk4dGNBMDduR2plSl83cVBnTlpYNmxnd3ZyMEVocFFwYlZESFh3UWxwMk5EemtXd0JMSlIwLVY1MFEwYS1MMFFENjl3cWVFYXFpMXhhUkFmeDJHd24yRmdDZ01VV3pLZVdfcWtFQlAwdG5OLXB6bDdqMzFFT25tS2hzaGxPdHcBAQ==
504 Unrecognized Authentication Type

Setup keycloak standalone (run by shell script/ did not succeed with standalone keycloak docker) with James SMTP config towards to keycloak => Succeed with valid ID Token:

Targeting keycloak backed by LDAP then set up a testing docker compose..

chibenwa

Looks good to me.

Thanks for putting this together.

chibenwa · 2022-01-05T06:27:56Z

Failed

org.apache.james.MemoryJmapJamesServerTest$JmapJamesServerTest.mailsShouldBeWellReceived{GuiceJamesServer} (from org.apache.james.MemoryJmapJamesServerTest) 

org.apache.james.utils.SMTPSendingException: 
Error upon step Authentication: 504 Unrecognized Authentication Type

Smells related. Could you have a look?

chibenwa · 2022-01-05T06:30:41Z

& rebase needed, which might solve the aforementionned issues.

I will do the rebase.

chibenwa · 2022-01-05T07:13:17Z

IMO we need auth=Bearer token and not just auth=token like in @vttranlina pr #796

vttranlina · 2022-01-05T09:17:15Z

protocols/smtp/src/main/java/org/apache/james/protocols/smtp/core/esmtp/AuthCmdHandler.java

-                return ESMTP_FEATURES;
+                authTypesBuilder.add(AUTH_TYPE_LOGIN, AUTH_TYPE_PLAIN);
+            }
+            if (session.getConfiguration().saslConfiguration().isPresent()) {


IMO session.getConfiguration().enableOAuth() is better.
Example: org.apache.james.smtpserver.SMTPSaslTest#oauthWithNoTLSConnectShouldFail

…uld not require SSL

vttranlina · 2022-01-07T07:43:44Z

I will resolve it

vttranlina · 2022-01-07T08:11:00Z

Rebase master & resolve conflict
Fix error: JAMES-3680 SMTP enable OAUTHBEARER authentication #810 (comment)

vttranlina · 2022-01-07T08:14:35Z

IMO we need auth=Bearer token and not just auth=token like in @vttranlina pr #796

Resolved!
After rebasing the latest master, It has been updated auto

chibenwa · 2022-01-07T08:35:56Z

Ok. Can we try this thing with TB ? :-P

vttranlina · 2022-01-07T09:51:19Z

server/mailet/integration-testing/src/main/resources/smtpserver.xml

@@ -55,9 +55,9 @@
    </smtpserver>
    <smtpserver enabled="true">
        <jmxName>smtpserver-TLS</jmxName>
-        <bind>0.0.0.0:0</bind>
+        <bind>0.0.0.0:465</bind>


This change makes some integration tests fail.
java.net.SocketException: Permission denied

revert smtp config file

Should support XOAUTH2 type

vttranlina · 2022-01-07T12:12:24Z

POC - Demo Video with Thunderbird

Setup OIDC Provider

Create realm
Create app client
Create user:
- user2@james.org

Add Oauth Provider Server to Thunderbird
Adding manually at thunderbird path /thunderbird/omni.ja/OAuth2Providers.jsm
(ref https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat#OAuth2)
Setup James Server

Config enable SMTP oidc authentication
Create user + mailbox:
- user1@james.org (receiver via IMAP)
- user2@james.org (sender via SMTP - OAUTH)

Testing

user1@james.org : Using IMAP for fetching email
user2@james.org : Send email to user1@james.org via SMTP OAUTH2

OAUTH_SMTP_JAMES_DEMO.mp4

chibenwa reviewed Jan 4, 2022

View reviewed changes

Arsnael approved these changes Jan 4, 2022

View reviewed changes

chibenwa mentioned this pull request Jan 4, 2022

POC implementation of SASL for SMTP #792

Closed

chibenwa force-pushed the smtp-oauth branch from 0305336 to af61023 Compare January 5, 2022 07:11

vttranlina reviewed Jan 5, 2022

View reviewed changes

quantranhong1999 added 2 commits January 7, 2022 14:31

JAMES-3680 SMTP enable OAUTHBEARER authentication

3a74b5e

JAMES-3680 Sample configurations: SMTP server without TLS enabled sho…

f8f40aa

…uld not require SSL

fixup! JAMES-3680 SMTP enable OAUTHBEARER authentication

5655678

vttranlina force-pushed the smtp-oauth branch from af61023 to 5655678 Compare January 7, 2022 08:09

vttranlina reviewed Jan 7, 2022

View reviewed changes

vttranlina added 2 commits January 7, 2022 17:27

fixup! JAMES-3680 SMTP enable OAUTHBEARER authentication

c9f0c92

revert smtp config file

fixup! JAMES-3680 SMTP enable OAUTHBEARER authentication

8a32f73

Should support XOAUTH2 type

chibenwa approved these changes Jan 7, 2022

View reviewed changes

chibenwa merged commit 47d443c into apache:master Jan 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JAMES-3680 SMTP enable OAUTHBEARER authentication #810

JAMES-3680 SMTP enable OAUTHBEARER authentication #810

quantranhong1999 commented Dec 27, 2021 •

edited

Loading

quantranhong1999 commented Dec 29, 2021 •

edited

Loading

chibenwa left a comment

chibenwa commented Jan 5, 2022

chibenwa commented Jan 5, 2022

chibenwa commented Jan 5, 2022

vttranlina Jan 5, 2022

vttranlina commented Jan 7, 2022

vttranlina commented Jan 7, 2022

vttranlina commented Jan 7, 2022

chibenwa commented Jan 7, 2022

vttranlina Jan 7, 2022

vttranlina Jan 7, 2022

vttranlina commented Jan 7, 2022

JAMES-3680 SMTP enable OAUTHBEARER authentication #810

JAMES-3680 SMTP enable OAUTHBEARER authentication #810

Conversation

quantranhong1999 commented Dec 27, 2021 • edited Loading

quantranhong1999 commented Dec 29, 2021 • edited Loading

chibenwa left a comment

Choose a reason for hiding this comment

chibenwa commented Jan 5, 2022

chibenwa commented Jan 5, 2022

chibenwa commented Jan 5, 2022

vttranlina Jan 5, 2022

Choose a reason for hiding this comment

vttranlina commented Jan 7, 2022

vttranlina commented Jan 7, 2022

vttranlina commented Jan 7, 2022

chibenwa commented Jan 7, 2022

vttranlina Jan 7, 2022

Choose a reason for hiding this comment

vttranlina Jan 7, 2022

Choose a reason for hiding this comment

vttranlina commented Jan 7, 2022

POC - Demo Video with Thunderbird

quantranhong1999 commented Dec 27, 2021 •

edited

Loading

quantranhong1999 commented Dec 29, 2021 •

edited

Loading