SMTP Auth is not using XOAUTH2 even when specified with --login-options 'AUTH=XOAUTH2' \ #10259
Comments
|
UPDATE: See my follow-up comment.
Compared to the reporter @ndevln they have cited two versions of curl they tested? While This is a weird one... I have no issue with You can reproduce a working example I shared here, but use This is running on Windows 10 23H2 Build 22631.3007, via WSL2 (Ubuntu): # Start the `compose.yaml` example:
$ docker compose up -d
# Correctly uses XOAUTH2:
$ docker exec -it dms-mail \
curl -sv --url 'smtp://localhost:587' \
--login-options 'AUTH=XOAUTH2' --oauth2-bearer DMS_YWNjZXNzX3Rva2Vu --user john.doe@example.test \
--mail-from john.doe@example.test --mail-rcpt jane.doe@example.test --upload-file - <<< 'Hello Jane!'
* Trying 127.0.0.1:587...
* Connected to localhost (127.0.0.1) port 587 (#0)
< 220 mail.example.test ESMTP
> EHLO mail
< 250-mail.example.test
< 250-PIPELINING
< 250-SIZE 10240000
< 250-ETRN
< 250-AUTH PLAIN LOGIN OAUTHBEARER XOAUTH2
< 250-AUTH=PLAIN LOGIN OAUTHBEARER XOAUTH2
< 250-ENHANCEDSTATUSCODES
< 250-8BITMIME
< 250-DSN
< 250 CHUNKING
> AUTH XOAUTH2
< 334
> dXNlcj1qb2huLmRvZUBleGFtcGxlLnRlc3QBYXV0aD1CZWFyZXIgRE1TX1lXTmpaWE56WDNSdmEyVnUBAQ==
< 235 2.7.0 Authentication successful
> MAIL FROM:<john.doe@example.test>
< 250 2.1.0 Ok
> RCPT TO:<jane.doe@example.test>
< 250 2.1.5 Ok
> DATA
< 354 End data with <CR><LF>.<CR><LF>
} [12 bytes data]
< 521 5.5.2 mail.example.test Error: bare <LF> received
* Connection #0 to host localhost left intact
# Postfix log:
$ docker logs dms-mail | grep 'postfix/submission/smtpd.*sasl_method'
Jan 24 00:01:40 mail postfix/submission/smtpd[1011]: 03A4B1EAEC: client=localhost[127.0.0.1], sasl_method=XOAUTH2, sasl_username=john.doe@example.test
# Curl version info:
$ curl -V
curl 7.74.0 (x86_64-pc-linux-gnu) libcurl/7.74.0 OpenSSL/1.1.1w zlib/1.2.11 brotli/1.0.9 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.43.0 librtmp/2.3
Release-Date: 2020-12-09, security patched: 7.74.0-1.3+deb11u11
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSocketsDebian 12 / WSL2 Ubuntu curl versions: # WSL2
$ curl -V
curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.14
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd
# Debian 12:
$ docker run --rm -it debian:12-slim bash
$ apt-get update && apt-get install curl
$ curl -V
curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1 OpenSSL/3.0.11 zlib/1.2.13 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libpsl/0.21.2 (+libidn2/2.3.3) libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3 OpenLDAP/2.5.13
Release-Date: 2023-02-20, security patched: 7.88.1-10+deb12u5
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstdSame happens with IMAP auth: $ curl -sv --url 'imap://localhost:143' \
--login-options 'AUTH=XOAUTH2' --oauth2-bearer DMS_YWNjZXNzX3Rva2Vu --user john.doe@example.test \
-X LOGOUT
* Trying 127.0.0.1:143...
* Connected to localhost (127.0.0.1) port 143 (#0)
< * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN AUTH=OAUTHBEARER AUTH=XOAUTH2] Dovecot ready.
> A001 CAPABILITY
< * CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN AUTH=OAUTHBEARER AUTH=XOAUTH2
< A001 OK Pre-login capabilities listed, post-login capabilities have more.
> A002 AUTHENTICATE OAUTHBEARER bixhPWpvaG4uZG9lQGV4YW1wbGUudGVzdCwBaG9zdD1sb2NhbGhvc3QBcG9ydD0xNDMBYXV0aD1CZWFyZXIgRE1TX1lXTmpaWE56WDNSdmEyVnUBAQ==
< * CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE QUOTA
< A002 OK Logged in
> A003 LOGOUT
< * BYE Logging out
< A003 OK Logout completed (0.001 + 0.000 secs).
* Connection #0 to host localhost left intact
# Dovecot log:
$ docker logs dms-mail | grep 'dovecot: imap-login: Login:.*method='
Jan 24 00:39:26 mail dovecot: imap-login: Login: user=<john.doe@example.test>, method=OAUTHBEARER, rip=172.19.0.1, lip=172.19.0.4, mpid=4539, session=<jkaKS6YPQsisEwAB>There is nothing special about the Debian 11 container environment, I can reproduce the correct behaviour via a plain Debian 11 container with curl installed there too. |
|
Taking Debian out of the mix, there is versioned curl containers available on DockerHub (which saves me from building/installing curl release binaries myself). This shows the failure was introduced after At a guess, perhaps this change had unintended consequences (no other commits in that range have an Lines 216 to 217 in 2620aa9 Line 363 in 2620aa9 Which somehow results in the Lines 456 to 477 in 2620aa9 Lines 681 to 697 in 2620aa9 |
Yes, I found it: a part of this commit maps given http authentication options to sasl ones and this works fine in the library. However the cli tool sets the http login option I suggest to not preset anymore the sasl flags with http ones, but rather use them as a default, i.e. use them only if no login options has been specified. If this solution is satisfactory, I can have a PR for it very soon. |
If some are given, sasl maps http authentication options to sasl ones and merges them to the login options. This may cause problems with the cli tool that sets the http login option CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag maps to more than one sasl mechanisms and the latter cannot be cleared individually by the login options string. The solution retained here is to consider http authentication options as a default (overriding the hardcoded default mask for the protocol) that is ignored if a login option string is given. New test 992 checks this. Fixes curl#10259
Below should provide an easy to test reproduction environment. One command to get it up and running, then test via local curl build like shown below should verify correct functionality? (at least regarding with mail servers?) Reproduction - Commands# Start the `compose.yaml` example:
$ docker compose up -d
# Verify correct auth method is chosen:
$ curl -sv --url 'imap://localhost:143' \
--login-options 'AUTH=XOAUTH2' --oauth2-bearer DMS_YWNjZXNzX3Rva2Vu --user john.doe@example.test \
-X LOGOUT \
| grep 'AUTHENTICATE XOAUTH2'
$ curl -sv --url 'smtp://localhost:587' \
--login-options 'AUTH=XOAUTH2' --oauth2-bearer DMS_YWNjZXNzX3Rva2Vu --user john.doe@example.test \
--mail-from john.doe@example.test --mail-rcpt jane.doe@example.test --upload-file - <<< 'Hello Jane! \
| grep 'AUTH XOAUTH2'
# Repeat for OAUTHBEARER instead of XOAUTH2Reproduction - Docker Compose config# Normally this would provide TLS configured for both services, that's been omitted to keep the reproduction simple
services:
# Quick and easy mailserver setup with Postfix + Dovecot for testing OAuth2 support
dms:
image: docker.io/mailserver/docker-mailserver:13.3
container_name: dms-mail
hostname: mail.example.test
environment:
# Enable the OAuth2 support in Dovecot and configure it for our mocked service (caddy):
ENABLE_OAUTH2: 1
OAUTH2_INTROSPECTION_URL: http://auth.example.test/userinfo/
# Test authentication against these ports:
ports:
- "143:143" # IMAP STARTTLS (Dovecot)
- "587:587" # SMTP STARTTLS (Postfix)
configs:
- source: dms-accounts
target: /tmp/docker-mailserver/postfix-accounts.cf
# This would normally be a proper auth service, this is sufficient to mock out the required behaviour for testing
caddy-oauth2:
image: caddy:2.7
container_name: dms-oauth2
# Leverage Docker's internal DNS for the private network bridge it creates between services:
hostname: auth.example.test
ports:
- "80:80"
configs:
- source: mock-auth-service
target: /etc/caddy/Caddyfile
# Using the Docker Compose `configs.content` feature instead of volume mounting separate files.
# NOTE: This feature requires Docker Compose v2.23.1 (Nov 2023) or newer:
# https://github.com/compose-spec/compose-spec/pull/446
configs:
# Basic Caddyfile example, see a better documented equivalent at:
# https://github.com/docker-mailserver/docker-mailserver/blob/v13.3.0/test/config/oauth2/Caddyfile
mock-auth-service:
content: |
:80 {
@auth header Authorization "Bearer DMS_YWNjZXNzX3Rva2Vu"
handle @auth {
respond `{ "email": "john.doe@example.test", "email_verified": true }`
}
# Otherwise fail when expected auth header and value were not matched:
respond 401 {
close
}
}
# DMS expects an account to be configured to run, this config provides one
# You can add new accounts with `docker compose exec dms setup email add user@example.test bad-password`
# Login credentials:
# user: "john.doe@example.test" password: "secret"
# user: "jane.doe@example.test" password: "secret"
dms-accounts:
# NOTE: `$` needed to be repeated to escape it,
# which opts out of the `compose.yaml` variable interpolation feature.
content: |
john.doe@example.test|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.
jane.doe@example.test|{SHA512-CRYPT}$$6$$o65y1ZXC4ooOPLwZ$$7TF1nYowEtNJpH6BwJBgdj2pPAxaCvhIKQA6ww5zdHm/AA7aemY9eoHC91DOgYNaKj1HLxSeWNDdvrp6mbtUY. |
I did this
I want to send an E-Mail over SMTP using OAUTH2.
For GMail this fails with
The bearer token is sent with
AUTH OAUTHBEARERand formatted according to the format on line 72:curl/lib/vauth/oauth2.c
Lines 68 to 73 in 5a9a5e1
Which should be supported according to:
< 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTHBut since I specified
--login-options 'AUTH=XOAUTH2'this method should be used.I expected the following
According to Google the bearer token should be sent with
AUTH XOAUTH2https://developers.google.com/gmail/imap/xoauth2-protocol
And this token format should be used:
curl/lib/vauth/oauth2.c
Lines 95 to 100 in 5a9a5e1
Microsoft describes the same standard.
https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
According to this site these are the main provider using SASL XOAUTH2 as the authentication mechanism:
https://mailtrap.io/blog/smtp-auth/
Since I never got it working, I don't know if this is the reason for the login the problem. But curl should use the specified login mechanism.
Thank you for all your work.
curl/libcurl version
AND
operating system
Window 10 22H2 Build 19045.2364
The text was updated successfully, but these errors were encountered: