Email proxy terminating when using allow_catch_all_accounts and authenticating a new email address

[ray-magini]

Hi,

first of all: Great work!

Issue
I am experiencing an issue where the email proxy terminates after an authentication without any error.

My config (emailproxy.config)

[Server setup]
[IMAP-993]
local_address = 192.168.5.113
server_address = outlook.office365.com
server_port = 993
local_certificate_path = /mnt/data/certificates/live/EXAMPLE.COM/fullchain.pem
local_key_path = /mnt/data/certificates/live/EXAMPLE.COM/privkey.pem

[SMTP-587]
local_address = 192.168.5.113
server_address = smtp.office365.com
server_port = 587
starttls = True
local_certificate_path = /mnt/data/certificates/live/EXAMPLE.COM/fullchain.pem
local_key_path = /mnt/data/certificates/live/EXAMPLE.COM/privkey.pem

[Account setup]
[@]
permission_url = https://login.microsoftonline.com/TENANTID/oauth2/v2.0/authorize
token_url = https://login.microsoftonline.com/TENANTID/oauth2/v2.0/token
oauth2_scope = https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/POP.AccessAsUser.All https://outlook.office365.com/SMTP.Send offline_access
redirect_listen_address = http://localhost:8181
redirect_uri = https://emailproxy.EXAMPLE.COM/
client_id = CLIENTID
client_secret = CLIENTSECRET

[emailproxy]
delete_account_token_on_password_error = True
encrypt_client_secret_on_first_use = False
allow_catch_all_accounts = True

Starting the email proxy with the following command
/usr/bin/python3 -vd /mnt/data/emailproxy/email-oauth2-proxy/emailproxy.py --no-gui --config-file /mnt/data/emailproxy/config/emailproxy.config --cache-store /mnt/data/emailproxy/data/cache.store --local-server-auth --debug

Description
If I set a catch-all domain [@example.com] or [@] and enable allow_catch_all_accounts = True with an already existing authentication it works fine. If I try to add a new one (different email address eg. newaddress@example.com) the listener starts and provides the URL, I can authenticate via oauth successfully but afterwards the email proxy terminates (unfortunately without any issue). The cache store is not updated. There is nothing in the system logs about the termination.

Just to confirm:

• If I use the email proxy with a specific address and domain (eg. [address@example.com]) and set allow_catch_all_accounts = False all works fine as expected. The cache.store is updated. The next authentication works fine.
• Even if I set the email proxy with a specific address and domain (eg. [address@example.com]) and set allow_catch_all_accounts = True all works fine as expected. The cache.store is updated. The next authentication works fine. Finally multiple entries appear in the cache store. In this case I can switch back to [@] until another authentication happens

Assumption
The termination must happen when address [@] and allow_catch_all_accounts = True is used, right after closing local server and returning response

Last log

2023-08-25 10:16:53: Authorisation request received for newaddress@example.com (local server auth mode)
2023-08-25 10:16:53: Email OAuth 2.0 Proxy Local server auth mode: please authorise a request for account newaddress@example.com
2023-08-25 10:16:53: Local server auth mode (localhost:8181): starting server to listen for authentication response
2023-08-25 10:16:53: Please visit the following URL to authenticate account newaddress@example.com: https://login.microsoftonline.com/.........
2023-08-25 10:17:20: Local server auth mode (localhost:8181): received authentication response GET /?code=............. HTTP/1.1 200 381
2023-08-25 10:17:20: Local server auth mode (localhost:8181): closing local server and returning response https://emailproxy.example.com/?code=......
PyThreadState_Clear: warning: thread still has a frame
PyThreadState_Clear: warning: thread still has a frame
#

My system
Email proxy: latest main (Version 2023-08-24 as of writing this)
Debian (latest with all updates)

# python3 --version
Python 3.9.2

# pip list
Package              Version
-------------------- --------------
acme                 1.29.0
beautifulsoup4       4.12.2
certbot              1.29.0
certbot-dns-powerdns 0.2.1
certbot-pdns         1.2.0
certifi              2020.6.20
cffi                 1.15.1
chardet              4.0.0
ConfigArgParse       1.5.3
configobj            5.0.6
cryptography         3.4.8
distro               1.7.0
dns-lexicon          3.5.6
dnspython            2.0.0
filelock             3.11.0
future               0.18.3
gpg                  1.14.0-unknown
gyp                  0.1
httplib2             0.18.1
idna                 2.10
iotop                0.6
josepy               1.13.0
Markdown             3.3.4
mock                 5.0.1
parsedatetime        2.6
pip                  20.3.4
prompt-toolkit       3.0.39
pyasyncore           1.0.2
pycparser            2.21
pycurl               7.43.0.6
Pygments             2.7.1
PyICU                2.5
pyOpenSSL            21.0.0
pyRFC3339            1.1
PySimpleSOAP         1.16.2
python-apt           2.2.1
python-debian        0.1.39
python-debianbts     3.1.0
pytz                 2022.2.1
PyYAML               5.3.1
reportbug            7.10.3+deb11u1
requests             2.25.1
requests-file        1.5.1
requests-toolbelt    0.9.1
setuptools           52.0.0
six                  1.16.0
soupsieve            2.4
tldextract           3.4.0
urllib3              1.26.5
wcwidth              0.2.6
wheel                0.34.2
zope.component       5.0.1
zope.event           4.5.0
zope.hookable        5.1.0
zope.interface       5.4.0

Thanks,
Dirk

[simonrob]

Thanks for the really helpful and detailed report here. This was caused by an errant break rather than continue in 4c5ede3, but is fixed in the latest main (2023-08-25).

[ray-magini]

I have testet it now and it seems to work as expected (no termination, cache store written).

Thanks a lot for the very fast fix! Great work!