incoming TLS/SSL connections

[bmesuere]

Hi Simon,

Thank you for your script. I managed to get everything up and running, but have hit a little snag. Right now, it only allows for unencrypted incoming connections. I want to use your script to allow gmail to use a different smtp server for sending mails. However, gmail only allows one of 3 ports to be used and unencrypted connections are only allowed on port 25. Unfortunately, port forwarding on this port is not allowed by my provider.

The only solution I can think of is adding support for connections over TLS/SSL to the script. Do you think this would be feasible and could you potentially give some pointers on where to start?

Cheers
Bart

[simonrob]

Hi Bart - thanks for the interesting question. This isn't a use-case I'd considered before, but it could be useful to allow Gmail to send through OAuth-authenticated SMTP servers. It's odd that Gmail allows unencrypted connections at all, to be honest, but yes, 25, 465 and 587 are the only ports it permits for linked SMTP accounts.

I'm assuming that you can handle the technical aspects of setting up certificates. Once that is done, it may be as simple as overriding create_socket (for which there is actually an example elsewhere in the proxy, because it is required to be able to handle TLS/SSL connections to email servers) and adding something like ssl.wrap_socket(new_socket, certfile=your_certificate_file, server_side=True).

As a side note, if you're doing this to link an O365 account to Gmail then it may not actually be necessary - while OAuth is supported for SMTP, it is not actually a requirement ("there are no plans to disable Basic Authentication for SMTP AUTH clients at this time").

Let me know how you get on - I'm happy to accept a pull request if you can get this working well.
Simon

[bmesuere]

Hi, thanks for the info. I should be able to figure this out this weekend and will let you know and create a pull request if it works.

Unfortunately, basic auth is disabled on my corporate account which is why I'm looking at alternatives 😉

[simonrob]

I had a very quick look at this just now, and adding the following as a new method in the OAuth2Proxy class seems to work setting up a secure connection for me. I've only tried locally (via openssl s_client -crlf -connect localhost:1993 -quiet), not on a server, but if you could let me know whether this works for you I'll go ahead and integrate it as a configuration option.

def create_socket(self, socket_family=socket.AF_INET, socket_type=socket.SOCK_STREAM):
    new_socket = socket.socket(socket_family, socket_type)
    new_socket.setblocking(False)

    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    ssl_context.load_cert_chain(
        certfile='[ your certificate path ]',
        keyfile='[ your key path ]')
    self.set_socket(ssl_context.wrap_socket(new_socket, server_side=True))

[bmesuere]

Sorry for the delay. I received my new macbook last week and was busy configuring it.

I added the function you proposed in combination with my let's encrypt certificates and this works great with gmail if I select connect over SSL, thanks.

The only thing I'm still trying to figure out is what will happen if the script crashes. I would have hoped gmail would show me an immediate error upon trying to send an email. However, after killing the script manually it still shows the mail was sent. (this if of course beyond the scope of this script)

[bmesuere]

After running this for a few hours, sending an email sometimes fails with the following error:

error: uncaptured python exception, closing channel <__main__.SMTPOAuth2ClientConnection connected 209.85.208.170:35445 at 0x7f12718c8a60> (<class 'ssl.SSLWantReadError'>:The operation did not complete (read) (_ssl.c:2635) [/usr/lib/python3.8/asyncore.py|read|83] [/usr/lib/python3.8/asyncore.py|handle_read_event|420] [emailproxy.py|handle_read|471] [/usr/lib/python3.8/asyncore.py|recv|371] [/usr/lib/python3.8/ssl.py|recv|1226] [/usr/lib/python3.8/ssl.py|read|1101])

[simonrob]

Thanks for the update and confirming that this tweak works. I'll push an update sometime soon with a way to optionally specify certificates in the configuration file.

Regarding the ssl.SSLWantReadError error, is this causing the script to crash or do you just get the occasional failed email on the first attempt? (i.e., does a retry work successfully?). I haven't encountered errors like this before, but it looks like it may need serious refactoring work to move from asyncore to asyncio - see these documentation notes. That's not something that I'm likely to be able to do anytime soon, so hopefully this is just causing an occasional need to re-send on failure. But I'll look into whether there's a way to recover from the error to avoid this.

You asked what happens if the script crashes - in my experience it doesn't crash (though sometimes individual threads fail and another one is created, as in the example above). It's a shame that Gmail doesn't show this correctly as an error, but you're right that this is unfortunately beyond the scope of the script.

[simonrob]

Just to follow up - I've committed in 6bf3b7e a way to specify local certificates/keys in the configuration file, but haven't yet looked at addressing the SSLWantReadError (and presumably also SSLWantWriteError).

One option would be to explicitly handle these when using secure connections. There's an example in this test that is probably fairly easy to adapt. I don't use the proxy in the way you're hoping to, so can't really test this, but I'd happily accept a patch that fixes it.

[simonrob]

I've just updated the buffered-pre-authentication branch with a potential fix for the SSLWantReadError you were encountering. Could you test this with your setup?

[simonrob]

I've been using this with a local certificate for a fortnight or so with no issues, so am going to merge these changes. If there are any problems as a result, please reopen this issue.