Office 365 - SMTP error "first record does not look like a TLS handshake"

[phillip-white-sociomantic]

Hi All,

I have this issue when trying to test my email setup. I am running v 1.0 in Docker.

Errors:

[2015/10/07 11:24:27 UTC] [DEBG] /api/v1/admin/config
[2015/10/07 11:24:31 UTC] [DEBG] /api/v1/admin/test_email
[2015/10/07 11:24:31 UTC] [EROR] /api/v1/admin/test_email:SendMail code=500 rid=rsqndjxn3f8r7puttryj6epm9y uid=5jtgo5cftbbc7du17n63nznxac ip=192.168.3.68 Failed to open TLS connection [details: tls: first record does not look like a TLS handshake]
[2015/10/07 11:24:32 UTC] [DEBG] /api/v1/admin/test_email
[2015/10/07 11:24:33 UTC] [EROR] /api/v1/admin/test_email:SendMail code=500 rid=ybbi4ppexpggdeupkfhcjbkr1r uid=5jtgo5cftbbc7du17n63nznxac ip=192.168.3.68 Failed to open TLS connection [details: tls: first record does not look like a TLS handshake]
[2015/10/07 11:24:33 UTC] [DEBG] /api/v1/admin/test_email
[2015/10/07 11:24:33 UTC] [EROR] /api/v1/admin/test_email:SendMail code=500 rid=ok85dy6eiffjpqe3so6f6qs6jc uid=5jtgo5cftbbc7du17n63nznxac ip=192.168.3.68 Failed to open TLS connection [details: tls: first record does not look like a TLS handshake]
[2015/10/07 11:24:34 UTC] [DEBG] /api/v1/admin/test_email
[2015/10/07 11:24:34 UTC] [EROR] /api/v1/admin/test_email:SendMail code=500 rid=nmohb5fdzfnjuq8njtq1cz7f5h uid=5jtgo5cftbbc7du17n63nznxac ip=192.168.3.68 Failed to open TLS connection [details: tls: first record does not look like a TLS handshake]

I have configured the settings via the web console, which reflects to config_docker.json:

    "EmailSettings": {
        "EnableSignUpWithEmail": true,
        "SendEmailNotifications": true,
        "RequireEmailVerification": true,
        "FeedbackName": "no-reply",
        "FeedbackEmail": "no-reply@<mydomain>",
        "SMTPUsername": "<user>@<mydomain>",
        "SMTPPassword": "<password>",
        "SMTPServer": "<mysmtpserver>",
        "SMTPPort": "587",
        "ConnectionSecurity": "TLS",
        "InviteSalt": "bjlSR4QqkXFBr7TP4oDzlfZmcNuH9YoS",
        "PasswordResetSalt": "vZ4DcKyVVRlKHHJpexcuXzojkE5PZ5eL",
        "ApplePushServer": "",
        "ApplePushCertPublic": "",
        "ApplePushCertPrivate": ""

It is configured fine with my settings, and i have double checked them.

The errors indicate that it is trying to send from the ip of the computer i am using the web portal from (192.168.3.68).

How do i fix this?

[coreyhulen]

The info at [details: tls: first record does not look like a TLS handshake] is what https://github.com/mattermost/platform/blob/master/utils/mail.go#L46 is reporting. The other info is just our standard logging. Can you maybe try STARTTLS vs TLS ? what email sending service are you using?

The ip address logged is standard (nothing todo with email). It's looks like the ip stuff is just mis-reporting the real ip. We try to take it from the header first via "X-Forwarded-For" or "X-Real-IP" then Remote Address. In the last case this can be your load balancer ip vs the end users ip.

[phillip-white-sociomantic]

Yes I have tried STARTTLS too. The smtp server is smtp.office365.com. It works in Thunderbird and other smtp clients fine. I will post the error of STARTTLS tomorrow.

[phillip-white-sociomantic]

I have these errors when trying with STARTTLS:

2015/10/08 08:27:06 UTC] [DEBG] /api/v1/admin/test_email
[2015/10/08 08:27:20 UTC] [EROR] /api/v1/admin/test_email:SendMail code=500 rid=f6o6up6ebirbun5s7eh89a991c uid=5jtgo5cftbbc7du17n63nznxac ip=192.168.2.112 Failed to authenticate on SMTP server [details: 504 5.7.4 Unrecognized authentication type]
[2015/10/08 08:27:24 UTC] [DEBG] /api/v1/admin/logs
[2015/10/08 08:27:55 UTC] [DEBG] /api/v1/admin/test_email
[2015/10/08 08:28:05 UTC] [EROR] /api/v1/admin/test_email:SendMail code=500 rid=os9anks597gybgq35e5imudx5y uid=5jtgo5cftbbc7du17n63nznxac ip=192.168.2.112 Failed to authenticate on SMTP server [details: 504 5.7.4 Unrecognized authentication type]
[2015/10/08 08:28:08 UTC] [DEBG] /api/v1/admin/logs

I followed the install guide here https://github.com/mattermost/platform/blob/master/doc/install/Docker-Single-Container.md#ubuntu, For Ubuntu 14.04. I also followed the guide here https://github.com/mattermost/platform/blob/master/doc/install/SMTP-Email-Setup.md to enable email. However i do not have "ByPassEmail" in /config_docker.json. My config is:

    "EmailSettings": {
        "EnableSignUpWithEmail": true,
        "SendEmailNotifications": true,
        "RequireEmailVerification": true,
        "FeedbackName": "no-reply",
        "FeedbackEmail": "no-reply@<mydomain>",
        "SMTPUsername": "<user>@<mydomain>",
        "SMTPPassword": "<password>",
        "SMTPServer": "smtp.office365.com",
        "SMTPPort": "587",
        "ConnectionSecurity": "TLS",
        "InviteSalt": "bjlSR4QqkXFBr7TP4oDzlfZmcNuH9YoS",
        "PasswordResetSalt": "vZ4DcKyVVRlKHHJpexcuXzojkE5PZ5eL",
        "ApplePushServer": "",
        "ApplePushCertPublic": "",
        "ApplePushCertPrivate": ""

[phillip-white-sociomantic]

Any help would be appreciated 😄

[coreyhulen]

Hi @phillip-white-sociomantic, sorry I'm at a loss. When I find some time I'll create an O365 account and try it. My guess is there is some low level mis-match between how golang and exchange/windows implement TLS connections.

[coreyhulen]

may be related golang/go#5742 (not sure)

[coreyhulen]

I don't know that much about O365, but as a test you might try to disable TSL 1.1 and TSL 1.2 and only allow TLS 1.0 to see if it works (if that's even possible on O365).

I also saw some references to O365 using port 25 with STARTTLS.

[coreyhulen]

Linking issues. See https://forum.mattermost.org/t/email-authentication-with-v1-0-not-working/314

[phillip-white-sociomantic]

I have tried the smtp service with hotmail. Which is STARTTLS:587. It works with that. It is not working for Office 365 still.

[RobbieMcKinstry]

I have the same problem with a GMail server.

[trashcan]

@phillip-white-sociomantic if you want some help troubleshooting this i'd be happy to help. is there a demo username/password i could test with?

[trashcan]

and fyi based on my experience supporting microsoft email products, they sometimes use nonstandard SSL (where its fully encrypted when you connect on ports you would expect to require STARTTLS and vice versa)

[phillip-white-sociomantic]

I have it working with other MS services. Just Office 365 is an issue. @trashcan I can't give you access to our O365 system, but you can sign up for a free trial on their website.

[coreyhulen]

@phillip-white-sociomantic would you mind sharing a sample of EmailSettings for hotmail? We're trying to collect sample configurations to share with the community.

        "FeedbackName": "",
        "FeedbackEmail": "",
        "SMTPUsername": "",
        "SMTPPassword": "",
        "SMTPServer": "",
        "SMTPPort": "",
        "ConnectionSecurity": "",

[phillip-white-sociomantic]

Sure, my hotmail settings are:

    "EmailSettings": {
        "EnableSignUpWithEmail": true,
        "SendEmailNotifications": true,
        "RequireEmailVerification": true,
        "FeedbackName": "no-reply",
        "FeedbackEmail": "<email>@hotmail.com",
        "SMTPUsername": "<email>@hotmail.com",
        "SMTPPassword": "<password>",
        "SMTPServer": "smtp-mail.outlook.com",
        "SMTPPort": "587",
        "ConnectionSecurity": "STARTTLS",
        "InviteSalt": "bjlSR4QqkXFBr7TP4oDzlfZmcNuH9YoS",
        "PasswordResetSalt": "vZ4DcKyVVRlKHHJpexcuXzojkE5PZ5eL",
        "ApplePushServer": "",
        "ApplePushCertPublic": "",
        "ApplePushCertPrivate": ""

[phillip-white-sociomantic]

Still not working with Office 365. The same settings and credentials work on our printer smtp client, and in thunderbird. So i presume the mattermost config is missing something for office 365. I have setup a test o365 account to test this. It's only valid for 30 days so hopefully we can find the issue within that time.

user / email: mattermost@mattermosttest.onmicrosoft.com
password: MMtest2015
webmail login: https://outlook.office365.com
Server name: smtp.office365.com
Port: 587
Encryption method: TLS or STARTTLS (depends on client)

[coreyhulen]

Thanks. I've included the hotmail settings with https://github.com/mattermost/platform/blob/hotmail-sample/doc/install/SMTP-Email-Setup.md#hotmail

Also thanks for setting up the test account. I'll try to make some time to test with O365.

[it33]

Hi @phillip-white-sociomantic circling back on this issue. Would you be open to starting a Troubleshooting discussion on this?

There might be folks on the general forum who could help.

Adding @esethna to help monitor where we are on this.

[phillip-white-sociomantic]

@it33 Thanks for the feedback. There is an open discussion already https://forum.mattermost.org/t/email-authentication-with-v1-0-not-working/314.

I also tried this again with v1.2.1. It is the same issue with O365.

[it33]

Thanks @phillip-white-sociomantic, posted there to see if there's anyone who can help,

[eungjun-yi]

I have the same problem with gmail. Here is my email settings:

    "EmailSettings": {
        "EnableSignUpWithEmail": true,
        "SendEmailNotifications": true,
        "RequireEmailVerification": false,
        "FeedbackName": "",
        "FeedbackEmail": "",
        "SMTPUsername": "myname@gmail.com",
        "SMTPPassword": "<password>",
        "SMTPServer": "smtp.gmail.com",
        "SMTPPort": "587",
        "ConnectionSecurity": "TLS",
        "InviteSalt": "<salt>",
        "PasswordResetSalt": "<salt2>",
        "ApplePushServer": "",
        "ApplePushCertPublic": "",
        "ApplePushCertPrivate": ""
    },

[eungjun-yi]

I have changed ConnectionSecurity from TLS to STARTTLS then it works well.

[it33]

@phillip-white-sociomantic could you let us know if the above is able to solve your issue?

[phillip-white-sociomantic]

Unfortunately not. It was the same problem with v1.2.1

[seanws]

Hi @phillip-white-sociomantic. I work at Microsoft. Hopefully I can help a bit. Microsoft servers (i.e. Office 365/Exchange Online) use explicit TLS for authentication. That's also called STARTTLS and you can identify it as the endpoint requires port 587 (as @coreyhulen mentioned, we also accept port 25 for our smtp.office365.com endpoint). Here, you have to explicitly send a STARTTLS command to negotiate TLS after starting with SMTP. Implicit TLS is where you negotiate TLS at the beginning of the connection and then proceed with SMTP. This is called SSL/TLS or TLS depending on the client. This is normally done on port 443.

An SMTP error like "first record does not look like a TLS handshake" points to trying to use implicit TLS on an endpoint only supporting explicit TLS. As I mentioned, we only support STARTTLS for SMTP connections.

Moving on the error you see with STARTTLS about authentication. 504 5.7.4 Unrecognized authentication type occurs when you try to send an unsupported AUTH command. The client should be sending AUTH LOGIN which is standard and used by every mail service and email client.
Below is an example of the error if I try to send AUTH PLAIN when connecting to smtp.office365.com
250-CO1PR06CA048.outlook.office365.com Hello [167.220.2.190]
250-SIZE 157286400
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH LOGIN
250-8BITMIME
250-BINARYMIME
250 CHUNKING
AUTH PLAIN
504 5.7.4 Unrecognized authentication type

Hope this helps.
Sean

[it33]

Hi @seanws,

Would you know where to report a bug in Office365?

The Go library we're using is reporting low level errors in the protocol, it works for all other services we use, only Office365 is having the error.

[seanws]

@it33 You will need to create a service request under the Support tab in the Office 365 Admin Center.

[trashcan]

I looked into this with Corey, and the problem is this error when using smtp.office365.com:587 and STARTTLS:

504 5.7.4 Unrecognized authentication type

This is because AUTH PLAIN does not appear to be supported in this situation.

⋊> ~ openssl s_client -quiet -connect smtp.office365.com:587 -crlf -starttls smtp
depth=1 /C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/OU=Microsoft IT/CN=Microsoft IT SSL SHA2
verify error:num=20:unable to get local issuer certificate
verify return:0
250 CHUNKING
EHLO pat
250-BY2PR21CA0015.outlook.office365.com Hello [70.112.14.174]
250-SIZE 157286400
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH LOGIN
250-8BITMIME
250-BINARYMIME
250 CHUNKING

^ Notice that only AUTH LOGIN is advertised. Go's "net/smtp" doesn't support AUTH LOGIN, but there is some code linked here that does:

golang/go#9899

So ideally you would parse the EHLO response, determine what types of AUTH are allowed, and then use that to determine which AUTH to use in https://github.com/mattermost/platform/blob/master/utils/mail.go#L49

(Sorry for the wall of text)

[it33]

Thanks @trashcan!!

High appreciate your figuring this out. Opening an APR ticket for anyone in the community who would like to help solve the Office 365 SMTP issue

So glad we got to a solution :)

[benwilde]

I was getting the same error but realised I had 2FA setup for my gmail account, created a test gmail account and it works fine. Just in case others are trawling for resolutions on this discussion...

[frol]

I am a bit lost here. I still see "Failed to open TLS connection [details: tls: first record does not look like a TLS handshake]" when I try to use smtp.office365.com:587 TLS. I am using Mattermost 3.7.3.

Is there a workaround of some kind?

[abunet]

We are receiving the same error configuring Mattermost 3.8.2 with Office 365
using STARTTLS and port 587 with a test gmail account does work.

[frol]

JFYI, I have worked around the issue with SMTP Relay server: https://hub.docker.com/r/turgon37/smtp-relay/

[rorymbyrne]

Hi,

Am having the same problem with Office 365 integration with Mattermost. Has anyone any suggested simple work around?

[kafferejsarn]

I have the same problem. This is unfortunately a stopper for us to deploy this tool in a production environment in out company.
Anyone made it work with Office365 email account?

[lfbrock]

@kafferejsarn - we just added support for LOGIN method authentication, which may help with the issue: #8140

The change should ship in April, we haven't tested with Office365 yet but are working on it. If you'd also like to help test it out with Office365 that would be great!

[gerbsen]

Why is this issue closed? This is still not working for 4.8.1 bundled with GitLab. What can I do?