Magic links don't work when Outlook "safe links" are enabled

[eirikur-grid]

Magic links don't work when Outlook "safe links" are enabled

Description

Outlook will rewrite email links to direct them towards https://*.safelinks.protection.outlook.com/ with the original url encoded as a query parameter, when a feature named "safe links" is enabled. In our testing, this appears to break magic links.

Using the standard email templates, we got a rather cryptic error message, stating that the redirect_uri was missing. By tweaking the email template to include the client_id and redirect_uri, we get a different error, stating that the magic link is invalid or expired.

Steps to reproduce

Steps to reproduce the behavior:

1. Create an email account on outlook.com
2. Register a user with the @outlook.com (or @hotmail.com) email address
3. Ask for a magic link to be delivered to the outlook email address
4. Click the link

Expected behavior

The magic link works and the user is authenticated

Screenshots

Platform

(Please complete the following information)

• Device: MacBook Pro
• OS: macos
• Browser: Chrome 81
• FusionAuth version: 1.15.5

Additional context

The "safe-links" feature can be turned off (see the last screenshot). If we do, then the magic links work as expected.

We have two hypothesis as to what might be causing this:
a) The passwordless code is somehow distorted as it is URL path encoded and decoded by the safe-links mechanism.
b) The safe-links mechanism makes a GET request to the magic link, thus using the code and making it invalid for future requests.

[robotdan]

Interesting, your b) hypothesis seems plausible.

Reading through this: https://docs.microsoft.com/en-us/microsoft-365/security/office-365-security/how-atp-safe-links-works?view=o365-worldwide

It isn't clear if they are making a GET request to the URL ahead of time, but it seems likely they would have to do that in order to perform necessary checks.

One way to confirm this - in our updated templates, we are using a two prong approach to preserve the necessary information to complete login. The first is that we save off all of the state and then look it up with the passwordless code at the time of use. This is happening today, and if the code is used, and then on a second request the code is invalid we cannot lookup the state to complete login, hence the error you're seeing about an invalid redirect_uri (I think).

In the new templates, we are jamming all of the values on the URL as well so that if the code is expired, we can in most cases allow you to restart the workflow.

If you want to update your template to add these additional parameters as outlined in the stock template example and then retry, I would expect it to fail as it does now, but with a different error. Such as 'your code has expired'.

https://fusionauth.io/docs/v1/tech/email-templates/email-templates#passwordless-login

[steinnes]

We did some tests and noticed a preceding HTTP HEAD request with a User-Agent of Go-http-client/1.1, which we believe to be the safelinks service.

Does the OTP get invalidated by a HEAD request?

[eirikur-grid]

If you want to update your template to add these additional parameters as outlined in the stock template example and then retry, I would expect it to fail as it does now, but with a different error. Such as 'your code has expired'.

We can confirm that we get a "Your link has expired or is invalid" message after having updated the email template.

[robotdan]

@steinnes I'd have to look at the w3 spec to see if we are doing this correctly, but in our MVC we treat a HEAD request just like a GET and then only stop short by not writing the response body after we've written the response headers.

So in this particular case if a HEAD request is being made with the URL, this indeed will "use" the code and then it will be invalid when subsequent GET request is made.

@eirikur-grid great, that confirms that we do put up the "correct" message thinking that the code has already been used.

I'll have to think about this more, I don't know if there is anything we can do about this or not. I suppose in theory we could analyze the User-Agent string when we see a HEAD request and then no-op the request. I don't know what the security risks of that would be, the User-Agent string is not secure, it can be set or modified easily. But if we just no-op the request it may be safe.

I wonder how many other services this type of "safe link" feature breaks.

[robotdan]

We could re-work this request to use a POST binding of sorts. In this scenario we'd respond to GET and only render a minimal form that we would then submit via JavaScript as a POST request to complete the action.

<script>
window.onload = function () { document.forms[0].submit(); }
</script>
<form method="POST" action="/oauth2/passwordless">
  <input type="hidden" name="code" />
</form>

I think this would make us immune to strategies such as the SafeLink feature.

[fdlk]

Same thing happens for Email Verification

[eirikur-grid]

@robotdan The solution method that you propose looks good to me

If there's any chance this could be prioritized, then that would be appreciated. We do not feel comfortable with enabling magic links for our users while this issue persists.

[robotdan]

I do want to get to this one, we have several large projects underway at the moment.

[shortstack]

+1 for this!

it keeps us from being able to use passwordless logins for a large percentage of clients :(

[Betree]

Hi! We're not using FusionAuth, but I stumbled upon this issue because we encountered the same problem with magic links generated for opencollective.com and Outlook's ATP feature. The tricky part is that their bot actually runs the Javascript on your page, so it triggers the requests just like a regular user would.

We deployed an attempt at fixing it last week and two users just confirmed that it solved the bug for them. If you're interested in the how, feel free to take what you want from opencollective/opencollective-frontend#5476. It's not perfect, and future changes on Outlook's bots could break it, but it seems to do the job as a hot fix.

[robotdan]

Thanks @Betree - much appreciated.

[robotdan]

From what @Betree has suggested, the possible solution outlined here #629 (comment) may not work.

It seems to me that if Outlook is going to add a feature like this, they should be the ones to document a solution that they will support so that we don't just hack something that may or may not work in the future. I'm an idealist. 😀

If anyone on this thread has a support contract with Microsoft, I would love to hear what they have to say about how they intend this to work with the "normal" world. Perhaps they have some clever suggestion that we haven't thought of yet.

[steinnes]

We've encountered the same issue (Outlook sending a HEAD request) when verifying e-mails as well. The e-mail does get verified by the HEAD request and the verificationId is invalidated so the user sees an error page (while in fact their operation has succeeded).

I think Outlook should provide information on why they do this, but I also think as a general rule systems should not modify resources due to a HEAD request. What exactly they should do is a different question though :-(

[eirikur-grid]

@robotdan Don't have a support contract with Microsoft, but perhaps their identity specialists might be sympathetic. I'm thinking of Christos Matskas @cmatskas (https://twitter.com/christosmatskas) and John Patrick Dandison @jpda (https://twitter.com/azureandchill) in particular, after having heard them on one of my favorite podcasts the other day.

[robotdan]

If all they are doing is making a HEAD request, we should be able to handle that ok. I can take a look at what we are doing in the case of a HEAD request, perhaps we can solve some of the use cases by ensuring we are at least not using up the id on this request.

From some of the comments it sounds like they actually make full GET requests and actually execute JavaScript returned on a GET. If this is the case, it will be much trickier to solve I think.

[jpda]

Thanks @eirikur-grid - we don't work with the teams that own the safelinks product but will be happy to ask around internally to see what we can find. I have this same problem with lastpass emails.

Safelinks for office 365 can be turned off or ignore certain urls through an allow list, but that is for an organization administrator (consumer accounts can turn off safelinks entirely), although that's not a very scalable solution if you're building saas to ask your customers to do more than use your product.

My personal opinion is there may be workarounds for now - like the OTP being burned only when submitted via form post (not with a form.submit() but via submit click), as I don't believe safelinks will submit any forms. Something like a "confirm sign-in" button or similar, rather than marking the OTP as used on any GET. I haven't thought through all of the security issues with allowing the OTP GET link to be used multiple times (as I am not a security professional), but if access to the protected resources is conferred at the same point as when the OTP is burned (e.g., on a form submission), and the OTP is time-bound and truly one-time-use, I don't see how that would be meaningfully less secure than burning the OTP on a GET.

[robotdan]

Thanks for the suggestion @jpda - this may be the better route for us to take - to simply prompt the user to click "validate".

[matt-allan]

I don't use FusionAuth but I found this issue because I had a similar problem. Ignoring HEAD requests fixes the issue for Outlook safe links. However, there are lots of other email security products that do similar things, and some of those make GET requests, not HEAD requests. IIRC Mimecast is one of those.

[robotdan]

Thanks @matt-allan - that is ver helpful.

If anyone has a HAR file from using Outlook Safe Links, that would be helpful for us to confirm how Outlook is making these requests.

[robotdan]

Has anyone been able to verify this now works with Outlook Safe Links?

[mooreds]

We had another user (@wodka ) mention that he was having an issue with Proofpoint AV software and passwordless logins.

[robotdan]

We had another user (@wodka ) mention that he was having an issue with Proofpoint AV software and passwordless logins.

Is the user on the latest version, indicating our solution didn't work for Proofpoint AV?

[wodka]

we are running on "1.25.0" - but will update next week also to the latest version (a bit tricker for us as we use cockroach as underlying db)

[robotdan]

@wodka interesting, never heard of cockroach db. Looks to be wire compatible with PostgreSQL.

When you say it is a bit trickier - does this mean you have to modify migration scripts or something as it is not 100% SQL compatible?

I like their business model, looks like they have an on-premise free version, very similar to ours.

[mooreds]

@robotdan I believe @wodka wrote these forum posts: https://fusionauth.io/community/forum/topic/950/cockroach-compatibility-problem-on-connector-signin which document some of the adventures of running FusionAuth on a PostgreSQL compatible datastore.

[oxyuranus-scutellatus]

We came across exactly the problem described by @eirikur-grid. In general, it can be said that the upgrade to FushionAuth 1.27.2 apparently solves the problem, but creates a new one: With 1.24.0 the link was valid exactly once.
But if now with version 1.27.2 the link is clicked a second time, the browser gets into an endless loop instead of being considered invalid:

• GET http://localhost:9011/email/verify/bCfV2Cz8dEJ1Op3KVy0qxFDB5u4-qc14GLLYSEB-ERI?client_id=e5b44aaa-e458-4409-9a14-8079fcf793df&postMethod=true&tenantId=30466e99-2809-11eb-80e9-0242ac1b1000
• GET http://localhost:9011/js/prime-min-1.5.3.js?version=1.27.2
• GET http://localhost:9011/js/oauth2/LocaleSelect.js?version=1.27.2
• POST http://localhost:9011/email/verify
  This repeats ad infinitum!

[mooreds]

@oxyuranus-scutellatus

Oof, that's no good. Can you please open a new tracking issue for what you see (including as many details and reproduction steps as you can. A HAR file would be great too: https://toolbox.googleapps.com/apps/har_analyzer/ )?

https://github.com/fusionauth/fusionauth-issues/issues

[oxyuranus-scutellatus]

@mooreds: I personally interpreted "Oof, that's no good." as a sign of going ahead. But now, not much happening. Did I miss anything?

[mooreds]

Thank you for filing the tracking issue, we really appreciate the repro steps. I don't have any guideline on when we can look at this particular issue, but can let you know we regularly review github issues to determine which bugs the engineering team will work on. I can tell you that we are a relatively small team juggling a lot of different efforts, however.

Here's our general guidance on our roadmap: https://fusionauth.io/docs/v1/tech/core-concepts/roadmap/

[MicahParks]

The magiclinksdev project had this same issue with security products consuming magic links. I thought I'd leave a note here for anyone looking for another solutions.

The merged solution was to render an HTML page with reCAPTCHA v3 and a "bypass button". This solution does require rendering a frontend page and a user following the magic link may briefly experience the page before being automatically redirected.

On page load, as soon as reCAPTCHA v3 is ready, it will HTTP POST to the backend with its token. After the backend verifies the token, the magic link target URL would be in the response body. The frontend then redirects with window.location.replace. Using this does not allow the user to navigate back to the magic link page using the browser's back button.

The "bypass button" is used in case the reCAPTCHA v3 verification fails or the user's web browser is not running JavaScript. It is an HTML <form> element with one <button> where the type attribute is set to "submit" and the action attribute is set to the served URL plus an additional query parameter. The backend templates the current URL value and new query parameter. When the HTML <form> is submitted, the backend will respond with an HTTP redirect, so no JavaScript is required. The "bypass button" is disabled by default. I think there's a low chance of a security product submitting an HTML form when detonating a link, but it's possible.

I would much rather prefer a backend only solution to preventing security products from consuming magic links, but due to the diversity and evolution of security products, it doesn't seem like that's currently possible. I think the reCAPTCHA v3 solution we've implemented should be more than enough to stop email scanners from consuming magic links for the foreseeable future.

Here are the links to the issue and the pull request, if anyone wanted to view the code.
Hope this helps someone else out 🙂

[nizarmah]

I am dealing with this problem on a personal project.

I love you folks. This thread saved me hours.

[mooreds]

@nizarmah in that case, you might be interested in this as well: #2443

[witto13]

Hello everyone. Yesterday we had the big launch of our product and sadly some of our users said that the login magiclink didn't work. When they open it it was expired already. We don't want to ask our users to deactivate safe links in outlook so we are looking for some solutions. The solutions that you mentioned were implemented with fusion auth, but we are working with nextauth. Has anybody have a solution to this with the nextauth system?

[mooreds]

@witto13 move to FusionAuth https://next-auth.js.org/providers/fusionauth :) .

But seriously, I'd suggest opening a support request in the next auth repo or asking in their community. You're more likely to get some good answers regarding nextauth options there.