Add confirmation when redirecting logged-out requests to permalink

[Gargron]

If you open a link in the form of https://mastodon.social/@alice@example.com while logged out, Mastodon will immediately redirect you to https://example.com/@alice. While this is good for many reasons, it also creates potential for phishing, as the user clicking the mastodon.social link may be convinced they are opening a mastodon.social page. We will now show a screen in that scenario that tells you explicitly that you are leaving mastodon.social.

This does not affect logged in browsing, or even normal browsing while logged out, as the redirect only occurs when you open such a page directly, not when navigating to it from the already loaded web interface.

[ClearlyClaire]

I think this is a good change overall, though I'm slightly worried about the friction this may add.

Hitting F5 while logged out on some remote content now presents this page instead of silently redirecting, which is expected. But trying to navigate back stays on the same page (I assume the browser is trying to do history API-based navigation because that's how that particular history state was reached, but this page does not have anything to handle these history API events). This is a confusing regression, but I am not sure how it should be fixed.

[ClearlyClaire]

Another thing I'm noticing is that it breaks the redirect with Accept: application/activity+json queries by showing the confirmation unconditionally.

Hitting F5 while logged out on some remote content now presents this page instead of silently redirecting, which is expected. But trying to navigate back stays on the same page (I assume the browser is trying to do history API-based navigation because that's how that particular history state was reached, but this page does not have anything to handle these history API events). This is a confusing regression, but I am not sure how it should be fixed.

Redirecting to an intermediary path (e.g. /redirect/accounts/:id itself rendering the confirmation) instead of rendering in the application path fixes this.

Pushed commits addressing those issues.

[ClearlyClaire]

A few notes: at @Gargron's request, I merged the two redirect endpoints into one, using a signed query parameter, but I do not like that solution very much because:

• it results in long (potentially too long) query strings
• it uses crypto when not really needed
• it prevents us from knowing anything about the account/post before showing the link, e.g. honoring suspensions

In addition, there is an issue with link previews, in that with the automatic redirection, link previews used to work even when users shared “non-canonical” links to a remote resource, but that won't work with the confirmation page. If we revert to passing post/account ID, we can generate our own OpenGraph tags for that, which I guess would be better, if still not ideal.

[Joshix-1]

we can generate our own OpenGraph tags for that, which I guess would be better

That wouldn't be good. The instance of the original post should be the one to generate OpenGraph. Then they can decide whether they want to allow that. Generating OpenGraph for content from other instances is imho not a good idea.

Maybe the open redirect could stay for certain bot user agents

[ThisIsMissEm]

Might be good to do a comparison of what other link roadblocks like there look like. I know both twitter, linkedin and instagram have these, but I couldn't find any UX analysis on them

[ClearlyClaire]

we can generate our own OpenGraph tags for that, which I guess would be better

That wouldn't be good. The instance of the original post should be the one to generate OpenGraph. Then they can decide whether they want to allow that. Generating OpenGraph for content from other instances is imho not a good idea.

Yeah, you're right, that makes sense. Maybe giving up on OpenGraph in that case and “punishing” users for using the “wrong” URL is the best course of action. Still not really satisfactory given how easy it is to do.

Maybe the open redirect could stay for certain bot user agents

I think this would preclude reverse-proxy caching.

[Joshix-1]

Maybe giving up on OpenGraph in that case and “punishing” users for using the “wrong” URL is the best course of action.

I'd really dislike that. I always thought the current system is really clever.

[ClearlyClaire]

I rebased and force-pushed because a yanked dependency made CI fail. In the meantime, the controller specs had been rewritten as request specs and the rebase uncovered new tests failures. I fixed them.

[codecov]

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (7ecf7f5) 85.08% compared to head (1ece1ed) 85.08%.
Report is 2 commits behind head on main.

Files	Patch %	Lines
app/lib/permalink_redirector.rb	66.66%	6 Missing and 3 partials ⚠️
app/controllers/redirect/base_controller.rb	66.66%	4 Missing ⚠️
app/controllers/redirect/accounts_controller.rb	60.00%	2 Missing ⚠️
app/controllers/redirect/statuses_controller.rb	60.00%	2 Missing ⚠️
...controllers/concerns/web_app_controller_concern.rb	85.71%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #27792    +/-   ##
========================================
  Coverage   85.08%   85.08%            
========================================
  Files        1038     1041     +3     
  Lines       28147    28294   +147     
  Branches     4532     4566    +34     
========================================
+ Hits        23948    24074   +126     
- Misses       3039     3060    +21     
  Partials     1160     1160            

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[renchap]

Looks good to me.

After internal discussions, we chose to go with this solution for now, and not worry about Opengraph data. We will check how it behaves in real world, and if the lack of Opengraph data is really a widespread issue, we will try to implement it.

So far, the 3 alternatives to implement it would be:

• generate it locally. This is not ideal as it might not be the same as the remote data, or the remote server might not want to provide OG for this object
• download and cache the OG data from the remote URL, and serve it. Not ideal as we need to make a synchronous request, and have a list of what meta tags we want to use
• use a double-redirect method to serve an HTTP confirmation page for human users, and a 302 otherwise (to bot users, for example those making OG requests). This would involve a first 302 redirect to an internal endpoint, setting a cookie. Then the second endpoint can check if the cookie is there to see if this is a real user or not. This might not work in real life, and machine users might set the cookie on the redirect, so this would need to be tested further.

[Joshix-1]

Why would you just remove a feature? That could destroy Opengraph for the whole fediverse for a fairly long time.

if the lack of Opengraph data is really a widespread issue

How will you meausure that? Did you measure first if it currently gets used? (You could do that with your big instances)

[renchap]

Why would you just remove a feature?

We have not found a way to have it done easily. I outlined the 3 ways we see to do it above, but they are not without problems and/or add significant complexity.

Maybe we missed an easy way to do it?

That could destroy Opengraph for the whole fediverse for a fairly long time.

Only when people are using non-canonical URLs for sharing

if the lack of Opengraph data is really a widespread issue

How will you meausure that? Did you measure first if it currently gets used? (You could do that with your big instances)

As we do not collect data, we mostly rely on user feedback for this.

[Gargron]

Why would you just remove a feature? That could destroy Opengraph for the whole fediverse for a fairly long time.

For the record, we're not removing anything. We never generated OpenGraph previews for remote content, and none of this affects OpenGraph previews for local content. If people use the "Copy link" option on posts and profiles, there will be no issues when sharing that link on other platforms.

[Joshix-1]

But I see people often copy the url from the browser bar and send that. And thanks to the redirect Open Graph is generated (if the original poster (or the instance) wants that)
That's a great feature

[lapcat]

My web browser extension Homecoming for Mastodon relies on the redirect, so I find this change to be very unfortunate.

It actually has the feel of when Twitter censored Mastodon links.

https://underpassapp.com/homecoming/

[lapcat]

The distributed nature of Mastodon is already difficult for users. This change makes it worse in my opinion. We should be finding ways of making navigation between instances easier not harder.

[lapcat]

As we do not collect data, we mostly rely on user feedback for this.

Was this change very motivated by any user feedback about an alleged phishing issue?

I don't know why we're now teaching users to be distrustful of other Mastodon instances. The redirect page sows distrust, as if visiting other instances is harmful. That seems counterproductive, a self-inflicted wound.

[ClearlyClaire]

My web browser extension Homecoming for Mastodon relies on the redirect, so I find this change to be very unfortunate.

Is there something we can do about that, e.g. a meta tag on the redirect screen making it extra clear what is happening?

I don't know why we're now teaching users to be distrustful of other Mastodon instances. The redirect page sows distrust, as if visiting other instances is harmful. That seems counterproductive, a self-inflicted wound.

Maybe the wording can be changed, but the reason for the confirmation screen is basically that you do not end up on a different instance than yours thinking it is yours. e.g. you were logged in, but for some reason your authentication lapsed, you fail to notice you are redirected somewhere else, and proceed to give your credentials. Or someone crafts a link that purposefully redirects you to something that looks like your instance's log-in screen.

[lapcat]

Is there something we can do about that, e.g. a meta tag on the redirect screen making it extra clear what is happening?

I don't want to get into the business of parsing responses as HTML, but there are a couple of things that would help:

First, if the User-Agent of the request is curl, then restore the old behavior rather than giving the /redirect/ URL. That would fix my extension immediately. (I'm not actually using curl, but I spoof the User-Agent as curl.) The redirect HTML page is for web browsers, not for tools such as curl.

Second, provide a specific HTTP API to get the redirected URL, analogous to the /authorize_interaction?uri= API for example.

Or someone crafts a link that purposefully redirects you to something that looks like your instance's log-in screen.

This seems like a rather obscure and unlikely attack. Are there many malicious Mastodon instances? (I mean malware, not simply hosting abhorrent posts.) I'm assuming that one Mastodon instance won't redirect to another Mastodon instance if the latter is defederated, correct? If my assumption is incorrect, then that might be most of the problem.

Anyway, you'd need a maliciously-crafted Mastodon instance, post a mastodon.myinstance/@foobar link somewhere, somehow find a logged-out user of mastodon.myinstance, hope they're not suspicious of the @foobar path of the URL, induce them to click the URL, and trick them into entering their credentials. Has this ever happened?

[lapcat]

I think the bigger issue here, beyond just my extension, is that 99.99% of the time, when someone posts a mastodon.myinstance link, they're simply copying/sharing it from their own web browser for non-malicious purposes, and this new redirect screen makes Mastodon a lot more unfriendly, especially to people who don't yet use Mastodon and are immediately taught to fear Mastodon links.