oAuth apps missing in databases - Client authentication failed

[hugogameiro]

Steps to reproduce the problem

Attempt to authenticate using oAuth

Expected behaviour

Authorization confirmation page

Actual behaviour

Page with error: Client authentication failed due to unknown client, no client authentication included, or unsupported authentication method.

Detailed description

As mentioned in snarfed/bridgy#1602 (comment), the problem seems to be related to the app being deleted from the oauth_applications database table.

My assumption is that the first time someone on an instance attempts to authenticate with that app, it all works, the app information is stored in the oauth_applications, and the remote developer stores the uid.

Then the user ends up never using that app or removing the authorization from their account "Authorized Apps". This should not delete the app from oauth_applications because the remote developer still believes the uid exists.

Months later, when that user or another user on the instance attempts to authenticate again, the remote app requests to authenticate with the uid they have stored, and the authentication fails with Doorkeeper invalid_client: "Client authentication failed due to unknown client, no client authentication included, or unsupported authentication method".

I have reports of this issue happening with Buffer, Bridgy and elk.zone. I could confirm this issue on a couple of instances that I host and on other instances not hosted by me.

Mastodon instance

Multiple

Mastodon version

v4.2.1

[renchap]

@ClearlyClaire Can this be caused by the new vacuum we added?

[wiegelmann]

Confirmed with the toot.io instance.

[ClearlyClaire]

@ClearlyClaire Can this be caused by the new vacuum we added?

Definitely, this is #24871 at work here. Ideally, apps should gracefully re-register.

[snarfed]

@ClearlyClaire the problem is that apps don't know they need to re-register. The failure mode is, when an app redirects a new user to the instance's /oauth/authorize with one of these deleted client_ids, the instance shows the user this error, and the UX flow stops. The app itself doesn't get an error from anywhere to notify it of the deletion.

[ClearlyClaire]

Hm, this is a good point. I'm not sure what an OAuth app can do, and what we can do to ease that. @ThisIsMissEm, do you perhaps have an idea regarding this?

[ThisIsMissEm]

The solution here would be to attempt to authenticate as the client, and do a GET /api/v1/apps/verify_credentials (docs) call, which would allow you to check the client is still good.

If either fails, then you need to re-register the client (clients can expire with OAuth 2.0, in theory).

So when you store the client, you'd do a last_used_at and if it's greater than a given time period, then re-validate the client credentials.

[ThisIsMissEm]

That said though, if the client is really deleted, then the call to GET /oauth/authorize HTTP/1.1 should fail...

[ThisIsMissEm]

That said though, if the client is really deleted, then the call to GET /oauth/authorize HTTP/1.1 should fail...

Actually, just re-checked the spec, that endpoint works as expected (failure happens after approval/rejection)

[ThisIsMissEm]

@ClearlyClaire I think client's would have to test their client credentials periodically to ensure they're still valid.

Doorkeeper doesn't support RFC 7592 which would allow for a client to read/update/delete it's client registration, with just a "application registration token", which is returned when the client is initially registered.

But OAuth doesn't actually specify behavior here to my knowledge, and in any case, even if Mastodon returned an error immediately on GET /oauth/authorize, that wouldn't help the client, as you'd not be redirected back to the application.

The only way the client has knowledge that it's still usable is to use it's client_id/client_secret to request a client credential grant. We could consider expiring clients after a month, but that would require all clients to reauthenticate after a month, which wouldn't be ideal.

[ThisIsMissEm]

@ClearlyClaire another option we could look at is doing something similar to our AccessTokenTrackingConcern but at OAuth Application level, such that we know when an application's credentials were last used, that way any usage of the OAuth Application would result in it being touched, which would mean that OAuth Applications that have Access Tokens that were deleted would still show up as recently used in the clean up process.

But this wouldn't 100% solve it, and clients should revalidate their client credentials if not used recently.

[snarfed]

/api/v1/apps/verify_credentials makes sense. Unfortunate additional overhead, but definitely not a big deal. Thanks for the idea!

I'm a bit curious about the motivation for garbage collecting OAuth clients, btw. It seems like they wouldn't be a big problem in the common case...were there DoSes where tons of them were registered and filled up db space? Not important, and I'm fine with using verify_credentials, just wondering!

[snarfed]

Oh, also, any chance you all could formalize the garbage collection period or inactivity threshold a bit? Either hard coded or something we could fetch in eg /api/v2/instance? If we're out of sync at all, the silent failure for users is pretty awful UX during the period between when an app gets garbage collected and we later call /api/v1/apps/verify_credentials and determine that we need to recreate our app.