Investigate if there are unspecced endpoints that can be removed #8334

clokep · 2020-09-16T20:44:58Z

We should investigate whether there are any unspecced (and unused) endpoints in Synapse.

See #8177 about some of the admin APIs being exposed under /_matrix/client.

I took a look at some of the federation APIs and client APIs also, will put some results in here tomorrow.

#8154 also has some related info.

The text was updated successfully, but these errors were encountered:

clokep · 2020-09-17T12:15:48Z

I collated a bunch of this on a spreadsheet, but a summary is below.

Federation

Unspecced

The various groups endpoints (under /_matrix/federation/v1/groups/), but are to be replaced (I believe).
A room complexity endpoint (/_matrix/federation/unstable/rooms/{roomId}/complexity) was added in Implement the SHHS complexity API #5216.

Errors

/_matrix/federation/v1/3pid/onbind (implemented by On3pidBindServlet) accepts POST, while the specification says it should accept PUT.

Client

Prefix

All endpoints appear under /_matrix/client/unstable/, which is probably OK.
Most endpoints appear under /_matrix/client/r0/, unless they're unstable endpoints.
Some endpoints appear under /_matrix/client/api/v1/.

Unspecced

The various groups endpoints (under /_matrix/client/(unstable|r0)/groups/, /_matrix/client/(unstable|r0)/create_group, under /_matrix/client/(unstable|r0)/publicised_groups, and /_matrix/client/(unstable|r0)/joined_groups), but are to be replaced (I believe).
/_matrix/client/(unstable|api/v1|r0)/pushers/remove (implemented by PushersRemoveRestServlet).
Listing appservices (/_matrix/client/(unstable|api/v1|r0)/directory/list/appservice/{networkId}/{roomId}), added in Extended protocol for content co-creation, beyond chat rooms matrix-spec#379. See Synapse supports unspecced DELETE /directory/list/appservice/<network ID>/<room ID> endpoint #14178
Token refresh (/_matrix/client/(unstable|r0)/tokenrefresh), this endpoint simply raises an error, it can probably be deleted.
Account validity (under /_matrix/client/(unstable|r0)/account_validity/), added in Add time-based account expiration #5027 and Send out emails with links to extend an account's validity period #5047.

Obsolete MSCs

Endpoints from MSCs which are obsolete (not accepted into the specification). Hopefully these can be removed.

Relations and aggregations (under /_matrix/client/unstable/rooms/{roomId}/relations/, /_matrix/client/unstable/rooms/{roomId}/aggregations/, and /_matrix/client/unstable/rooms/{roomId}/send_relation/) from MSC1849: Proposal for m.relates_to aggregations matrix-spec-proposals#1849.
Room keys (under /_matrix/client/(unstable|r0)/room_keys/) from MSC1687: Encrypting recovery keys for online megolm backups matrix-spec-proposals#1687.

Proposed MSCs

Endpoints for MSCs which are proposals and not yet accepted into the specification. Generally these are fine as is.

/_matrix/client/(unstable|r0)/password_policy from MSC2000: Server-side password policies matrix-spec-proposals#2000 -- this should only be appearing under unstable, but also appears under r0.
/_matrix/client/unstable/uk.half-shot.msc2666/user/shared_rooms/{userId} from MSC2666: Get rooms in common with another user matrix-spec-proposals#2666. (This seems fine.)

Merged MSCs

Endpoints for MSCs which were merged, but are not yet part of the specification. These should be unstable only.

/_matrix/client/(unstable|r0)/keys/signatures/upload (implemented by SignaturesUploadServlet) from MSC1756: cross-signing devices using a master identity key matrix-spec-proposals#1756.

Implementation specific

Some endpoints aren't specced and are implementation specific (i.e. there are sent to the client as a redirect or sent to confirm email addresses, etc.). These could potentially be moved under the /_synapse/client/ prefix instead:

/_matrix/client/(unstable|api/v1|r0)/login/cas/ticket
/_matrix/client/unstable/registration/{medium}/submit_token
/_matrix/client/unstable/add_threepid/email/submit_token
- noted on add_threepid email confirmation emails link to /unstable/ namespace #9197
/_matrix/client/unstable/add_threepid/msisdn/submit_token

Errors

/_matrix/client/(unstable|api/v1|r0)/directory/list/room/{roomId} (implemented by ClientDirectoryListServer) supports an unspecced DELETE method.
- Synapse supports unspecced DELETE /directory/list/room/<room ID> endpoint #13111
/_matrix/client/(unstable|r0)/keys/upload/{deviceId} (implemented by KeyUploadServlet) in unspecced (the device ID is treated as optional by Synapse).
There's a few endpoints that accept a txnId parameter at the end which is unspecced. (Search for callers of register_txn_path.) At least one caller of this also requires the txnId and shouldn't be registered without it.

Unimplemented

Specified endpoints that are not implemented in Synapse.

/_matrix/client/r0/account/password/msisdn/requestToken, see spec. The corresponding email version is implemented.
/_matrix/client/r0/rooms/{roomId}/aliases, see spec. This is only implemented as the unstable /_matrix/client/unstable/org\.matrix\.msc2432/rooms/{roomId}/aliases.

Deprecated

The specification has some deprecated endpoints as well. These are generally implemented by Synapse, but unless support for older versions of the spec are dropped, these cannot be dropped (e.g. the initialSync endpoint).

clokep · 2020-09-17T12:16:53Z

I think the next step here would be to:

Identify endpoints that we would like to remove (e.g. they were an experiment and there's no value in them).
Look at the logs from matrix.org and see if they've been used recently.
Decide whether we can remove them. 😄

tulir · 2020-09-17T12:28:41Z

The relation MSC isn't obsolete, it was just split up. The endpoints are now in MSC2675: Serverside aggregations of message relationships matrix-spec-proposals#2675 and partially MSC2674: Event Relationships matrix-spec-proposals#2674. (although the /send_relation endpoint is mostly useless since it does nothing that /send doesn't do)
/aliases is unstable because Synapse doesn't support C-S r0.6.1 yet

clokep · 2020-09-17T12:37:22Z

All endpoints appear under /_matrix/client/unstable/, which is probably OK.

The MSC process actually says the opposite:

Note that unstable namespaces do not automatically inherit endpoints from stable namespaces: for example, the fact that /_matrix/client/r0/sync exists does not imply that /_matrix/client/unstable/com.example/sync exists.

clokep · 2020-09-17T12:40:09Z

The relation MSC isn't obsolete, it was just split up. The endpoints are now in matrix-org/matrix-doc#2675 and partially matrix-org/matrix-doc#2674. (although the /send_relation endpoint is mostly useless since it does nothing that /send doesn't do)

I must have missed that that MSC got replaced! Thanks for pointing this out!

/aliases is unstable because Synapse doesn't support C-S r0.6.1 yet

Fair enough, just documenting it. I thought we had an issue about this, but we don't seem to. I'll file a separate one.

ShadowJonathan · 2020-09-18T09:31:41Z

Should everything used by historical_admin_path_patterns be a part of this? (such as shutdown_room)

clokep · 2020-09-18T11:58:05Z

Should everything used by historical_admin_path_patterns be a part of this? (such as shutdown_room)

As I said in the description, this is tracked in a separate issue:

See #8177 about some of the admin APIs being exposed under /_matrix/client.

clokep · 2020-10-20T18:01:36Z

/_matrix/client/(unstable|api/v1|r0)/pushers/remove (implemented by PushersRemoveRestServlet).

I think this is matrix-org/matrix-doc#1757.

[MSC2432](matrix-org/matrix-spec-proposals#2432) added this endpoint originally but it has since been included in the spec for nearly a year. We can't reasonably remove the unstable endpoint as client implementations still rely on it, and those clients cannot rely on `r0.6.x` appearing in the `/versions` to detect if this stable endpoint exists. It's proposed to introduce a transition period for implementations which may be stuck on the old endpoint of 2 months (minimum). After that point, Synapse can (and should) remove the unstable endpoint without warning. The exact timelines are to be determined by the synapse team, however. This is progress towards #8334

Related: #8334 Deprecated in: #9429 - Synapse 1.28.0 (2021-02-25) `GET /_synapse/admin/v1/users/<user_id>` has no - unit tests - documentation API in v2 is available (#5925 - 12/2019, v1.7.0). API is misleading. It expects `user_id` and returns a list of all users. Signed-off-by: Dirk Klimpel dirk@klimpel.org

richvdh · 2021-04-09T09:53:45Z

Listing appservices (/_matrix/client/(unstable|api/v1|r0)/directory/list/appservice/{networkId}/{roomId}), added in matrix-org/matrix-spec#379.

this is specced as part of the AS API - https://spec.matrix.org/unstable/application-service-api/#put_matrixclientr0directorylistappservicenetworkidroomid - but only in its PUT guise.

[MSC2432](matrix-org/matrix-spec-proposals#2432) added this endpoint originally but it has since been included in the spec for nearly a year. This is progress towards #8334

clokep · 2022-06-06T13:04:37Z

Note that #11584 removed the various groups/communities endpoints.

This increases compatibility with homeservers and allows them to remove Element Android specific workaround. fixes element-hq#4830 see ruma/ruma#936 see matrix-org/synapse#8334 see matrix-org/synapse#9224 Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>

deepbluev7 · 2022-06-30T21:12:27Z

As of element-hq/element-android#6288 Element Android now uses the stable version of /_matrix/client/unstable/org.matrix.msc2432/rooms/:room_id/aliases, so it should be possible to remove that after some buffer period for releases to go out and people to update. At least I believe at least that that was the last user?

richvdh · 2022-07-01T10:27:20Z

I've attempted to update the spreadsheet, though I suspect there are more endpoints which are not listed.

clokep · 2022-07-05T12:16:37Z

I wonder if at this point it would make sense to split this into separate issues for each endpoint?

clokep · 2023-05-31T16:57:39Z

I looked briefly today and the only requests going to matrix.org on /_matrix/client/api/v1/ are some requests to /login with no user agent. I'm inclined to think we can rip that out now.

clokep added the A-Spec-Compliance places where synapse does not conform to the spec label Sep 16, 2020

clokep self-assigned this Sep 16, 2020

clokep mentioned this issue Sep 16, 2020

Use admin_patterns for all admin APIs. #8331

Merged

clokep removed their assignment Sep 17, 2020

aaronraimist mentioned this issue Jan 12, 2021

Deprecate and then remove web_client_location #9078

Closed

turt2live mentioned this issue Jan 26, 2021

r0.6.1 support: /rooms/:roomId/aliases endpoint #9224

Merged

4 tasks

This was referenced Feb 12, 2021

Remove old admin API GET /_synapse/admin/v1/users/<user_id> #9401

Merged

Deprecate old admin API GET /_synapse/admin/v1/users/<user_id> #9429

Merged

richvdh added the Z-Future-Maintenance Things that can't yet be done, but will need cleaning up in a couple of months/releases label Apr 9, 2021

erikjohnston added the T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. label Jul 26, 2021

callahad added this to the Revisit: Quarterly milestone Sep 15, 2021

callahad modified the milestones: Revisit: Next Quarter, Revisit: Next Month Oct 7, 2021

callahad removed this from the Revisit: Next Month milestone Nov 4, 2021

dklimpel mentioned this issue Nov 8, 2021

Remove DELETE /directory/list/room/{roomId} #2953

Open

clokep mentioned this issue Dec 15, 2021

Remove groups/communities support from Synapse #11584

Closed

5 tasks

deepbluev7 mentioned this issue Jan 1, 2022

Use stable version of the room alias endpoints instead of MSC2432 element-hq/element-android#4830

Closed

richvdh added this to the Revisit: Next Month milestone Apr 28, 2022

richvdh removed the Z-Future-Maintenance Things that can't yet be done, but will need cleaning up in a couple of months/releases label May 5, 2022

richvdh removed this from the Revisit: Next Month milestone May 5, 2022

deepbluev7 mentioned this issue Jun 12, 2022

Use stable endpoint for alias management element-hq/element-android#6288

Merged

15 tasks

clokep mentioned this issue Jun 27, 2022

Synapse supports unspecced DELETE /directory/list/room/<room ID> endpoint #13111

Closed

DMRobertson added the Z-Cleanup Things we want to get rid of, but aren't actively causing pain label Sep 16, 2022

DMRobertson mentioned this issue Sep 16, 2022

add_threepid email confirmation emails link to /unstable/ namespace #9197

Open

DMRobertson mentioned this issue Oct 4, 2022

Audit and/or add validation to synapse/rest/client/account_validity.py #13184

Closed

2 tasks

This was referenced May 31, 2023

Remove the unspecced and bugged PUT /knock/{roomIdOrAlias} endpoint #15189

Merged

Remove unspecced and buggy PUT method on the unstable /rooms/<room_id>/batch_send endpoint. #15199

Merged

matrixbot mentioned this issue Dec 21, 2023

Investigate if there are unspecced endpoints that can be removed element-hq/synapse#8334

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Investigate if there are unspecced endpoints that can be removed #8334

Investigate if there are unspecced endpoints that can be removed #8334

clokep commented Sep 16, 2020 •

edited

Loading

clokep commented Sep 17, 2020 •

edited by DMRobertson

Loading

clokep commented Sep 17, 2020

tulir commented Sep 17, 2020 •

edited

Loading

clokep commented Sep 17, 2020

clokep commented Sep 17, 2020

ShadowJonathan commented Sep 18, 2020 •

edited

Loading

clokep commented Sep 18, 2020

clokep commented Oct 20, 2020

richvdh commented Apr 9, 2021

clokep commented Jun 6, 2022

deepbluev7 commented Jun 30, 2022

richvdh commented Jul 1, 2022

clokep commented Jul 5, 2022

clokep commented May 31, 2023

Investigate if there are unspecced endpoints that can be removed #8334

Investigate if there are unspecced endpoints that can be removed #8334

Comments

clokep commented Sep 16, 2020 • edited Loading

clokep commented Sep 17, 2020 • edited by DMRobertson Loading

Federation

Unspecced

Errors

Client

Prefix

Unspecced

Obsolete MSCs

Proposed MSCs

Merged MSCs

Implementation specific

Errors

Unimplemented

Deprecated

clokep commented Sep 17, 2020

tulir commented Sep 17, 2020 • edited Loading

clokep commented Sep 17, 2020

clokep commented Sep 17, 2020

ShadowJonathan commented Sep 18, 2020 • edited Loading

clokep commented Sep 18, 2020

clokep commented Oct 20, 2020

richvdh commented Apr 9, 2021

clokep commented Jun 6, 2022

deepbluev7 commented Jun 30, 2022

richvdh commented Jul 1, 2022

clokep commented Jul 5, 2022

clokep commented May 31, 2023

clokep commented Sep 16, 2020 •

edited

Loading

clokep commented Sep 17, 2020 •

edited by DMRobertson

Loading

tulir commented Sep 17, 2020 •

edited

Loading

ShadowJonathan commented Sep 18, 2020 •

edited

Loading