MSC3723: Federation /versions

[ShadowJonathan]

Rendered

Implementations;

• Add federation /versions endpoint (MSC3723) synapse#11978
• MSC3723: Add federation versions endpoint ruma/ruma#895

Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>

Fixes matrix-org/matrix-spec#267

Preview: https://pr3723--matrix-org-previews.netlify.app

[ShadowJonathan]

As I wrote this, I realised that this is effectively trying to mimic what the pre-global-versioning "server-server" spec tried to do in spirit.

I'm personally convinced that disjointing the version range that a federation-side of a server exposes makes sense, as federation could become much more unstable if changes happen quickly, as it could fragment that federation.

(If the SCT or experienced community members think this is not a good idea (possibly for much the same reasons the global version MSC was instigated in the first place), i'm 100% fine with requiring this to be matched to exactly the same range /client/versions would respond with this.)

[turt2live]

it's still an http hit though?

[ShadowJonathan]

Yes, but it's not aiming to eliminate that (though caching helps), it tries to help future MSCs in not having to describe version-incompatibility-related fallback cases. ("If this endpoint returns 404, then assume this server does not understand the endpoint, and fall back to this older endpoint")

With /versions, the server instead can reliably understand what endpoints the other server does support, and model its subsequent server behaviour just off of that, this is entirely analogous to client's /versions in that sense. It could help reduce a lot of spec cruft, and help reduce implementation and S2S specification complexity.

[turt2live]

It doesn't help unstable implementations though as this MSC doesn't carry over the unstable_features description. Even if it did though, I don't really see the benefit of using /versions over 404 semantics.

It's the same number of words and same amount of resources to do either.

[ShadowJonathan]

I think this variant would (in the long term) be more stable, clokep mentioned in #spec that unknown endpoints don't really have a consistent error value. Plus, an unknown-endpoint-404 can be confused with a resource-not-found-404. I think every endpoint soft-deprecated that way would then have their 404-semantics have to be dealt with individually and separately. They could differ per implementation, which could lead to more implementation and spec complexity to move the spec forward on federation.

If unstable_features does fit this MSC though, I can add it. (Initially I didn't, because I thought that'd be stretching the initial usability of this endpoint, though now im convinced otherwise)

It's the same number of words and same amount of resources to do either.

(Maintenance) resources might, but eventual implementation complexity most likely not.

Even if, I personally believe that dealing with version incompatibilities between different components of the matrix ecosystem, as a fact, shouldn't have to be done by probing the other component, but by mutually agreeing on a set of supported versions within a range, it's a standard and method that's time-tested (with things such as version and cipher-suite selection in SSL/TLS).

[ShadowJonathan]

My main concern is the ramping requirement of implementation complexity, the federation API today is fairly simple, and I agree that (now) it's easy to implement fallbacks, but that's likely not possible once we get a sufficiently enough complex federation API where some kind of compatibility needs to be noted up-front, with things such as partial mesh federation.

I'll ask about endpoint probing in #spec later, I think that there's some bits I'm missing, I'm most likely overestimating the issue, but I'm also trying to project all those issues into the future, where they may turn from mere paper cuts to spec cruft and overdue technical debt.

[clokep]

I suspect it might be simpler to spec what an unknown endpoint is supposed to return. The implication of this endpoint is that a sever needs to use it for every other server and then cache that info for an undefined period of time.

As @turt2live already mentioned it doesn't help with experimental stuff so we kind of need to spec that anyway. I'd be in favor of doing that instead.

[turt2live]

To summarize my thoughts: I think this MSC is probably fine, but it doesn't really seem to have a use-case at the moment, so might be best for the backburner while we wait for another MSC which really needs it to help push it forward.

I'm not opposed to the idea of /versions on all the API surfaces, just questioning how practical/fulfilling it would be for a change like this to land today rather than later.

[ShadowJonathan]

I'll add in the unstable_features object as well, it seems it's better to add here than to not.

Edit: Added that.

[clokep]

I suspect it might be simpler to spec what an unknown endpoint is supposed to return.

To cross-reference, this is essentially #1492.

[ShadowJonathan]

To cross-reference, this is essentially #1492.

For future reference, this is now matrix-org/matrix-spec#339

[turt2live]

(keeping needs-implementation label until an implementation which demonstrates how this is needed is added - at the moment the current implementations add the endpoint, but the use case isn't really there)

[neilalexander]

While I can somewhat see the benefit of unstable_features somewhat, I am totally unconvinced that this MSC actually solves a real problem otherwise.

At best it provides some mechanism for two homeservers to identify that they are trying to talk two different versions of the spec, but without a solution as to how to deal with that case. Unless we're either going to set the expectation that servers have to maintain legacy support for possibly deprecated endpoints, parameters or fields, or we're going to simply refuse to talk to servers that claim to support a version that we don't, what good does it do to know either way?

Compliance with a specific API version is incredibly difficult to prove and a server can simply lie about it, like how Conduit outright lied to the federation about which room versions it supports or how Dendrite claimed the wrong CS API versions for a while and no one noticed, which leaves us in a situation that's exactly the same as today: the only real option you have is to just try hitting the endpoint anyway and see what happens.