[feature] Mastodon-compatible API extensions discovery mechanism

[VyrCossont]

Is your feature request related to a problem ?

Inspired by discussion on #1880:

I'd love to support per-post federated/boostable/replyable/likeable (and maybe a future discoverable, indexable, or searchable?) in the Feditext client. I could do this today, because they're just extra boolean parameters that only GtS has, and we know how to version-sniff GtS already.

But… going cross-platform gets interesting. Let's take federated as an example. GtS has it, Glitch doesn't support local-only posts except through the magic emoji scheme that #1880 proposed we adopt, and Hometown has a local_only bool param that's the opposite sense of federated, and it's not possible to know that any of these are available except through version sniffing.

Describe the solution you'd like.

extensions

I wonder if it would make sense to start declaring versioned extensions to the Mastodon-ish API:

"configuration": {
  "extensions": {
    "gotosocial:visibility:v1": {}
  }
}

If you see that gotosocial:visibility:v1 extension in an instance API response, you know you have the optional federated, likeable, replyable, and boostable extra status params with bool values and nothing else, and can light up client UI for them. (This extension has no config parameters and is thus associated with a {} value in the extensions map.)

Extensions that GtS supports can be typed and documented as fields on the InstanceV1Configuration and InstanceV2Configuration structs, and we can annotate which API endpoints are affected by a given extension using OpenAPI operation tags, which we're already using. Extensions should also be documented with an overview readme on a https://docs.gotosocial.org/en/latest/api/extensions/ docs page (which doesn't exist yet).

Other instance servers can implement gotosocial:visibility:v1, and provided they stick to the same meaning as GtS, any client that supports gotosocial:visibility:v1 will work. And sticking to the same meaning should be easier for other projects implementing GtS extensions because we have nice docs.

versioning

Want to add a looking_for_a_fight visibility option?

"configuration": {
  "extensions": {
    "gotosocial:visibility:v1": {},
    "gotosocial:visibility:v2": {}
  }
}

v2 adds that new option. Meanwhile, you're still compatible with clients that only know v1.

other implementations

This is the approach Pleroma and Akkoma take, more or less. Their nodeinfo API has a metadata field (also available from the instance API as pleroma.metadata) with a features list:

"metadata": {
  "features": [
    "pleroma_api",
    "akkoma_api",
    "mastodon_api",
    "mastodon_api_streaming",
    "polls",
    "v2_suggestions",
    "pleroma_explicit_addressing",
    "shareable_emoji_packs",
    "multifetch",
    "pleroma:api/v1/notifications:include_types_filter",
    "quote_posting",
    "editing",
    "bubble_timeline",
    "pleroma_emoji_reactions",
    "exposable_reactions",
    "profile_directory",
    "akkoma:machine_translation",
    "custom_emoji_reactions",
    "pleroma:get:main/ostatus"
  ]
}

Describe alternatives you've considered.

extended configuration

One route we could go for this specific case is extending /api/vX/instance to return additional fields in configuration.statuses dedicated to visibility options, similiarly to the way I got GtS (#1363) and Glitch (glitch-soc/mastodon#2091) to add one for supported MIME types, and the proposed way Mastodon will be reporting supported posting languages (mastodon/mastodon#24443):

"configuration": {
  "statuses": {
    "max_characters": 5000,
    "max_media_attachments": 6,
    "characters_reserved_per_url": 25,
    "supported_mime_types": [
      "text/plain",
      "text/markdown"
    ],
    "supported_visibility_options": {
      "federated": [false, true],
      "boostable": [false, true],
      "replyable": [false, true],
      "likeable": [false, true],
      "searchable": ["everyone", "mufos", "no_one"],
      "quotable": ["everyone", "mufos", "no_one"]
    }
  }
}

But this only works for simple bool/enum params for post visibility, so it's kind of overengineered in the wrong direction. As a client dev, you still need to know what supported_visibility_options means, you still need to know that they map to extra params that you need to add to your client's new-status model, you still need to build UI and localized names for any params and values your client intends to support. Same way that supported_mime_types still requires you to know that it maps to a content_type param on the client.

Additional context.

no but for real Vyr how do we get other projects to care about this

I don't know. It might be an easy sell for Akkoma specifically, since they can trivially translate their features list into an extensions map. But from the little experience I have trying to get useful features into Mastodon, Pleroma, Glitch, and now GtS, if someone manages to get a feature into one instance server project, it's easier to point to that one and say "hey those folks already have this".

Client devs, I think, will welcome a formal extension mechanism, because it's easier than version sniffing and doesn't break when someone decides to customize their software name or version or do an outright fork. Certainly I'm on board. The TootSDK people might be worth talking to as well — I know their flavor detection code involves a lot of version checks.

[VyrCossont]

Notes on one other implementation: the Fedibird fork of Mastodon also has to deal with this general problem, and they've done it with a mix of configuration.statuses extensions, feature flags like feature_quote, and the fedibird_capabilities list:

{
  "configuration": {
    "statuses": {
      "max_characters": 560,
      "max_media_attachments": 4,
      "characters_reserved_per_url": 23,
      "min_expiration": 60,
      "max_expiration": 37152000,
      "supported_expires_actions": [
        "delete",
        "mark"
      ]
    },
    "…": {}
  },
  "feature_quote": true,
  "fedibird_capabilities": [
    "favourite_hashtag",
    "favourite_domain",
    "favourite_list",
    "status_expire",
    "follow_no_delivery",
    "follow_hashtag",
    "subscribe_account",
    "subscribe_domain",
    "subscribe_keyword",
    "timeline_home_visibility",
    "timeline_no_local",
    "timeline_domain",
    "timeline_group",
    "timeline_group_directory",
    "visibility_mutual",
    "visibility_limited",
    "visibility_personal",
    "emoji_reaction",
    "misskey_birthday",
    "misskey_location",
    "status_reference",
    "searchability",
    "status_compact_mode",
    "account_conversations",
    "enable_wide_emoji",
    "enable_wide_emoji_reaction",
    "timeline_bookmark_media_option",
    "timeline_favourite_media_option",
    "timeline_emoji_reaction_media_option",
    "timeline_personal_media_option",
    "bulk_get_api_for_accounts",
    "bulk_get_api_for_statuses",
    "sorted_custom_emojis"
  ],
  "…": {}
}

[VyrCossont]

Further prior art: POP3 and IMAP have a CAPABILITIES command, IRC has the RPL_ISUPPORT reply and IRCv3 CAP command, SFTP has its extensions mechanism, and LDAP has the supportedFeatures, supportedControl, and supportedExtension operational attributes, and they all boil down to "here is a list of strings".

(I tried looking for info on how XMPP and Matrix handle extensions discovery but ended up with a headache.)

[noerw]

Wow, thanks for doing this reasearch!

I currently don't have the capacity to iterate deeper on this, so for now I'll just elaborate on the idea I had in mind when I wrote this:

In the long run, clients should have a proper GUI for setting flags like federated / boostable / ... - ie servers would expose a well specified™ interface to retrieve post options & how they map to API parameters. This will take a collaborative effort from at least 1-2 client devs, and 1-2 server projects - last i checked (6 months ago), nobody worked in that direction.
Implementing the :local_only: emoji seems like a good stopgap to enable posting things that you don't want to be sent all over the place.

While compatibility with an existing capabilities interface is great, I'm not a big fan of the indirection of hiding capabilities (or even a group of them) behind an alias name. What I had in mind was providing some JSON-schema (or similar), which for each custom parameter (or actually better, all parameters) defines type, enums, GUI display name, as well as the mapping to an API parameter (URI of the API endpoint this relates to, parameter name). Maybe a spec like OpenAPI (just for POST status endpoint, or even for the full API) is well suited for this.
This meta-feature could then again be listed in one of the existing capability-name lists as implemented by other server projects.

• This way, custom capabilities could be directly mapped for all server APIs that provide such structured capabilities. That makes it much simpler for clients to support multiple server APIs — where this interface is available, no more version / capability sniffing is needed.
• The much more interesting feature this brings: client interfaces can automatically add GUI for custom capabilities at runtime, i.e. without knowing what a server can do in advance (in theory - it's likely tricky to combine this with good UI). I really like this aspect, it's so much in the fedi spirit of enabling community-tailored spaces.

Whether this more generalized (and for clients likely easier to implement approach) is worth doing an XKCD 927 I can't really tell - just wanted to put the idea out there.

Without being experienced with many fedi projects, I assume that to bring this to widespread adoption / avoid further API customizations, it'd be best to bring this "upstream" into the mastodon C2S API spec, or - when considered infeasible - have a project independent spec for extending the Masto C2S spec — a project on its own that would aim to consolidate the lovely chaos of all these slightly different APIs

[VyrCossont]

The much more interesting feature this brings: client interfaces can automatically add GUI for custom capabilities at runtime, i.e. without knowing what a server can do in advance (in theory - it's likely tricky to combine this with good UI). I really like this aspect, it's so much in the fedi spirit of enabling community-tailored spaces.

It sounds like a good idea, but what you end up with in practice is a bloated API that tries to encapsulate some subset of the controls that HTML, UIKit, Android, etc. provide, but sucks at all of it. You can't internationalize it unless your server provides all the translations and regionally appropriate icons etc. and those probably won't line up with the regions your client knows about. You can't validate constraints that your API's schema language doesn't understand, such as "this post can't be posted with the searchable flag set if discoverable is not also set". There's always one more exception that has to be added.

My professional experience with a production system that tried to do this kind of server-controlled UI in order to make UI experimentation easier was that it was a complete nightmare. It's not tricky to combine this with good UI, it's impossible. There's a reason no actual HATEOAS apps exist and it's because every frontend person on earth laughed at them.

If you want that kind of thing, the way to go is to make it solely a webapp, served from the same system that's serving the API, and accept tight coupling between the webapp and the API. That's pretty much how vanilla Mastodon works today: the web GUI doesn't need to do capability checks because all that info is compiled into the GUI at build time. It'll never need to make an API call to look up the max post length or the supported image types. It'll never try to call an API that doesn't exist because it doesn't expect them to.

So I don't recommend that route.

[VyrCossont]

bring this "upstream" into the mastodon C2S API spec

This will never happen. Eugen massively dislikes the idea that other projects are implementing Mastodon's API and benefiting from third-party Mastodon clients, and has not historically incorporated API changes from other projects, even something as simple as Glitch's max_toot_chars. (It exists now as configuration.statuses.max_characters. Vanilla Mastodon does not provide the historical alias.)

I think the realistic best case scenario for the extensions discovery mechanism is that we can get a consensus to use it among a few of the newer Mastodon-compatible projects.

[nikclayton]

We've been thinking along similar lines. I've jotted down some initial thoughts in https://gist.github.com/nikclayton/7bc5166b910878fd433c4b23350227ef.