Extensions field in records for third-party usage

[mary-ext]

Is your feature request related to a problem? Please describe.

There is friction in third-party clients wanting to add quality-of-life functionality that requires adding additional/custom public metadata to something like profiles, where the records aren't being exposed as is to app.bsky.* endpoints (unlike posts, which does have them)

Describe the solution you'd like

An extensions field added within said record types, this is an object where basically anything goes, and is up for third-party clients to do what they want with it (can be subject to limits). This field can then be forwarded as-is to any app.bsky.* queries, without the need for said queries to expose the actual record itself.

Describe alternatives you've considered

Right now, clients has to retrieve two different subsets of profile informations, say, getProfile to get follow counts, and the actual record itself for these sorts of extensions. It's doable, but it's also a hassle to do, and takes up precious network usage that could be used towards other things (even if they're small responses)

Additional context

It would be nice if we can get something as basic as public profile timezones right now.

[bnewbold]

This is a great idea, thanks for the suggestion! I've transferred this to a longer-form "discussion".

We have had a bit of talk internally about how to better support third party extensions to record schemas. One proposal is basically exactly what you propose: a special dedicated field (probably prefixed with $ to minimize name collisions, and make it clear this is an atproto-level design pattern), where folks can stash extra metadata. This could be "always" an object, and have first-level field names which are NSIDs, which would prevent collisions and even (in theory, eventually) allow for Lexicon specification and validation.

An alternative is to lean more heavily in to "paired records" in different Lexicon namespaces, but same rkey for easier lookup. The threadgate mechanism for controlling post replies uses this pattern (though that hasn't been documented or even shipped yet).

Some open questions:

• where these extension fields can exist: only at top-level? Anywhere a $type would be required to disambiguate the object type? Eg, if you wanted to add per-image metadata to posts
• does this mean we would change expectations and state that folks should not ever stash metadata in random other fields? or would both extension/evolution mechanisms be ok?
• do we want to encourage this as a full-featured extension pattern, or is it a "last resort" / "relief valve" sort of thing?

[mary-ext]

1. I could see per-image custom metadata to be a necessity, say for indicating that this one image is a speciality that points to another resource, but I'd imagine having it be this open could spiral out of control real fast. Top-level should be fine for now, but for an object lexicon type perhaps it might be worth having an extensible field that determines this?
2. I think we definitely should, although these custom metadata doesn't appear in any official means, these fields can be confusing when they do appear especially when you don't always have the documentation besides you all the time.
3. I'd say last resort, but it sure is tempting 😅

[DavidBuchanan314]

Another potential solution is just to include the full record data in getProfile, in a new field. For example, getPostThread responses include a record field, which makes it easy to extract any extended information from them. This does mean some information is duplicated, but it doesn't seem so bad overall.

[bnewbold]

(necrothreading, but relevant)

yes! I think including the original record verbatim is a pattern we definitely want to encourage. have been meaning to write a Lexicon-authoring style guide and just haven't gotten to it so far.

[mikestaub]

I would be cautious to implement something custom here, as this is a generic problem that JSON schema and OpenAPI 3 have solved with $ref. How hard would it be to allow Lexicons to 'inherit and extend' other Lexicons?