MSC2545: Image Packs (Emoticons & Stickers)

[Sorunome]

Rendered

Signed-off-by: Sorunome mail@sorunome.de

Author: @Sorunome
Shepherd: @anoadragon453

Future MSCs

• Image pack cross-room referencing / suggesting (MSC2545: Image Packs (Emoticons & Stickers) #2545 (comment))
• Additional metadata for stickers and emotes to find their pack (MSC2545: Image Packs (Emoticons & Stickers) #2545 (comment))
• Image pack ordering (MSC2545: Image Packs (Emoticons & Stickers) #2545 (comment))
• Multiple shortcodes for a single image (MSC2545: Image Packs (Emoticons & Stickers) #2545 (comment))
• E2EE support (MSC2545: Image Packs (Emoticons & Stickers) #2545 (comment))
• Image-Pack rooms (needing a room type) (MSC2545: Image Packs (Emoticons & Stickers) #2545 (comment))

Current implementations

Emote rendering (rendering of the <img> tag)

• element-web
• revolution (even handles data-mx-emoticon)
• nheko
• fluffychat (even handles data-mx-emoticon)
• neochat
• cinny

Sending, using the mentioned events here

• revolution (emoticons)
• fluffychat (emoticons, stickers)
• neochat (emoticons, at least partial)
• nheko (emoticons, stickers)
• cinny (emoticons, stickers)
• kazv (stickers)

Bridges

• mx-puppet-discord
• matrix-appservice-discord (partially, sending data-mx-emoticon only)

Implementation PRs

FluffyChat

Data model: https://gitlab.com/famedly/company/frontend/libraries/matrix_api_lite/-/merge_requests/26
SDK: https://gitlab.com/famedly/company/frontend/famedlysdk/-/merge_requests/726
Emoticons: https://gitlab.com/famedly/fluffychat/-/merge_requests/433
Stickers: https://gitlab.com/famedly/fluffychat/-/merge_requests/452

Nheko

Stickers: Nheko-Reborn/nheko#648
Sticker editor: Nheko-Reborn/nheko#669
Choosing emoticons: Nheko-Reborn/nheko@ea6b19b

Cinny

Emoticons and Stickers: cinnyapp/cinny#686

kazv

Creating and sending stickers: https://lily-is.land/kazv/kazv/-/merge_requests/71

---

SCT Stuff:

FCP tickyboxes

MSC Checklist

[anoadragon453]

People wanting to try out an implementation of custom emotes on the web can use their existing homeserver account through this riot-web instance.

[MTRNord]

People wanting to try out an implementation of custom emotes on the web can use their existing homeserver account through this riot-web instance.

Fluffychat Android works too :)

[turt2live]

Generally seems to be in a sensible direction, though the duplicated effort for custom emoji is a bit bothersome :(

[MatMaul]

What about adding a mandatory level of pack indirection ?

im.ponies.room_emotes would only contains something like:

{
  "packs": ["im.ponies.room_emotes.mypack1", "im.ponies.room_emotes.mypack2"]
}

and then use what is proposed here inside im.ponies.room_emotes.mypackX state events.

This allows bundling and updating packs independently, and overcome the 65k limit by quite a margin.

[Sorunome]

What about adding a mandatory level of pack indirection ?

im.ponies.room_emotes would only contains something like:

{
  "packs": ["im.ponies.room_emotes.mypack1", "im.ponies.room_emotes.mypack2"]
}

and then use what is proposed here inside im.ponies.room_emotes.mypackX state events.

This allows bundling and updating packs independently, and overcome the 65k limit by quite a margin.

Not quite sure what you mean, here. That, for room emotes, you can specify other state keys that just extend that one pack?

While not explicitly stated in this MSC (yet?), if multiple packs have the same pack.short they are expected to be merged together - that way you can also overcome the 65k limit, however, when adding it to your account data you will have to add multiple entries for just a single pack. Maybe it would be better to explicitly add related state keys?

[MatMaul]

Ok nevermind, I misread. I thought im.ponies.room_emotes was the state key, not the type.

[theotheroracle]

how hard would it be to add versioning and uuids to emoji packs
like for example if i'm in a room with a sticker pack that matches the UUID and version of the pack i'm using, then it won't add those emotes to my emote list

[Sorunome]

how hard would it be to add versioning and uuids to emoji packs
like for example if i'm in a room with a sticker pack that matches the UUID and version of the pack i'm using, then it won't add those emotes to my emote list

You already have versioning with state events - or do you mean something like v1.0? It would be trivial to add more information like that, or a uuid, to the pack object.

That being said, what if multiple packs have the same uuid but different emotes? Merge them? What if you spread one pack over multiple state events to overcome the 65k chars limit and then give them all the same uuid? Should they appear as one pack? How should that work with emote_rooms?

[theotheroracle]

how hard would it be to add versioning and uuids to emoji packs
like for example if i'm in a room with a sticker pack that matches the UUID and version of the pack i'm using, then it won't add those emotes to my emote list

You already have versioning with state events - or do you mean something like v1.0? It would be trivial to add more information like that, or a uuid, to the pack object.

That being said, what if multiple packs have the same uuid but different emotes? Merge them? What if you spread one pack over multiple state events to overcome the 65k chars limit and then give them all the same uuid? Should they appear as one pack? How should that work with emote_rooms?

if emotes are added to a pack, then the version number should be changed, and yeah i meant like a 1.0 thing, if multiple packs have the same uuid and the same version number, but different emotes, then the conflict should be reported to the user, after thinking about it might make more sense to have a last modified date instead of a version number, and it just uses the newest one, because if you want to remove an emote from a pack, then it shouldn't add it back from an older version of the pack, and i don't see how any of this is affected by it being in a room or a user account

[Sorunome]

and i don't see how any of this is affected by it being in a room or a user account

emote_rooms is where you say that you want emotes in a room (by room id and by state key) visible everywhere for yourself. How to handle now, if you spread one pack over multiple state events by putting the same uuid? Do you have to enable all the state keys? Should it autodetermine? If the later, wouldn't that be complex for client development?

and yeah i meant like a 1.0 thing, if multiple packs have the same uuid and the same version number, but different emotes, then the conflict should be reported to the user

Would a user care about the version of an emote pack? How to determine which version is newer? Grammer for version numbers would need to be defined, etc.

[Sorunome]

Like, maybe the uuid and versioning stuff could be added in a separate msc building onto this - the goal of this was to keep things as simple as possible

[theotheroracle]

emote_rooms is where you say that you want emotes in a room (by room id and by state key) visible everywhere for yourself. How to handle now, if you spread one pack over multiple state events by putting the same uuid? Do you have to enable all the state keys? Should it autodetermine? If the later, wouldn't that be complex for client development?

ok if one pack was spread over multiple events, i would have them share the uuid and also have part numbers, like "part":1 "part":2 etc for A, mainting the order of emotes, and B, to prevent them from overlapping eachother.

Would a user care about the version of an emote pack? How to determine which version is newer? Grammer for version numbers would need to be defined, etc.

state events have timestamps right, so you could just use the newest state event instead of bothering with version numbers

[Sorunome]

state events have timestamps right, so you could just use the newest state event instead of bothering with version numbers

origin_server_ts can't be trusted and can easily be forged

ok if one pack was spread over multiple events, i would have them share the uuid and also have part numbers, like "part":1 "part":2 etc for A, mainting the order of emotes, and B, to prevent them from overlapping eachother.

What would it need part for? It shouldn't matter if they are ordered correctly or not, emotes are an unordered set

[theotheroracle]

state events have timestamps right, so you could just use the newest state event instead of bothering with version numbers

oh you could just put a modified tag in then with like unixtime or something

What would it need part for? It shouldn't matter if they are ordered correctly or not, emotes are an unordered set

the most important reason for the part value is to distinguish continuations of the pack from updates and being able to order the emotes could matter to some pack maintainers, i would want to be able to do that for example

[Sorunome]

oh you could just put a modified tag in then with like unixtime or something

In a federated system there is no one true timesource. That is a mathematically unsolvable problem.

the most important reason for the part value is to distinguish continuations of the pack from updates and being able to order the emotes could matter to some pack maintainers, i would want to be able to do that for example

Updates would replace the previous state event. You don't need any "part" attribute for that. As for ordering, you could say they are ordered alphanumerically by the state key.

[tulir]

All old threads seem to be mostly resolved and the rest of the checklist is complete too:

@mscbot resolve checklist incomplete