MSC4027: Propose method of specifying custom images in reactions #4027

sumnerevans · 2023-06-10T00:25:53Z

A proposal to render MXC URIs in reaction keys as images.

Rendered

Implementations

Clients

Schildichat
- Android: SchildiChat/SchildiChat-android@85a26ae
- Desktop implementation (hasn't landed in a release yet): SchildiChat/matrix-react-sdk@4d104fc
Beeper (all platforms)
Element Web: Render custom images in reactions matrix-react-sdk#11087
Element X Android: Implement MSC4027 (Custom images in reactions) element-hq/element-x-android#2160
Cinny: cinnyapp/cinny@519f620

Bridges

mautrix-discord: https://github.com/mautrix/discord/blob/00465bb71509b984614ed41fa4f147a9d6bd4b3a/example-config.yaml#L145-L147

Supercedes #3746

FCP tickyboxes

Signed-off-by: Sumner Evans <sumner@beeper.com>

proposals/4027-custom-images-in-reactions.md

Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans · 2023-06-12T15:12:18Z

I don't think this deserves the needs-implementation label as it is already implemented across multiple clients and at least one bridge.

The Discord bridge already uses com.beeper.reaction.shortcode unstable field name. Signed-off-by: Sumner Evans <sumner@beeper.com>

SpiritCroc · 2023-06-12T20:01:55Z

Some extra details concerning SchildiChat:

Android implementation: SchildiChat/SchildiChat-android@85a26ae
Desktop implementation (hasn't landed in a release yet): https://github.com/SchildiChat/matrix-react-sdk/commits/4d104fc87f8d32da679476967850c5f7a9c20c8a
We don't do anything with shortcode yet 🙃

anoadragon453

This LGTM at a high level.

Once we have a mechanism for image packs, it may be nice to link back to that pack from the reaction. But that change should live in the image packs MSC, instead of this one.

proposals/4027-custom-images-in-reactions.md

anoadragon453 · 2023-06-16T19:45:58Z

proposals/4027-custom-images-in-reactions.md

+    "event_id": "$abcdefg",
+    "key": "mxc://matrix.org/VOczFYqjdGaUKNwkKsTjDwUa"
+  },
+  "shortcode": ":partyparrot:"


Should we enforce that shortcode's must begin and end with :? I think it'd be confusing to users if some clients started picking some other character.

Added a "should" for that. It's not enforceable other than by convention.

A "must" snuck back in on line 33.

The type of the field is fairly easily enforceable by servers. We are already saying that they should verify the length. (Obviously if a rouge server doesn't validate these fields, we could have "bad" events come across federation.)

I think the question here is what part of the verification is "should" vs "must"? Right now, the servers "must" verify type and length of the shortcode, but the leading and trailing : is "should".

Maybe everything should be "must" and force servers to verify all three things, but leave it up to the server implementation if they want to locally redact invalid events that come across federation?

I think it would eliminate confusion in implementations for all three to be a MUST on the Client-Server API.

leave it up to the server implementation if they want to locally redact invalid events that come across federation?

I would say yes, though I imagine most implementations would not redact such events - instead just allowing any UI bugs in the client that would surface.

proposals/4027-custom-images-in-reactions.md

anoadragon453 · 2023-06-16T20:06:56Z

I played around with matrix-org/matrix-react-sdk#11087 and it looks good! Removing the needs-implementation label.

Signed-off-by: Sumner Evans <sumner@beeper.com>

cloudrac3r · 2023-09-22T01:21:38Z

This proposal is implemented in Out Of Your Element: https://gitdab.com/cadence/out-of-your-element/compare/c7ddf638dbd658057ec6a7a018a23b0693a97890~..044ccc08e06f11b48e16c7f7637d3c088008119a#diff-0f01ec9090ff5700793cc2f6399fb95cf3a7115f

It follows the spec and it works with Schildi and Cinny.

kimabjorkede · 2023-09-25T10:44:44Z

It seems like Element now has this setting in Labs, and it works very well.

anoadragon453

Barring the below concern, I think this MSC is ready for review from the rest of the Spec Core Team.

@mscbot fcp merge

anoadragon453 · 2023-09-25T17:06:43Z

proposals/4027-custom-images-in-reactions.md

+
+2. When the annotation's key is an MXC URI, a new (optional) `shortcode` key can
+   be added to the content of the event with a textual name for the image. This
+   field must be a string and should start and end with the `:` (colon)


@mscbot concern Requiring two : characters in every shortcode string.

Could you explain why we should encourage clients to put :s in every shortcode instead of restricting : from being put in the string?

I know we can't enforce it over federation, but the spec could at least enforce it in the Client-Server API. I think the occasional extra : appearing in clients is worth the bandwidth saving.

For bridges, if they connect to a network that includes : on the ends of their custom emote shortcodes (or another character), then the bridge can just strip it, no?

mscbot · 2023-09-25T17:10:45Z

turt2live · 2023-09-25T17:12:31Z

proposals/4027-custom-images-in-reactions.md

@mscbot concern Needs to be reviewed against other custom emoji/sticker MSCs

I think in particular, this MSC sets up grammar for emoji shortcodes in Matrix, which we'll want to double-check aligns with whatever MSC defines sharable image packs.

@turt2live did you have anything else in mind? The rest is fairly reaction-specific IMO.

Shortcodes and feature as a whole are conflicting, imo. Specific impact to be determined once the remainder of the stack is looked at :)

(I plan to do this over the next 2ish weeks, barring other commitments - will hand over ~next week if things go poorly on my side)

proposals/4027-custom-images-in-reactions.md

turt2live · 2024-04-01T21:42:45Z

As this is tied to the media linking series and unable to progress otherwise, I'm cancelling FCP for the time being.

This does not mean the MSC is unimportant, but rather the focus is elsewhere at the moment.

@mscbot fcp cancel

sumnerevans changed the title ~~MXCXXXX: Propose method of specifying custom images in reactions~~ MXC4027: Propose method of specifying custom images in reactions Jun 10, 2023

Propose method of specifying custom images in reactions

d4ff1c7

Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans force-pushed the custom-images-in-reactions branch from fb1c880 to d4ff1c7 Compare June 10, 2023 00:26

turt2live added proposal A matrix spec change proposal client-server Client-Server API kind:feature MSC for not-core and not-maintenance stuff needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Jun 10, 2023

turt2live changed the title ~~MXC4027: Propose method of specifying custom images in reactions~~ MSC4027: Propose method of specifying custom images in reactions Jun 10, 2023

clokep reviewed Jun 10, 2023

View reviewed changes

proposals/4027-custom-images-in-reactions.md Show resolved Hide resolved

Add mention of multiple MXC URIs pointing to the same file

cdfe089

Signed-off-by: Sumner Evans <sumner@beeper.com>

Update shortcode unstable field name

c86338a

The Discord bridge already uses com.beeper.reaction.shortcode unstable field name. Signed-off-by: Sumner Evans <sumner@beeper.com>

anoadragon453 approved these changes Jun 16, 2023

View reviewed changes

proposals/4027-custom-images-in-reactions.md Outdated Show resolved Hide resolved

anoadragon453 reviewed Jun 16, 2023

View reviewed changes

proposals/4027-custom-images-in-reactions.md Outdated Show resolved Hide resolved

anoadragon453 reviewed Jun 16, 2023

View reviewed changes

proposals/4027-custom-images-in-reactions.md Show resolved Hide resolved

anoadragon453 removed the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Jun 16, 2023

sumnerevans added 2 commits June 30, 2023 11:34

Remove recommendation for detecting if a string is an MXC URI

75473cc

Signed-off-by: Sumner Evans <sumner@beeper.com>

Specify that shortcode should start & end with ':'

5424ea8

Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans mentioned this pull request Jun 30, 2023

Render custom images in reactions matrix-org/matrix-react-sdk#11087

Merged

3 tasks

Clarify that shortcode must be a string

2ac5328

Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans mentioned this pull request Jun 30, 2023

MSC3746: Render image data in reactions #3746

Closed

Add shortcode length check verbiage

badc1ba

Signed-off-by: Sumner Evans <sumner@beeper.com>

gabrc52 mentioned this pull request Sep 12, 2023

Custom reactions gabrc52/mattermost-to-matrix#24

Open

cloudrac3r approved these changes Sep 25, 2023

View reviewed changes

anoadragon453 approved these changes Sep 25, 2023

View reviewed changes

This comment was marked as off-topic.

Sign in to view

This comment was marked as spam.

Sign in to view

mscbot added proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. disposition-merge unresolved-concerns This proposal has at least one outstanding concern labels Sep 25, 2023

turt2live reviewed Sep 25, 2023

View reviewed changes

anoadragon453 mentioned this pull request Sep 25, 2023

The bot cleared FCP votes after a concern was raised matrix-org/mscbot-python#34

Open

turt2live self-assigned this Oct 24, 2023

eleanor-clifford reviewed Nov 2, 2023

View reviewed changes

proposals/4027-custom-images-in-reactions.md Show resolved Hide resolved

turt2live mentioned this pull request Nov 14, 2023

MSC3911: Linking media to events #3911

Open

turt2live mentioned this pull request Nov 27, 2023

MSC discussion: Custom emoji/stickers and interactions with authenticated & linked media matrix-org/matrix-spec#1679

Open

turt2live added this to Ready for FCP ticks in Spec Core Team Backlog Dec 21, 2023

surakin mentioned this pull request Jan 4, 2024

Implement MSC4027 (Custom Images in Reactions) element-hq/element-x-android#2159

Closed

mscbot added proposal-in-review and removed proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. disposition-merge labels Apr 1, 2024

turt2live removed this from Ready for FCP ticks in Spec Core Team Backlog Apr 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC4027: Propose method of specifying custom images in reactions #4027

MSC4027: Propose method of specifying custom images in reactions #4027

sumnerevans commented Jun 10, 2023 •

edited by tulir

Loading

sumnerevans commented Jun 12, 2023

SpiritCroc commented Jun 12, 2023

anoadragon453 left a comment •

edited

Loading

anoadragon453 Jun 16, 2023

sumnerevans Jun 30, 2023

anoadragon453 Sep 25, 2023

sumnerevans Sep 25, 2023

anoadragon453 Sep 25, 2023

anoadragon453 commented Jun 16, 2023 •

edited

Loading

cloudrac3r commented Sep 22, 2023 •

edited

Loading

kimabjorkede commented Sep 25, 2023

anoadragon453 left a comment

anoadragon453 Sep 25, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as spam.

mscbot commented Sep 25, 2023 •

edited

Loading

turt2live Sep 25, 2023

anoadragon453 Sep 25, 2023

turt2live Sep 25, 2023

turt2live commented Apr 1, 2024

MSC4027: Propose method of specifying custom images in reactions #4027

Are you sure you want to change the base?

MSC4027: Propose method of specifying custom images in reactions #4027

Conversation

sumnerevans commented Jun 10, 2023 • edited by tulir Loading

Implementations

Clients

Bridges

sumnerevans commented Jun 12, 2023

SpiritCroc commented Jun 12, 2023

anoadragon453 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anoadragon453 commented Jun 16, 2023 • edited Loading

cloudrac3r commented Sep 22, 2023 • edited Loading

kimabjorkede commented Sep 25, 2023

anoadragon453 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as spam.

mscbot commented Sep 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turt2live commented Apr 1, 2024

sumnerevans commented Jun 10, 2023 •

edited by tulir

Loading

anoadragon453 left a comment •

edited

Loading

anoadragon453 commented Jun 16, 2023 •

edited

Loading

cloudrac3r commented Sep 22, 2023 •

edited

Loading

mscbot commented Sep 25, 2023 •

edited

Loading