Currently replies require clients to send and parse a fallback representation of the replied to message. While at least in theory fallbacks should make it simpler to start supporting replies in a new client, they actually introduce a lot of complexity and implementation issues and block a few valuable features. This MSC proposes to deprecate and eventually remove those fallbacks. It was an alternative to MSC2589, which chose a different representation of the reply fallback, but was ultimately closed in favor of this proposal.
Remove the rich reply fallback from the
specification.
Clients should stop sending them and should consider treating <mx-reply> parts
as either something to be unconditionally stripped or as something to be escaped
as invalid html. Clients may send replies without a formatted_body now using
arbitrary message events (not state events), which is currently still disallowed
by the
specification.
As a result of this, you would be able to reply with an image. New clients would also be able to implement edits and replies more easily, as they can sidestep a lot of pitfalls.
Given clients have had enough time to implement replies completely, the overall look & feel of replies should be unchanged or even improved by this proposal.
An extended motivation is provided at the end of this document.
Obviously you can't remove the fallback from old events. As such clients would still need to do something with them in the near future. I'd say just not handling them in a special way should be okay after some unspecified period of time.
Clients not implementing rich replies or edits may show some slightly more confusing messages to users as well. I'd argue though that in most cases, the reply is close enough to the replied to message, that you would be able to guess the correct context. Replies would also be somewhat easier to implement and worst case, a client could very easily implement a little "this is a reply" marker to at least mark replies visually. Not having a reply fallback might also prompt some clients to implement support for rich replies sooner. Users will now complain about no context. Previously there was some context for replies to text messages, which made it easy to accept the solution as good enough. However the experience with replies to images was not acceptable. Motivating clients to implement rich replies is a good thing in the long run and will improve the Matrix experience overall.
You might not get notifications any more for replies to your messages. This was a feature of the reply fallback, because it included the username, but users had no control over it. Notifications can be added back by an MSC like MSC3664 or a similar proposal and give the user more control over the notifications while also being an explicit solution.
MSC2589: This adds the reply text as an additional key. While this solves the parsing issues, it doesn't address the other issues with fallbacks.
One could also just stick with the current fallbacks and make all clients pay the cost for a small number of clients actually benefitting from them.
Lastly one could introduce an alternative relation type for replies without fallback and deprecate the current relation type (since it does not fit the new format for relations anyway). We could specify, that the server is supposed to send the replied_to event in unsigned to the client, so that clients just need to stitch those two events together, but don't need to fetch the replied_to event from the server. It would make replies slightly harder to implement for clients, but it would be simpler than what this MSC proposes.
Removing the fallback from the spec may lead to issues, when clients experience the fallback in old events. This should not add any security issues the client didn't already have from interpreting untrusted html, though. In all other cases this should reduce security issues (see https://github.com/vector-im/element-web/releases/tag/v1.7.3 or the appendix for examples).
Of the 23 clients listed in the matrix client matrix 16 are listed as not supporting replies (updated January 2022):
- Element Android: Relies on the reply fallback.
- Element iOS: Does not support rich replies
- weechat-matrix: Actually has an implementation to send replies although it seems to be broken. Doesn't render rich replies. Hard to implement because of the single socket implementation, but may be easier in the Rust version.
- Quaternion: Blocked because of fallbacks.
- matrixcli: Doesn't support formatted messages.
- Ditto Chat: Seems to rely on the fallback
- Mirage: Supports rich replies, but doesn't strip the fallback correctly and uses the fallback to render them.
- Nio: Unsupported.
- Pattle: Client is not being developed anymore.
- Seaglass: Doesn't support rich replies, but is unhappy with how the fallback looks?
- Miitrix: Somewhat unlikely to support it, I guess?
- matrix-commander: No idea, but doesn't look like it.
- gotktrix: Seems to rely on the reply fallback
- Hydrogen: Seems to use the reply fallback
- kazv: Doesn't seem to support replies at all
- Syphon: Uses the reply fallback in body
So in summary, 3/4 of the listed clients don't support replies. At least one client doesn't support it because of the fallback (Quaternion). 3 of the command line clients probably won't support replies, since they don't support formatted messages and replies require html support for at least sending.
Only one client implemented rich replies in the last 1.5 years after the original list was done in October 2020. Other clients are either new in my list or didn't change their reply rendering. I would appreciate to hear, why those client developers decided not to support rich reply rendering and if dropping the reply fallback would be an issue for them.
Changes from 1.5 years ago as of January 2022:
- Fractal: Seems to support replies now!
- Commune: Seems to support rich reply rendering and style them very nicely.
- NeoChat: Supports rich replies
- Cinny: Seems to support rich replies
- gomuks: Strips the reply fallback
- Lots of other new clients!
So far I haven't found a client that completely breaks without the fallback.
All clients that support rendering rich replies don't break, when there is no
fallback according to my tests (at least Nheko, Element/Web, FluffyChat and
NeoChat were tested and some events without fallback are in #nheko:nheko.im and
I haven't heard of any breakage). Those clients just show the reply as normal
and otherwise seem to work completely fine as well. Element Android and Element
iOS just don't show what message was replied to. Other clients haven't been
tested by the author, but since the content of an event is untrusted, a client
should not break if there is no reply fallback. Otherwise this would be a
trivial abuse vector.
This section was moved to the back of this MSC, because it is fairly long and exhaustive. It lists all the issues the proposal author personally experienced with fallbacks in their client and its interactions with the ecosystem.
To reply to a reply, a client needs to strip the existing fallback of the first reply. Otherwise replies will just infinitely nest replies. While the spec doesn't necessarily require stripping the fallback in replies to replies (only for rendering), not doing so risks running into the event size limit, but more importantly, it just leads to a bad experience for clients actually relying on the fallback.
Stripping the fallback is not trivial. Multiple implementations had bugs in their fallback stripping logic. The edge cases are not covered in the specification in detail and some clients have interpreted them differently. Common mistakes include:
- Not stripping the fallback in body, which leads to a very long nested chain.
- Not dealing with mismatched
<mx-reply>tags, which can look like you were impersonating someone.
For the body extra attention needs to be paid to only strip lines starting
with > until the first empty line. Implementations either only stripped the
first line, stripped all lines starting with > until the first non empty line,
that does not start with > or stripped only the formatted_body. While those
are implementation bugs, they can't happen if you don't need to strip a
fallback.
To create a new fallback, a client needs to add untrusted html to its own events. This is an easy attack vector to inject your own content into someone elses reply. While this can be prevented with enough care, since Riot basically had to fix this issue twice, it can be expected that other clients can also be affected by this.
The spec requires rich replies to have a fallback using html:
Rich replies MUST have a format of org.matrix.custom.html and therefore a formatted_body alongside the body and appropriate msgtype.
This means you can't reply using only a body and you can't reply with an
image, since those don't have a formatted_body property currently. This means
a text only client, that doesn't want to display html, still needs to support
html anyway and that new features are blocked, because of fallbacks.
While the spec says how a fallback "should" look, there are variations in use which further complicates stripping the fallback or are common mistakes, when emitting the fallback. Some variations include localizing the fallback, missing suggested links or tags, using the body in replies to files or images or using the display name instead of the matrix id.
As a result the experience in clients relying on the fallback or stripping the fallback varies depending on the sending client.
A reply includes the body of another event. This means a reply to an event can
leak data to users, that joined this room at a later point, but shouldn't be
able to see the event because of visibility rules or encryption. While this
isn't a big issue, there is still an issue about it: https://github.com/matrix-org/matrix-doc/issues/1654
Historically clients have also sometimes localized the fallbacks. In those cases they leak the users language selection for their client, which may be personal information.
The above issues are minor, if reply fallbacks added sufficient value to clients. Bridges usually try to bridge to native replies, so they need to strip the reply fallback (https://github.com/matrix-org/matrix-doc/issues/1541). Even the IRC bridge seems to send a custom fallback, because the default fallback is not that welcome to the IRC crowd, although the use cases for simple, text only bridges is often touted as a good usecase for the fallback (sometimes even explicitly mentioning bridging to IRC). As a result there are very few bridges, that benefit from the fallback being present.
Some clients do choose not to implement rich reply rendering, but the experience tends to not be ideal, especially in cases where you reply to an image and now the user needs to guess, what image was being replied to.
As a result the fallbacks provide value to only a subset of the Matrix ecosystem.
- Edits explicitly mention
that a reply fallback should not be sent in the
m.new_content. This causes issues for clients relying on the fallback, because they won't show replies once a message has been edited (see Element Android as a current example) and similar edge cases. - Extensible events
require an update to the specification for fallbacks (because there is no
bodyorformatted_bodyanymore after the transition period). The current proposal also intends to just drop the fallbacks in extensible events.
Since the fallback is added as normal text into the message, it needs to be localized for the receiving party to understand it. This however proves to be a challenge, since users may switch languages freely in a room and it is not easy to guess, which language was used in a short message. One could also use the client's language, but that leaks the user's localization settings, which can be a privacy concern and the other party may not speak that language. Alternatively a client can just send english fallbacks, but that significantly worsens the experience for casual users in non-english speaking countries. The specification currently requires them to not be translated (although some clients don't follow that), but not sending a fallback at all completely sidesteps the need for the spec to specify that and clients relying on an english only fallback.
Clients should use the prefix im.nheko.msc2781. for all their event types, if
they implement this MSC in a publicly available release or events may otherwise
bleed into public rooms.