Implement MSC2228 Self destructing messages

[neilisfragile]

matrix-org/matrix-spec-proposals#2228

[richvdh]

I think this was done by #6409?

[richvdh]

I think this was done by #6409?

this was only a partial implementation.

Next step is for us to try to put an estimate on the remainder of the work.

[babolivier]

I had some thoughts over the past few days about finishing up the implementation of this MSC in Synapse (i.e. implementing synthetic redactions and support for m.self_destruct).

This plan makes one major assumption, which IIRC is left as an implementation detail in the MSC, which is what to do with people joining the room before the event if the history visibility setting of the room isn't joined. My opinion is that the server should always show them the synthetic redaction, so the event isn't "revived" each someone joins. We could also think about a slight variant of this strategy where, if the room's history visibility is invited, we also consider the users that were invited at the time the event was sent.

First, I think we should create a new table, let's name it ephemeral_messages for this explanation. Each time we receive an ephemeral message, we'd store in this table:

• The stream ID
• The event ID
• The room ID
• The value for m.self_destruct

We'd also need another table, let's say synthetic_redactions, to store synthetic redactions, more specifically:

• The stream ID at which the synthetic redaction was generated
• The event ID for the event to redact
• The user the synthetic redaction is for

This list might be expanded to include whatever is necessary to generate a synthetic redaction.

Then, each time we get a read receipt:

1. Fetch the stream ID of the previous read receipt for the same user in the same room
2. Check if there's an entry in ephemeral_messages_readers for this room which stream ID is after the one fetched in the previous step
3. If there is at least one entry, for each of them, as a background process:
   1. Check if there's already an entry in the synthetic_redactions table for this user and this event, if not continue to the next event
   2. Fetch the list of members in the room when the event was sent
   3. Check if the user is in this list of members, if not continue to the next event
   4. Wait for the amount of time specified in m.self_destruct
   5. Generate an entry in synthetic_redactions for this event and this user
   6. Notify the sync so the redaction is sent to clients

We would also need to introduce some changes in filter_events_for_client so that, if the event its sub-function allowed is an ephemeral message:

• If the user wasn't a member of the room when the event was sent, replace it with a redacted or pruned version of the event
• Otherwise:
  • If there is a synthetic redaction stored for this user and this event, replace it with a redacted/pruned version of the event
  • Otherwise just return the event as is (modulo additional checks in this function)

The sync handler would also need to read from the synthetic_redactions table and insert synthetic redactions at the right stream IDs.

This way when a user syncs, for each expired ephemeral message:

• If the user wasn't a member when the message was sent, they will only see the redacted/pruned event
• Otherwise:
  • If the sync is incremental and the sync token is after the event is sent, they will see the synthetic redaction and their client should process it as a normal m.room.redact event
  • Otherwise they will see both the redacted/pruned event and the synthetic redaction

Additionally, we could also add a looping background process, which, for each ephemeral message would check the event has been seen by every user that was in the room when it was sent, in which case we could plan censoring it like we currently do with redactions, as well as removing its entry in ephemeral_messages.

[richvdh]

sounds generally right to me!

Some addtional thoughts:

• what is the expected behaviour when a user opens a room that they haven't opened for a while? we'll send a RR for the most recent event, so does that mean that any ephemeral events before that will start their countdowns, even if they are off the top of the user's screen? Not sure if the MSC takes a view on this.
• what happens if synapse is restarted while a message is in its countdown period? Sadly we'll have to persist this somehow.
• I've got a feeling synthetic redactions might need their own stream IDs (and entry in the sync token), particularly in a world where event persistence happens off-master, which might add a little to the work involved, but probably not materially.

@babolivier I'd suggest moving what you've written out to a google doc for now, so that we can comment on it and refine it.

[babolivier]

Good calls, thanks, I missed these points.

I've copied all of this documentation and updated it in response to your comment on https://docs.google.com/document/d/1wWX514eG_a-KllsWD9v8Y36mV2pWnsGwqON8aPYk1-Y/edit#

[trymeouteh]

Many privacy chat apps like Signal support a feature which allow you to have messages delete itself. I think this can be useful feature in Matrix for a layer of privacy on your messages and chats. Here is how it can work.

-Any user can create a self destructing message
-Any admin of a chat can set all messages from here on to be self destructive (users cannot opt out)
-Self destructive messages can be set to expire after X amount of views, amount of time, or until a certain date and time.
-If chats have self destructive messages enabled, the admin can choose the range of when a message will expire after X amount of views, amount of time, or until a certain date and time

[zzaurak]

Hi, any update on this?

[HammyHavoc]

Just wanted to say that this is a must. Especially being able to view ephemeral content ala 'Stories'. Even BBM had a feed.

[mrx23dot]

In WireApp it can be set on Room level or user can choose to destruct sooner.
Nice way to auto destroy evidence while you sit in jail.

[MacTheZazou]

As a journalist we really need this for our security. Can a bot do it kind of securely in the meantime? Also is it possible to finance this features with a donation?

[wokawoka]

+1

[ThomasHalwax]

As a journalist we really need this for our security. Can a bot do it kind of securely in the meantime? Also is it possible to finance this features with a donation?

@MacTheZazou

Since there seems to be little progress in this matter, we have developed a Matrix burn-bot that can at least handle the simple "Burn-After-TTL" strategy.
Please take a look at it and give us feedback so that we can further improve its capabilities.

[heyakyra]

As a journalist we really need this for our security. Can a bot do it kind of securely in the meantime? Also is it possible to finance this features with a donation?

As far as I understand it, with any application/standard that implements this, it is a very thin layer of security in which you must necessarily trust the user, said user's client, and said user's [home]server to have implemented such a functionality honestly, and there is no real way to verify or enforce deletion from the other end.

Maybe in the not too distant future, verified reproducible builds can be coupled with key exchanges for something that gets partially there…but I don't really see how practical that could be when there could be any number of client implementations of the protocol.

[MacTheZazou]

@heyakyra

As a journalist we really need this for our security. Can a bot do it kind of securely in the meantime? Also is it possible to finance this features with a donation?

As far as I understand it, with any application/standard that implements this, it is a very thin layer of security in which you must necessarily trust the user, said user's client, and said user's [home]server to have implemented such a functionality honestly, and there is no real way to verify or enforce deletion from the other end.

Maybe in the not too distant future, verified reproducible builds can be coupled with key exchanges for something that gets partially there…but I don't really see how practical that could be when there could be any number of client implementations of the protocol.

I don't care if I can't trust the client I'm talkimg to! My contacts can take screenshot if they want & I won't ever know! I just need a reliable way to destruct message I send & receive on my devices!

[ThomasHalwax]

I don't care if I can't trust the client I'm talkimg to! My contacts can take screenshot if they want & I won't ever know! I just need a reliable way to destruct message I send & receive on my devices!

Unfortunately what the bot does on the server side is not enough. If the clients are offline they will not receive the redaction event and thus not remove the message from their local store.

Since redacting messages is idempotent a good thing would be to have some kind of self-destruction policy for the room that bot the server (or a bot) and the clients need to enforce.

[HammyHavoc]

I don't care if I can't trust the client I'm talkimg to! My contacts can take screenshot if they want & I won't ever know! I just need a reliable way to destruct message I send & receive on my devices!

Unfortunately what the bot does on the server side is not enough. If the clients are offline they will not receive the redaction event and thus not remove the message from their local store.

Since redacting messages is idempotent a good thing would be to have some kind of self-destruction policy for the room that bot the server (or a bot) and the clients need to enforce.

As well as at a room level, I'd be interested in a per-message implementation too. E.g., some messages are more sensitive than others, so when sending it, maybe an erase after reading option, or erase in x time.

[ThomasHalwax]

As well as at a room level, I'd be interested in a per-message implementation too. E.g., some messages are more sensitive than others, so when sending it, maybe an erase after reading option, or erase in x time.

I've been thinking about more advanced features as well. The next things could still be TTL related and - as you suggested - allow users to define a TTL for their own messages.

[babolivier]

As well as at a room level, I'd be interested in a per-message implementation too. E.g., some messages are more sensitive than others, so when sending it, maybe an erase after reading option, or erase in x time.

I've been thinking about more advanced features as well. The next things could still be TTL related and - as you suggested - allow users to define a TTL for their own messages.

MSC2228 (which is the topic of this issue) is specifically about message-level TTLs. For room-level retention policies you want MSC1763 (which MSC2228 is a subset of, and was turned into its own MSC to optimise scope), which is already partly implemented in Synapse (but currently lacks client support, at least as far as I'm aware): https://matrix-org.github.io/synapse/latest/message_retention_policies.html