MSC2176: Update the redaction rules by richvdh · Pull Request #2176 · matrix-org/matrix-spec-proposals

richvdh · 2019-07-14T21:51:38Z

ara4n · 2019-07-14T21:55:16Z

looks plausible to me. sounds like we should get it out there in a room v6 sooner than later

richvdh · 2019-07-15T09:36:26Z

We probably need to get agreement on #1996 and matrix-org/matrix-spec#357 as a prerequisite here. If people have opinions on them, perhaps they could weigh in on the relevant bugs.

bwindels · 2019-07-15T10:49:38Z

For completeness, there is also some ongoing discussion as part of MSC1849/2154 about preserving content/m.relates_to/rel_type on redaction for at least edits, if not all relations.

turt2live

lgtm - I'm more torn on whether notifications on power levels should be kept though.

fix typo Co-Authored-By: Kitsune Ral <Kitsune-Ral@users.sf.net>

richvdh · 2019-07-16T19:04:35Z

A quick note on some of my thinking here.

As I see it, there is a scale between properties which we absolutely must preserve on redaction, for fear of breaking the protocol (mostly those involved in event auth), and those which have very little effect on the protocol at all so can safely be redacted. Somewhere in the middle we have things like predecessor, which probably isn't really a thing you want to redact, but actually the protocol will mostly keep working.

I'm trying quite hard to limit the list of things that get preserved to things which completely break the protocol, rather than just things that are a bit annoying/surprising when you redact them, mostly to keep the server-server protocol as simple as possible. The fact that we have to bump the room version to change the redaction rules also means that it's really not great to have to keep adding new rules. Rather, I see it as the job of either clients (or, possibly, the C-S API) to protect you from foot-guns.

All that said, there is room for debate on where different keys sit on that scale, and where we draw the line, but I'd like each candidate to be retained to be argued on its own merits.

turt2live · 2019-07-17T05:39:45Z

+   * the `notifications` key. Rationale: symmetry with the other `power_levels`
+     settings. (Maybe? See
+     https://github.com/matrix-org/matrix-doc/issues/1601#issuecomment-511237744.)
+


m.room.encryption's algorithm should be preserved.

I disagree. It doesn't change the behaviour of the room at the server-server level.

But if we eventually have two algorithms rather than one how should a client treat m.room.encryption with redacted algorithm?

But if we eventually have two algorithms rather than one

I don't think there is a way to end up with two encryption algorithms in the room state?

how should a client treat m.room.encryption with redacted algorithm?

The same way that they should treat a new m.room.encryption event which disables encryption: with great suspicion.

How are new clients to the room meant to do encryption if they don't know the algorithm?

Can we please relax the restrictions on what goes into the algorithm? It affects client authors too, and it's ridiculous to exclude something because it doesn't affect the core protocol operation.

@KitsuneRal: I think there might be a misunderstanding here.

there's no good way for a client to handle any readable content of the room once that state is broken
...
this room is irreparably broken along with all of its encrypted messages already in DAG.

m.room.encryption is not supposed to define how you interpret received events, so I don't agree that the existing messages are invalidated once m.room.encryption is redacted.

again - repeated encryption events are not proper

there is nothing wrong with repeated m.room.encryption events?

m.room.encryption is not supposed to define how you interpret received events, so I don't agree that the existing messages are invalidated once m.room.encryption is redacted.

That's a pretty strong statement. Then what's the purpose of algorithm there?

there is nothing wrong with repeated m.room.encryption events?

Except they should be treated with great suspicion because there's no defined semantics for any m.room.encryption event except the first one?

Then what's the purpose of algorithm there?

It indicates to clients how they should encrypt messages in this room.

there's no defined semantics for any m.room.encryption event except the first one?

I don't think that's true. I can't find anything in the spec about this, but my mental model is that clients should keep a record of rooms which are encrypted (along with the encryption algorithm), and give the user a warning if they send a message in a room which was previously encrypted and is no longer (or if the encryption is 'weaker', for some handwavey definition of weaker).

But certainly it's valid for there to be multiple encryption events in a room, changing minor settings.

There's also the problem of clients joining a room which has a redacted encryption event: They won't have the prior knowledge that it was encrypted and will assume it's not.

ftr riot-web does exactly as you describe: it ignores changes to the encryption event it wasn't expecting.

I've created https://github.com/matrix-org/matrix-doc/issues/2272 for clarifying the semantics of m.room.encryption events. I'd suggest ignoring m.room.encryption for the purposes of this MSC, and discussing in that issue for a future MSC, as I think that it's a larger issue that shouldn't hold this one up.

* preserve *all* of `create` * don't preserve `notifications` or `algorithm`, and add some justifcation.

richvdh · 2019-07-22T16:56:38Z

I've updated the MSC to (1) change the create event so that all keys are preserved, and (2) clarify that I don't think the notifications property should be preserved, and (2b) to record why I think this is important.

Further input welcome...

KitsuneRal

Typo fixing - however, might be entirely unnecessary if we keep algorithm around :)

Co-Authored-By: Travis Ralston <travpc@gmail.com> Co-Authored-By: Kitsune Ral <Kitsune-Ral@users.sf.net>

richvdh · 2019-08-19T10:22:23Z

Nothing else seems to be happening here, so I'd like to propose FCP:

@mscbot fcp merge

mscbot · 2019-08-19T10:22:26Z

Team member @richvdh has proposed to merge this. The next step is review by the rest of the tagged people:

No concerns currently listed.

Once at least 75% of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

turt2live · 2022-05-04T20:41:25Z

This MSC is currently assigned to v10, though apparently we need to unblock MSC2174 too. #3604

turt2live · 2022-05-20T01:17:28Z

This went back to not having an assigned room version (pulled out of v10).

turt2live · 2023-06-26T17:18:11Z

This is now formally assigned to room version 11: #3820

turt2live · 2023-08-01T05:34:43Z

Spec PR: matrix-org/matrix-spec#1604

turt2live · 2023-08-15T06:19:57Z

Merged 🎉

@chris-ruecker

Synapse 1.26.0 (2021-01-27) =========================== This release brings a new schema version for Synapse and rolling back to a previous version is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes and for general upgrade guidance. No significant changes since 1.26.0rc2. Synapse 1.26.0rc2 (2021-01-25) ============================== Bugfixes -------- - Fix receipts and account data not being sent down sync. Introduced in v1.26.0rc1. ([\#9193](matrix-org/synapse#9193), [\#9195](matrix-org/synapse#9195)) - Fix chain cover update to handle events with duplicate auth events. Introduced in v1.26.0rc1. ([\#9210](matrix-org/synapse#9210)) Internal Changes ---------------- - Add an `oidc-` prefix to any `idp_id`s which are given in the `oidc_providers` configuration. ([\#9189](matrix-org/synapse#9189)) - Bump minimum `psycopg2` version to v2.8. ([\#9204](matrix-org/synapse#9204)) Synapse 1.26.0rc1 (2021-01-20) ============================== This release brings a new schema version for Synapse and rolling back to a previous version is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes and for general upgrade guidance. Features -------- - Add support for multiple SSO Identity Providers. ([\#9015](matrix-org/synapse#9015), [\#9017](matrix-org/synapse#9017), [\#9036](matrix-org/synapse#9036), [\#9067](matrix-org/synapse#9067), [\#9081](matrix-org/synapse#9081), [\#9082](matrix-org/synapse#9082), [\#9105](matrix-org/synapse#9105), [\#9107](matrix-org/synapse#9107), [\#9109](matrix-org/synapse#9109), [\#9110](matrix-org/synapse#9110), [\#9127](matrix-org/synapse#9127), [\#9153](matrix-org/synapse#9153), [\#9154](matrix-org/synapse#9154), [\#9177](matrix-org/synapse#9177)) - During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](matrix-org/synapse#9091)) - Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](matrix-org/synapse#9159)) - Improve performance when calculating ignored users in large rooms. ([\#9024](matrix-org/synapse#9024)) - Implement [MSC2176](matrix-org/matrix-spec-proposals#2176) in an experimental room version. ([\#8984](matrix-org/synapse#8984)) - Add an admin API for protecting local media from quarantine. ([\#9086](matrix-org/synapse#9086)) - Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](matrix-org/synapse#8932)) - Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](matrix-org/synapse#8948)) - Add experimental support for handling to-device messages on worker processes. ([\#9042](matrix-org/synapse#9042), [\#9043](matrix-org/synapse#9043), [\#9044](matrix-org/synapse#9044), [\#9130](matrix-org/synapse#9130)) - Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](matrix-org/synapse#9068)) - Add experimental support for handling `/devices` API on worker processes. ([\#9092](matrix-org/synapse#9092)) - Add experimental support for moving off receipts and account data persistence off master. ([\#9104](matrix-org/synapse#9104), [\#9166](matrix-org/synapse#9166)) Bugfixes -------- - Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](matrix-org/synapse#9023)) - Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](matrix-org/synapse#9028)) - Fix error handling during insertion of client IPs into the database. ([\#9051](matrix-org/synapse#9051)) - Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](matrix-org/synapse#9053)) - Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](matrix-org/synapse#9054)) - Fix incorrect exit code when there is an error at startup. ([\#9059](matrix-org/synapse#9059)) - Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](matrix-org/synapse#9070)) - Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](matrix-org/synapse#9071)) - Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](matrix-org/synapse#9114), [\#9116](matrix-org/synapse#9116)) - Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](matrix-org/synapse#9117)) - Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](matrix-org/synapse#9128)) - Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](matrix-org/synapse#9108)) - Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](matrix-org/synapse#9145)) - Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](matrix-org/synapse#9161)) Improved Documentation ---------------------- - Add some extra docs for getting Synapse running on macOS. ([\#8997](matrix-org/synapse#8997)) - Correct a typo in the `systemd-with-workers` documentation. ([\#9035](matrix-org/synapse#9035)) - Correct a typo in `INSTALL.md`. ([\#9040](matrix-org/synapse#9040)) - Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](matrix-org/synapse#9057)) - Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](matrix-org/synapse#9151)) Deprecations and Removals ------------------------- - Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](matrix-org/synapse#9039)) Internal Changes ---------------- - Improve efficiency of large state resolutions. ([\#8868](matrix-org/synapse#8868), [\#9029](matrix-org/synapse#9029), [\#9115](matrix-org/synapse#9115), [\#9118](matrix-org/synapse#9118), [\#9124](matrix-org/synapse#9124)) - Various clean-ups to the structured logging and logging context code. ([\#8939](matrix-org/synapse#8939)) - Ensure rejected events get added to some metadata tables. ([\#9016](matrix-org/synapse#9016)) - Ignore date-rotated homeserver logs saved to disk. ([\#9018](matrix-org/synapse#9018)) - Remove an unused column from `access_tokens` table. ([\#9025](matrix-org/synapse#9025)) - Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](matrix-org/synapse#9030)) - Fix running unit tests when optional dependencies are not installed. ([\#9031](matrix-org/synapse#9031)) - Allow bumping schema version when using split out state database. ([\#9033](matrix-org/synapse#9033)) - Configure the linters to run on a consistent set of files. ([\#9038](matrix-org/synapse#9038)) - Various cleanups to device inbox store. ([\#9041](matrix-org/synapse#9041)) - Drop unused database tables. ([\#9055](matrix-org/synapse#9055)) - Remove unused `SynapseService` class. ([\#9058](matrix-org/synapse#9058)) - Remove unnecessary declarations in the tests for the admin API. ([\#9063](matrix-org/synapse#9063)) - Remove `SynapseRequest.get_user_agent`. ([\#9069](matrix-org/synapse#9069)) - Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](matrix-org/synapse#9080)) - Add type hints to media repository. ([\#9093](matrix-org/synapse#9093)) - Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](matrix-org/synapse#9098)) - Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](matrix-org/synapse#9106)) - Improve `UsernamePickerTestCase`. ([\#9112](matrix-org/synapse#9112)) - Remove dependency on `distutils`. ([\#9125](matrix-org/synapse#9125)) - Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](matrix-org/synapse#9144)) - Fix the Python 3.5 / old dependencies build in CI. ([\#9146](matrix-org/synapse#9146)) - Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](matrix-org/synapse#9157))

Proposal to update the redaction algorithm

cd5549d

richvdh added the proposal A matrix spec change proposal label Jul 14, 2019

ara4n reviewed Jul 14, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md Outdated

turt2live self-requested a review July 14, 2019 22:02

babolivier reviewed Jul 15, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md

KitsuneRal reviewed Jul 15, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md Outdated

Comment thread proposals/2176-update-redaction-rules.md Outdated

uhoreg reviewed Jul 15, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md Outdated

Comment thread proposals/2176-update-redaction-rules.md Outdated

Comment thread proposals/2176-update-redaction-rules.md Outdated

turt2live added the proposal-in-review label Jul 16, 2019

turt2live approved these changes Jul 16, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md Outdated

richvdh and others added 2 commits July 16, 2019 19:26

Update proposals/2176-update-redaction-rules.md

b49a950

fix typo Co-Authored-By: Kitsune Ral <Kitsune-Ral@users.sf.net>

preserve powerlevel

d324cac

richvdh mentioned this pull request Jul 16, 2019

Redactions should preserve the notifications/invite key on power levels matrix-org/matrix-spec#357

Open

turt2live reviewed Jul 17, 2019

View reviewed changes

uhoreg reviewed Jul 18, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md Outdated

Updates

9e264fe

* preserve *all* of `create` * don't preserve `notifications` or `algorithm`, and add some justifcation.

turt2live self-requested a review July 22, 2019 16:55

turt2live added the unassigned-room-version Remove this label when things get versioned. label Jul 22, 2019

turt2live requested changes Jul 23, 2019

View reviewed changes

richvdh commented Jul 26, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md Outdated

KitsuneRal reviewed Jul 26, 2019

View reviewed changes

Comment thread proposals/2176-update-redaction-rules.md Outdated

Comment thread proposals/2176-update-redaction-rules.md Outdated

richvdh and others added 2 commits July 30, 2019 08:00

Apply suggestions from code review

f1f2936

Co-Authored-By: Travis Ralston <travpc@gmail.com> Co-Authored-By: Kitsune Ral <Kitsune-Ral@users.sf.net>

clarification

353b6cd

mscbot added the proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. label Aug 19, 2019

turt2live added unassigned-room-version Remove this label when things get versioned. and removed blocked Something needs to be done before action can be taken on this PR/issue. unassigned-room-version Remove this label when things get versioned. labels May 4, 2022

turt2live self-assigned this May 4, 2022

richvdh mentioned this pull request May 19, 2022

prune_event strips keys needed for authentication from events. matrix-org/synapse#1831

Open

turt2live added the unassigned-room-version Remove this label when things get versioned. label May 20, 2022

turt2live added the requires-room-version An idea which will require a bump in room version label May 20, 2022

This was referenced May 26, 2022

What the dickens is prev_state? matrix-org/matrix-spec#1091

Closed

Don't include redundant prev_state in new events matrix-org/synapse#12891

Closed

turt2live mentioned this pull request May 24, 2023

MSC3820: Room version 11 #3820

Merged

5 tasks

turt2live added the blocked Something needs to be done before action can be taken on this PR/issue. label Jun 19, 2023

turt2live removed blocked Something needs to be done before action can be taken on this PR/issue. unassigned-room-version Remove this label when things get versioned. labels Jun 26, 2023

turt2live mentioned this pull request Jul 19, 2023

Room version 11 & MSC2176: Only preserve entirety of content instead of whole event matrix-org/synapse#15963

Closed

turt2live added spec-pr-in-review A proposal which has been PR'd against the spec and is in review and removed spec-pr-missing Proposal has been implemented and is being used in the wild but hasn't yet been added to the spec labels Aug 1, 2023

zecakeh mentioned this pull request Aug 12, 2023

events: Update redacted types according to changes in room version 11 ruma/ruma#1619

Merged

turt2live added merged A proposal whose PR has merged into the spec! and removed spec-pr-in-review A proposal which has been PR'd against the spec and is in review labels Aug 15, 2023

clokep mentioned this pull request Aug 23, 2023

Declare support for Matrix v1.8 matrix-org/synapse#16176

Closed

4 tasks

zecakeh mentioned this pull request Aug 24, 2023

Implement Matrix 1.8 changes ruma/ruma#1626

Closed

28 tasks

S7evinK mentioned this pull request Oct 2, 2023

Declare support for Matrix 1.8 matrix-org/dendrite#3226

Open

7 tasks

matrixbot mentioned this pull request Nov 1, 2024

Declare support for Matrix 1.8 element-hq/dendrite#3226

Open

7 tasks

MadLittleMods mentioned this pull request Nov 7, 2024

Silently drop PDUs we fail to parse from federation, instead of failing entire transaction element-hq/synapse#17893

Open

3 tasks

Conversation

richvdh commented Jul 14, 2019 • edited by clokep Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ara4n commented Jul 14, 2019

Uh oh!

richvdh commented Jul 15, 2019

Uh oh!

bwindels commented Jul 15, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

turt2live left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

richvdh commented Jul 16, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KitsuneRal Jul 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

richvdh commented Jul 22, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KitsuneRal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

richvdh commented Aug 19, 2019

Uh oh!

mscbot commented Aug 19, 2019 • edited by anoadragon453 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

turt2live commented May 4, 2022

Uh oh!

turt2live commented May 20, 2022

Uh oh!

turt2live commented Jun 26, 2023

Uh oh!

turt2live commented Aug 1, 2023

Uh oh!

turt2live commented Aug 15, 2023

Uh oh!

Reviewers

Assignees

Labels

richvdh commented Jul 14, 2019 •

edited by clokep

Loading

KitsuneRal Jul 17, 2019 •

edited

Loading

mscbot commented Aug 19, 2019 •

edited by anoadragon453

Loading