Fix event size checks

[deepbluev7]

This fixes Synapse possibly defederating some rooms with conduit and dendrite<=0.9.5 as well as breaking some clients relying on the guarantees of the client-server API.

Spec link: https://spec.matrix.org/v1.3/client-server-api/#size-limits

Spec issue: matrix-org/matrix-spec#1001

This does break some rooms, if they have any events violating these constraints in their auth chain. This however should not affect that many rooms, considering how long it took to discover this. Most notably those rooms didn't federate with conduit or dendrite servers (although the latter has changed their auth code to match the current synapse implementation recently). More specifically:

• the event id and room id is not user controlled and all implementations so far use plain ASCII. There are some servers that patched in custom overrides for these IDs, but imo we can safely ignore those as any rooms I encountered so far that violate the tighter constraints were created to test this bug.
• similarly no event type in the specification currently violates this constraint and events outside of the spec are rarely an important part of the auth chain and can usually be safely dropped. Clients also generally use ascii and short event types when possible, but it might serve well to check the matrix.org database for any events falling into the now invalid area. Please ignore any rooms with room ids from maunium.net of course.
• The state key and user ids are trickier. Servers in general don't allow you to register non-ascii userids, many even properly restricting to lowercase characters only. So on unmodified servers such userids don't exist. However, you can somewhat trivially specify arbitrary state_keys using the devtools in Element. Though again, those events are usually not in the auth chain: https://spec.matrix.org/v1.3/server-server-api/#auth-events-selection. The only event with a non-empty state_key in the auth-chain is the m.room.member event, which uses the user_id as the state_key, which as previously discussed usually can't use non-ascii characters.

Considering all of the above, imo this should only break rooms intentionally abusing this issue to break rooms. It also makes Matrix a lot more predictable to handle and means you don't need to implement codepoint parsing in SDKs and other Matrix middleware.

Signed-off-by: Nicolas Werner nicolas.werner@hotmail.de

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Pull request includes a sign off
• Code style is correct
  (run the linters)

[reivilibre]

As far as I can tell, this was 'introduced' when we started to use Python 3. There isn't a clear introduction date since Synapse was, at one point in history, usable on either Python 2 or 3.

Unfortunately this is now blocked until the spec makes its mind up about what to do here :/

[erikjohnston]

Consensus on spec issue is that this is fine.

[DMRobertson]

I'm going to revert this on develop. We want to see this change land for the protocol's sake (and plan to un-revert it) but want to give this a little more time before releasing this.

[DMRobertson]

I'm going to revert this on develop. We want to see this change land for the protocol's sake (and plan to un-revert it) but want to give this a little more time before releasing this.

This ended up happening in #14664, due for release in Synapse 1.74.