Rooms created without any membership (only m.room.create event)

[matrixbot]

This issue has been migrated from #15005.

---

Description

Trying to tie up any loose-ends with the Gitter import as part of the Gitter migration process and noticing that we have some Matrix rooms recorded that seem to broken on the Synapse side. Some of these rooms only have a m.room.create event in it while others have no events which means we can't interact with it at all since permission denied errors everywhere (no membership or power-level events).

The only way we recorded this room on the Gitter side is if we got a 200-response from Synapse originally when it was created. If Synapse is returning with a error-free response, there should be much better guarantees on the room being created successfully (atomic transaction). Of course it's possible this could have been corrupted after the fact though. What most likely happened was the rooms were attempted to be created but timed out. Then we introduced code to reconcile the case where a Matrix room alias exists but we don't have the bridged Matrix room entry recorded, see matrix-org/synapse#15005 (comment)

They all seem to be created on January 10th, 2023 which is probably in the middle of importing messages and the import scripts.

Here is the list of affected rooms that I know of:

• !gHbYoGxiEuXmahoYeh:gitter.im (only m.room.create)
• !OpjflmXuauidodzmJH:gitter.im (only m.room.create)
• !QCVddUkjrHLjEqeXQd:gitter.im (no events)
• !wJBkwhqwLssHVjKErI:gitter.im (only m.room.create)
• !kkjacXiigblOffltrw:gitter.im (only m.room.create)
• !etJKlYVRXEtkkMdEOJ:gitter.im (no events)
• !VeVZEnGasXJXpAYECh:gitter.im (only m.room.create)
• !iJIbqcgSzsAFnwjVBI:gitter.im (no events)
• !koFtykttKGRpjbzflq:gitter.im (no events)
• !mxfIojpZcyQDBbmmoY:gitter.im (only m.room.create)
• !ezxePTXYHMSIkpGOiy:gitter.im (only m.room.create)
• !QlEqqNHuTSIHswllOX:gitter.im (only m.room.create)
• !XInKHVcDAmmyGBEUwr:gitter.im (no events)
• !KIKPFcoDnhBeZhLNUr:gitter.im (only m.room.create)
• !BcBAOZpxjTAvMDfSHf:gitter.im (only m.room.create)
• !DuyTBdNdlhfXoBFNqr:gitter.im (only m.room.create)
• !qgfkmcpByUudrFHTMX:gitter.im (no events)
• !tFwVYIoUNZQhRhjFNO:gitter.im (no events)
• !qoZZIEjvblGKIkqEdj:gitter.im (no events)

---

In terms of moving Gitter forward, I think we can safely reconcile this by removing the bridged room entry in the Gitter database and the Matrix room alias and have them re-created (tracked by https://gitlab.com/gitterHQ/webapp/-/issues/2858#note_1267698515). But I'm unable to remove the Matrix room alias from the old room (M_FORBIDDEN) 😬

And it would still be good to fix the Synapse bugs that allow this to happen.

Steps to reproduce

1. Not sure how this happened but it's all Gitter appservice interactions against Synapse

Homeserver

gitter.im

Synapse Version

1.74.0

Installation Method

I don't know

Database

PostgreSQL

Workers

Multiple workers

Platform

gitter.im (gitter.ems.host) runs on EMS but I don't know the details

Python version: 3.11.1

Configuration

No response

Relevant log output

The only relevant logs for this room from the time of creation is this message:

https://modular-euwest2-kibana.proxy.matrix.org/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-8w,to:now-3w))&_a=(columns:!(_source),index:'67745970-aa2d-11ea-96a7-efc84863d0ea',interval:auto,query:(language:kuery,query:'kubernetes.pod.labels.hostname:%22gitter.ems.host%22%20AND%20message:%22!!gHbYoGxiEuXmahoYeh:gitter.im%22'),sort:!(!('@timestamp',desc)))

2023-01-09 17:33:53,203 - synapse.replication.http.send_events - 149 - INFO - POST-4249- Got batch of events to send, last ID of batch is: $JeDmcg6bYb0s-d6pxIbgmfPaJoChH-qWlnTZ3n8xGdU, sending into room: !gHbYoGxiEuXmahoYeh:gitter.im

---

Database information:

# Connect to the database: https://gitlab.matrix.org/new-vector/internal/-/wikis/EMS-postgres#connecting-to-an-ems-database
$ ems-db-debug gitter.im

select * FROM rooms WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
            room_id            | is_public |         creator          | room_version | has_auth_chain_index
-------------------------------+-----------+--------------------------+--------------+----------------------
 !gHbYoGxiEuXmahoYeh:gitter.im | t         | @gitter-badger:gitter.im | 9            | t

select * FROM room_memberships WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
 event_id | user_id | sender | room_id | membership | forgotten | display_name | avatar_url
----------+---------+--------+---------+------------+-----------+--------------+------------
(0 rows)

select * FROM events WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
 topological_ordering |                   event_id                   |     type      |            room_id            | content | unrecognized_keys | processed | outlier | depth | origin_server_ts |  received_ts  |
   sender          | contains_url |       instance_name       | stream_ordering | state_key | rejection_reason
----------------------+----------------------------------------------+---------------+-------------------------------+---------+-------------------+-----------+---------+-------+------------------+---------------+-------
-------------------+--------------+---------------------------+-----------------+-----------+------------------
                    1 | $JeDmcg6bYb0s-d6pxIbgmfPaJoChH-qWlnTZ3n8xGdU | m.room.create | !gHbYoGxiEuXmahoYeh:gitter.im |         |                   | t         | f       |     1 |    1673285632772 | 1673285633224 | @gitte
r-badger:gitter.im | f            | synapse-event-persister-3 |        94262741 |           |
(1 row)

select * FROM room_aliases WHERE room_id = '!gHbYoGxiEuXmahoYeh:gitter.im';
            room_alias            |            room_id            |         creator
----------------------------------+-------------------------------+--------------------------
 #GreenShoeGarage_Lobby:gitter.im | !gHbYoGxiEuXmahoYeh:gitter.im | @gitter-badger:gitter.im
(1 row)

Anything else that would be useful to know?

No response