Stale device lists

[richvdh]

STR:

1. Alice and Bob share an e2e room
2. Bob leaves all of the e2e rooms he shares with Alice
3. Bob logs in on a new device
4. Bob joins (the same, or a different) e2e room with Alice
5. Alice sends a message
6. Bob sees "Unknown inbound session ID"

The problem here is that Alice's client is using a stale device list for Bob.

It's a bit of an edge case, since it requires doing a complicated dance of leaving and joining rooms and logging in on new devices - but it's an edge case that people tend to hit during testing.

[richvdh]

Some possible strategies for fixing:

1. Force a device list refresh whenever someone joins an e2e room
2. The above, but only if we don't already share an e2e room with them
3. Do the above lazily somehow: store a flag which means we think the device list might be stale

[richvdh]

This also happens even if Bob and Alice have never shared a room; Alice's client may still have cached an old list if she opened Bob's memberinfo.

[giordy]

I confirm that this happens pretty frequently in my experience.
We are using Riot in 2 people from the web and from the app (both iOS and Android).

[richvdh]

@giordy: this bug is something of an edge-case; it is unlikely you are seeing exactly this.

There are several potential causes of "Unknown inbound session ID" (many of them currently related to work still to be done on the mobile clients).

[giordy]

@richvdh ok thanks for the clarification!

[richvdh]

This is slightly tricky to fix from Alice's end. She can listen for new members while she is online, and do a device list download when she sees one, but that doesn't help when she is offline. During an intialsync, she can't tell the users who joined since she was last online from those who were there before (and she should thus expect to have an up-to-date device list for, or new_device events for).

Another idea would be to send a new_device event to all existing members when you join a room, thus prompting them to download your device list when they come online. However, this relies on the join events being well-ordered, which they are not (Alice and Bob could join at the same time on different servers; each of them see the room without the other when they first join; neither sends the other new_device events, expecting the other to sort it out).

We could just refresh the devicelist for all users in all encrypted rooms at the end of intialsync. This might be a bit high-traffic.

We could be a bit more intelligent in our caching, and store the timestamp of the devicelist response; then refresh the devicelist if we see join events (or new_device events) after that time. Probably requires server support to give us a timestamp, though.

[richvdh]

I've been thinking about this a bit more. I've realised the current method of sending m.new_device events to everyone you think is in the room when you add a new device is inherently racy. Consider:

• Bob is an existing member of the room
• Alice joins the room, and downloads Bob's device list (or fails to download it, but doesn't mark the cache as invalid).
• Alice's join event takes a while to reach Bob. Meanwhile, he adds a new device. He doesn't send an m.new_device to Alice, because he doesn't know she is a member.
• Bob finally sees Alice's join event.

Alternatively:

• Alice and Bob join a room at a similar time. Alice sees Bob's join event, but Bob does not (yet) see Alice's.
• Currently this doesn't work, but let's assume Alice realises that Bob is a new member, and refreshes his device list.
• Bob adds a new device, but because he doesn't realise that Alice is a member, doesn't send her a new_device.
• Bob finally sees Alice's join event.

In either of these situations, Alice has a stale copy of Bob's device list. Worse, this situation persists until Alice has cause to refresh the device list.

The fundamental problem here is that the m.new_device mechanism relies on there being a well-ordering of join events. Even treating join events as cache invalidation indicators (which is tricky to do efficiently on initialsync, as above) doesn't fully resolve this problem.

One solution would be to record, in the room state, a sequence number (could be a timestamp, any monotonically increasing number will do) which increases each time you update the device list. Similarly, the device list results would contain a corresponding sequence number, which you record alongside your cache. When you see that a member's device sequence state is higher than your cached version, you know the cache is out of date.

[richvdh]

I discussed my proposed solution with Erik. He suggested that:

• using room state in this way is not ideal, due to the duplication between rooms, and because rollback in room state (due to auth failures etc) could make it unreliable. We could instead use a similar mechanism to presence updates (which he insists already has hooks to send updates when we see a new server join a room).
• the use of a sequence number implies a lock, which it would be preferable to avoid. We could instead send the latest device list over federation. I remain slightly unclear on how client-side caches could be maintained in a way that doesn't end up just having locks elsewhere, but will take Erik's word on this for now.

[richvdh]

Modulo #3126, this should be fixed as of riot v0.9.7 and synapse v0.19.0.

[richvdh]

Still broken. We don't update the device list of users when they join an e2e room. I think we need to update the device list when we first see the user join an e2e room.

To do this right, we probably need to maintain a flag which tracks whether we are keeping the device list for a given user up-to-date.

[richvdh]

Hopefully finally fixed by matrix-org/matrix-js-sdk#425