Wrong message index when sending an image via Share Extension, message not decryptable in Element-Web, can break rooms when session-ID is affected!

[jacotec]

Steps to reproduce

I've posted the initial issue in Element-Web here: element-hq/element-web#25108

I've seen that sometimes messages can't be decoded in Element-Web. The "Retreieve encryption keys from other device" banner pops up, but does not solve the issue. Neither does clearing the cache. These messages are undecryptable forever in Element-Web and Desktop.

The raw message display in Element-Web shows this error:

"msgtype": "m.bad.encrypted",
    "body": "** Unable to decrypt: DecryptionError: Duplicate message index, possible replay attack: 

I was told in the Element-Web issue that this is a sender issue, so I investigated in this direction. I found that this always happens, when someone sends something (i.e. a picture) to the Element-IOS app via the IOS Share Extension. The very next message after this message can'be be decrypted in Element-Web:

1. Send an image from the Gallery via the Share Extension to an encrypted room
2. Write a text message afterwards into the same room
3. The text message can't be read in Element-Web and Element-Desktop. Error message "Message decryption not possible".
4. The next text message reads fine

This does not happen when the same picture is sent directly from within Element with the "+"-Button.

Maybe the message index is not updated/incremented after sending something via the Share Extension, causing the "Duplicate Index" error in Element-Web?

Element-IOS and Android are decrypting the message (maybe they don't care about duplicated message indices?)

Outcome

What did you expect?

Even after sending something via the IOS Share Extension, the next message shall be decryptable in all access ways (Element-Web, Element-Dsktop)

What happened instead?

The very next message after a message sent via Share Extension can never be decrypted in Element-Web and Element-Desktop, always showing "DecryptionError: Duplicate message index, possible replay attack"

Your phone model

iPhone 14 Pro Max, iPad Pro 10.5

Operating system version

IOS 16.4.1

Application version

Element 1.10.10

Homeserver

Synapse 1.81

Will you send logs?

No

[Anderas]

The root cause of this behaviour is stale cache for outbound group sessions between the main app and share extension, which run as two separate processes on the OS. This means they each need to open their own crypto database and do not share any in-memory cache.

1. Share extension sends an encrypted message, increments the ratched index, and saves changed session back to store
2. Main app resumed from background uses cached session to encrypt the next message, which still has the unchaged index. This causes the second message to have a duplicate index.

Note that this behaviour does not occur if the main app is not running, precisely because there are no values in cache and so the session has to be fetched from the store.

The most straightforward solution is to not use session cache and instead load the session from the store for each encryption. A more error-prone approach is to trigger cache refresh explicitely each time the app is brought to foreground

[jacotec]

Great analysis @Anderas! Thanks for that.
I hope one can develop a fix for this, we're using the share extension quite often.

[borisrunakov]

Occurred also with emoticons sent from iOS devices, on our servers.

[jacotec]

@Anderas As it seems you pretty well understand what's wrong with the Share Extension, another question:
I now face an issue where a direct chat between two users becomes completely broken when user A sends a picture via share extension. All (!) following messages which user A sends in the IOS app are not decryptable by any device of user B after that happened.

When user A sends a message in Element-Web, they are decodable.

For all messages sent by the IOS app of user A, user B gets an error that the messages can't be decrypted due to a wrong session ID (not duplicated index).

Do you think it's possible that the Share extension not only messes up the message index but also this "session ID" (which seems to be worse when it's wrong)? Any idea how to get these back in sync?

[jacotec]

@Velin92 Are there any plans to fix this encryption cache inconsistency in short term? I have daily complaints of my users regarding this, and the Share Extension is heavily used.

[angloromanticism]

this is happening without the "share" extension: sending plain text over iOS app sporadically returns the following when opening on MacOS desktop install:
"type": "m.room.message",
"content": {
"msgtype": "m.bad.encrypted",
"body": "** Unable to decrypt: DecryptionError: Duplicate message index, possible replay attack: "

[jacotec]

@Velin92 Any progress here? Why is this still "minor" as it 100% reproducibly appears with any use of the share extension at any single IOS user.

And there is the great analysis of the root cause by @Anderas ... what else is missing to fix this?

This is a real showstopper issue ...

[jacotec]

Addon: The same issue reproducibly occurs when answering to a message from the Apple Watch via the reply function.

This answer also gets a wrong message index and is not readable on web and Desktop.

[jacotec]

This issue is found long back in different scenario with notification extension. There is an issue in handling data storage and sharing between iOS extensions and the main app. Hope this will be considered in Element X development.

Sorry, wrong approach. I guess Element-X will not make it to a full release this year. This must be fixed in the current Element-IOS.

The chat with my wife fully broke yesterday. /discardsession and the next picture via Share Extension and it makes boom again. That's not acceptable.

[Galbar]

I just want to jump in to emphasize the impact of this bug: We have real users that are getting these errors daily. Some of them have learned to live with it because they have to but deeply hate the app and will invite other users to jump out to other chat apps/platforms when possible. This is no way of growing the user base of matrix.

[jacotec]

@Galbar 100% the same here. The additional mess is that the use of the Share Extension in encrypted rooms seems also to be responsible for the states where the room completely breaks and can be recovered only with a "/discardsession" on both sides. That seems to appear in situations where the cache inconsistency not only gives a wrong message ID but also a wrong (old) session ID because it recently changed in the main app.

@Velin92 @BillCarsonFr As this issue is IMHO responsible for >90% of all issues in encrypted rooms (which are default for DMs) and with the understanding that completely fixing the caching issue might take more time (although it's more than worth it with all the trouble it causes for IOS users) - what about a quick hotfix by simply disabling the session cache and always read from the store until it's finally fixed? Would the performance impact be noticeable at all?

[yoman88111]

The same issue on Android and iOS devices. Every time create a new room is too annoying.

[jacotec]

@yoman88111 What's the relation of room creation to this IOS app issue?

[borisrunakov]

I just want to jump in to emphasize the impact of this bug: We have real users that are getting these errors daily. Some of them have learned to live with it because they have to but deeply hate the app and will invite other users to jump out to other chat apps/platforms when possible. This is no way of growing the user base of matrix.

Same here unfortunately.
We get these issues regularly and there is a heated debate on why we should switch to other platform.

[sbrooke]

Well, guess it's a good thing it's not just me. Unfortunately, the bug ticket I came here from was closed. Good thing because it was opened 2 years ago! It's curious why this hasn't been fixed in 2 years.

[leematos]

@jacotec -- Do you have a workaround to fix this message index error via Share extension? I have a busted DM and I'd like to "repair" it, but I'm not sure if deleting the message/index will fix it?

[jacotec]

@leematos Try to issue the command

/discardsession

into the DM. Should be done by both sides.

[toshanmugaraj]

"msgtype": "m.bad.encrypted",
    "body": "** Unable to decrypt: DecryptionError: Duplicate message index, possible replay attack: 

@manuroe if duplicate message index leads to disabling the share extension, can we disable the replay attack validation in iOS ? because share feature is essential.

[maxkratz]

@manuroe if duplicate message index leads to disabling the share extension, can we disable the replay attack validation in iOS ? because share feature is essential.

I don't know if default disabling a security measure is a good idea. Maybe it can be made optional via a setting in the app?

[toshanmugaraj]

@manuroe if duplicate message index leads to disabling the share extension, can we disable the replay attack validation in iOS ? because share feature is essential.

I don't know if default disabling a security measure is a good idea. Maybe it can be made optional via a setting in the app?

yes can be optional

[toshanmugaraj]

What is the point in failing to decrypt in a replay attack. It can just highlight the user with warning that the message can be a replay attack.