Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing E2E encryption keys didn't decrypt chat history. #9450

Closed
xloem opened this issue Apr 12, 2019 · 9 comments
Closed

Importing E2E encryption keys didn't decrypt chat history. #9450

xloem opened this issue Apr 12, 2019 · 9 comments

Comments

@xloem
Copy link

xloem commented Apr 12, 2019

After the recent mass logout, I logged in and my history was gone. I set up new keys and enabled backup for the first time.

Then after some hopeful realization, I disconnected another system from the internet, logged into my account before it was logged out, and exported the room keys.

I then connected the system to the internet, rebooted, and after logging in imported these keys ... but I still can't see my chat history.

@n3niu
Copy link

n3niu commented Apr 14, 2019

Same here.

Did the export of the encryption keys long before the current security incident back in 2017 with the same user and pwd I'm using now (not sure if this matters). Importing the keys in all of the three clients mainly used went technically fine as the passphrase was accepted and no errors showed up.

However, chat history still shows the "Unable to decrypt" message in all clients.

@cyphar
Copy link

cyphar commented Apr 14, 2019

@n3niu Session keys are rotated every ~100 messages or 7 days (whichever comes first) for each room. This means that if you don't use the automated homeserver-based key backup, and instead use the key export feature, you need to be exporting your keys very often (otherwise you would only be able to decrypt very old messages).

This should almost certainly be made more clear, but I would say most users should be using the homeserver-based key backups (which work using an ephemeral Curve25519 key -- if you don't enable password-based backups you can store the Curve25519 private key and not need to do anything extra).

@xloem
Copy link
Author

xloem commented Apr 14, 2019

My keys are recent, recovered from a second system before it was logged out. I backed up the config dir and can still see the unencrypted messages if I restore that backup, but of course can't use the client for anything without logging out. I sent my debug log from an attempt to restore, to this issue.

@xloem
Copy link
Author

xloem commented Apr 14, 2019

@cyphar would you be able to give enough information for somebody familiar with javascript to use the development console to save what is needed from an old functioning client, and restore this stuff to a new logged-out client? somebody with less on their hands than you guys likely do could flush the detailed steps out for others.

@n3niu
Copy link

n3niu commented Apr 14, 2019

This should almost certainly be made more clear, but I would say most users should be using the homeserver-based key backups (which work using an ephemeral Curve25519 key -- if you don't enable password-based backups you can store the Curve25519 private key and not need to do anything extra).

Thanks @cyphar for clarifying this. Going back into chat history I see that messages between Dec 2016 and Feb 2017 are actually decrypted.

And, yes - this should be made more clear as I did not read about this behaviour before somewhere. Maybe someone can point me to the related docs in case I've missed it. Afaik back in those days the server based key backup did not even exist (could be wrong).

Looks like my chat hist is gone for good even though I thought I would be safe still having the keys in my hand.

Wishing you more luck @xloem

@cyphar
Copy link

cyphar commented Apr 14, 2019

Afaik back in those days the server based key backup did not even exist (could be wrong).

Yeah, it's unfortunately a very recent feature (only added a few months ago). I would suggest enabling it now if you haven't already.

@cyphar
Copy link

cyphar commented Apr 15, 2019

@xloem Unfortunately I'm not a Matrix developer, nor am I a JavaScript developer.

However, I would suggesting looking at the matrix-react-sdk (which is what riot-web is based on). A quick look suggests to me you'd want to try to trigger matrixClient.exportRoomKeys() -- and then once you have the keys you can pass them to encryptMegolmKeyfile() (after JSON-ifying the key list) which will then give you a file you can then import into a Riot client.

EDIT: I think I've figured it out. I was able to get the keys from a current session with:

>>> keys = await this.mxMatrixClientPeg.matrixClient.exportRoomKeys();

And then you just need to generate the backup from this. Unfortunately I can't figure out how to call encryptMegolmKeyfile(). Instead you could just copy the JSON-ified version of keys into megolm_backup.py which is a small script I wrote that allows you to create a new Megolm session backup (or decrypt an existing one -- which helps with working around element-hq/element-meta#1287).

>>> keys_json = JSON.stringify(keys)

% ./megolm_backup.py --into >riot-keys.txt
[ paste your JSON here ]
Backup passphrase [mode=encrypt]:

And then you can import riot-keys.txt using the passphrase you entered. Make sure to do this on a machine where you aren't worried about your clipboard being intercepted.

NOTE: Most terminals won't allow you to paste more than 4096 characters in a single line. You can fix this with stty -icanon but then you can't Ctrl-D to stop giving input -- so it might make more sense to save the plaintext keys to a file and then do

% ./megolm_backup.py --into <plaintext-keys.txt >riot-keys.txt

But obviously make sure you shred the file after you've done this!

@fuzzyTew
Copy link

fuzzyTew commented Apr 15, 2019

cyphar, that information sounds so helpful for people needing to export their keys from old client backups. Is there a centralized place to put it?

I'm still trying to import my keys. Turns out they are successfully loading, but they aren't being used. This is the console log:

bundle.js:41 E2e import/export: deriveKeys took 349ms
bundle.js:41 Update for megolm session Av1YgpPK8LWKQ80UdKdrf3mcIHmlYRftMh8YI5F7Mwg/IWfY5xq31hIbZjXmAL83iFBf/UpKFTuTppdjfVxHqA4
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session Av1YgpPK8LWKQ80UdKdrf3mcIHmlYRftMh8YI5F7Mwg/P+WUt+rvwE8JI3YaOa9TxHCGUzCBYSIMMJUf/Cj/240
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0//wRqp93ebJI8Rz59b3Shwd7dsIjAVTj2Z00tyYwXjhk
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/2F/q+hrNTYibObhGfrs8xa1+OZ1YgZz3B52FzBnO0bA
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/B2IQaUHVxAjwvOIVJsCb8Sx3LiVO7l1+gY74QoTEjAg
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/GXJdqYOcSQxPPrQbwkXL8g19ZKHv/Arn0BSANoLhqvk
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/HnWEZYKBFNIuXko9kF3HV5NzQ8Y6HWKvvhixVSok5p8
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/NtsiiUlYC36TsoA9cxSpU9WxY3UZzdZDbXLeN1DYlRA
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/P/FCQNGuxLrvZ9sYZxXPMjKvUpDwB08rOBxA7sXjIuA
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/UnRbmcp+z0SKP8fQGGuvnKd6aDW3qFdNUB0Hs5aQrgE
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/bBzdImOOVRKl9Njd9EhtmOjY/JyzLTex9Zz78pcdUPo
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/krI5kN1V1EKPW29NQiaYJGkke3+5xMpQmVh+8ua38XI
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/srVQywrWhbhXgrF6IE/mzG+sVSoqZWLdB/wfD+Livjk
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session KAUaOxlmjCIoErIaGtw+YMZCftxXCN+BFWGcQWOYxA0/yO+jGCHASZjiRA2rHY4s3jdIuteDd6Cg9u8ioLdirFY
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/B6YmX0cC3LOIunUH6BwbThruuKi8UgE1dQGaurwFKV0
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/L2YRKhLeLGJzat0mhKV9KisQeFsvbdC4DQvC8Tpcye4
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/QgMbJKH+DO5UboubMeYkHBD1XmbiMY2m+0Sm5SGZI28
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/azBUNyJULpMoxs3mhLVZV/77KdhGqXc4scScuEzfWv0
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/p6R+qed2Ji1R8vkmjnRJbEsKoQq85i4H2bVL3jUq4Ig
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/ppYIj2RbxeTL8nX2eiBmr2QlwRpI20nrYzPkc6ev4yc
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/qBzdgpK3OxfV+aRsb40CUvf36UihmaPtSvUKeBIC0Vg
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/s8R3Kzw2nkaBy2ANVCq4cf4A8Z1ET2jDJJZWwJBR5Yo
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session OLrR7GsudBw/tPjAjB0S4yc0c95N4SG14PNFuEhjEgc/tWbF14coUuEJQuQU7Uk5gjkE41yomekhrStckMZM2Z0
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session PW/xXJqLZrPgjt5Aifv+MTt0hReyCt/jMGe2iGLj4yQ//WHG0aVZCuXLz/IzYM70DIuCpIFFwQ3v/qqh5y0tAu0
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session PW/xXJqLZrPgjt5Aifv+MTt0hReyCt/jMGe2iGLj4yQ/emwq9G3hayna8/A6yPfWibSfp9+FQ1SBRqCJhV4LpcE
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session PW/xXJqLZrPgjt5Aifv+MTt0hReyCt/jMGe2iGLj4yQ/g7ka0Pfl45xNRZSsW2bM/QOUJVC1VnBLYv/oVjeG7I4
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session PW/xXJqLZrPgjt5Aifv+MTt0hReyCt/jMGe2iGLj4yQ/rpsKhsiPCGdCeb6FNAYq5TrBI99H/vOU8IllXvZj+ds
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session PW/xXJqLZrPgjt5Aifv+MTt0hReyCt/jMGe2iGLj4yQ/wp25mXvyfgbPkJMFksDForI611uTguMWwAe7DFOPYu4
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session PW/xXJqLZrPgjt5Aifv+MTt0hReyCt/jMGe2iGLj4yQ/xCMGQpfkTpHV3l27xKQt1wjOY2C0NYqMIKM2H/MCG1Q
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session jtTVmZdRI9Jevep6bZ+xkrCtYNoms/vp2s7M4TSSgnA//yMLptJei99pkdWHPJrPpyp4pvA6YuDq/FeyStcf9oo
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session jtTVmZdRI9Jevep6bZ+xkrCtYNoms/vp2s7M4TSSgnA/7S7CJTGqmWDHHarjrMm77MOautzdJtdmdFayT1m8DrY
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session jtTVmZdRI9Jevep6bZ+xkrCtYNoms/vp2s7M4TSSgnA/OcTTuSXLnJzerV6zrjWCZ2lJkzX8teiHUVihW+HN07M
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session jtTVmZdRI9Jevep6bZ+xkrCtYNoms/vp2s7M4TSSgnA/ZroFIEKZpssEef5lfIKPCzJ4fFrVxuA0NLB8OtlR54E
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session jtTVmZdRI9Jevep6bZ+xkrCtYNoms/vp2s7M4TSSgnA/aFHuvJR0feM+a2ohAmuj/61fmU0QGpP8cJQL4yVFi/w
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session jtTVmZdRI9Jevep6bZ+xkrCtYNoms/vp2s7M4TSSgnA/nagDHJu3L6LI4Zl0AbpExIfCON06nBfsAKg1v/G0coM
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session jtTVmZdRI9Jevep6bZ+xkrCtYNoms/vp2s7M4TSSgnA/uWjiaBqv1MXUh9x485jE3sp3ScQTBYDwkIYG10Kj95w
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session o1j2S6KQz7V/kLk1CgnY/jp17lnvt2vjH1CIFu8cGwo//Dd/GcvsDGnzVJ6JJpX9QhRqVNgNX1/l+4HRX1rYmHY
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session o1j2S6KQz7V/kLk1CgnY/jp17lnvt2vjH1CIFu8cGwo/IRVfgghEqHJ95WCpFmAQVgEs7dyY5jEKgbnZKPDps1g
bundle.js:41 Keeping existing session
bundle.js:41 Update for megolm session o1j2S6KQz7V/kLk1CgnY/jp17lnvt2vjH1CIFu8cGwo/tfWvq/jO9qtBG93udikXv8aS2I2DvvWYqsr1D+O1W+w
bundle.js:41 Keeping existing session

(EDIT: oops, this was xloem)

@kittykat
Copy link
Contributor

I'm going to close this issue for now as we cannot reproduce it. Please comment with @kittykat if you are still experiencing it and I will reopen the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants