Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explain manual and automated key backup better #8751

Open
aaronraimist opened this issue Feb 17, 2019 · 53 comments
Open

Explain manual and automated key backup better #8751

aaronraimist opened this issue Feb 17, 2019 · 53 comments
Labels
A-E2EE A-E2EE-Key-Backup O-Frequent Affects or can be seen by most users regularly or impacts most users' first experience S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Something isn't working: bugs, crashes, hangs and other reported problems T-Enhancement New features, changes in functionality, performance boosts, user-facing improvements

Comments

@aaronraimist
Copy link
Collaborator

aaronraimist commented Feb 17, 2019

Lots of people tend to think manual key backup is something like GPG, ie if you export the key as soon as you create your account you'll be able to decrypt all future messages that are sent after you exported the keys.

@aaronraimist
Copy link
Collaborator Author

aaronraimist commented Feb 17, 2019

It also needs to be more clear that key backups to the server are stored on your homeserver. Not some central matrix.org server.

@drathir
Copy link

drathir commented Feb 19, 2019

Agree clarify of remote storage in use would be nice in description, also taking on mind if that some kind of private kinda keys would be nice as well get possibility of gpg/passphrase/one time paste style send to email address protection type i guess (theoretically speaking only, bc not checked/searched for type of data send by this feature).

@matthijskooijman
Copy link

matthijskooijman commented Feb 22, 2019

I just tried setting up automatic key backup and found that there's a lot of stuff that's not really well-explained through the UI. Here's some questions that crossed my mind:

  • What is stored on the server exactly? A backup of my room keys, I suppose. Encrypted using a recovery key? Or my passphrase? Or both?
  • Where is the backup decrypted? I assume in the browser, so the server never sees the actual keys? Of course, a malicious server could still easily inject malicious javascript to sniff the passphrase and keys, of course...
  • How do the recovery key and the passphrase correspond to each other? It seems these are basically equivalent in how they can be used? Or does the passphrase normally encrypt the recovery key?
  • How does key backup work with multiple devices? Normally, multiple devices seem to have their own E2E keys I believe? Does key backup allow sharing a single set of keys among devices? Or will they just have their own parallel backups? How does this interact with the key sharing feature?
  • Does the recovery key ever change? I've tried a some things with different browsers (also coming back to existing browsers) and found that I got different keys.
  • As long as your browser (or other device using key backup) remains logged in, you can always access the original keys, and also setup new backups (even if you forgot your passphrase or recovery keys), right?
  • When I enter a passphrase or recovery key, is it stored in the browser? I guess it should (or perhaps an underlying backup encryption key) since it must be able to add new keys to the backup later?
  • What does it mean for a backup to (not) have a signature from a device?
  • How are backup versions handled? Does any change to the backup (content / keys / etc) result in a new version?

I don't think all of these questions should be answered in the interface directly, but perhaps the UI could be changed to support a more useful mental model of what goes on in the background, and perhaps also link to more complete documentation (I just found https://about.riot.im/help#end-to-end-encryption which has some info already).

@lampholder
Copy link
Member

lampholder commented Feb 26, 2019

Agreed - we're trying to strike a balance between informative and overwhelming, and to provide enough information to reassure encryption/privacy novices, experts and everyone in between.

This list of questions is very useful - we can take another look at the wording/UX with this in mind and see what we can change to answer more of these questions as you work your way through setup.

@lampholder lampholder added T-Defect Something isn't working: bugs, crashes, hangs and other reported problems ui/ux P1 [OBSOLETE LABEL] Next S-Major Severely degrades major functionality or product features, with no satisfactory workaround A-E2EE T-Enhancement New features, changes in functionality, performance boosts, user-facing improvements A-E2EE-Key-Backup labels Feb 26, 2019
@turt2live
Copy link
Member

turt2live commented Apr 15, 2019

This is becoming more prevalent thanks to recent events. There's numerous reports of people taking their exported keys from months/years ago and trying to import them, expecting that to work. We should fix the messaging so that in N years we don't have to tell people that export !== backup.

@cyphar
Copy link

cyphar commented Apr 16, 2019

I will admit when I first turned on key backups I went to read the MSC to make sure I understood how the system actually worked -- which is far from ideal if we want the security of the system to be self-apparent (as an aside I think the design is really brilliant). I think the simplest way to improve people's understanding of how key backups work is to apply a couple of helpful tips (which might help some of @matthijskooijman's questions):

  • Add a (probably red) warning to the key export feature explaining that in encrypted rooms the session keys will roll over quite regularly and thus exported keys will not be useful for decrypting future messages soon after you've done the export. This is the biggest mistake I've seen and it's a shame we didn't have better text on this before (I know I've made this mistake before -- and only realised when I noticed the key exports were getting larger each time I did an export).

    • There should also be a button for using key backup on the key export page to hopefully explain that key export requires more work on the users' side. Most of the text on key export could probably be dropped to make it more likely people will read it.
  • Change the key backup setup flow, so that you first get the recovery key and then are asked for the passphrase (with some text on the passphrase prompt saying that it's optional -- but recommended -- that the recovery key be encrypted and saved to the homeserver so that the user doesn't have to keep track of it). This might help users understand that the recovery key is the primary recovery system and the passphrase is just there to not require you to keep track of the recovery key (the current text is "We'll store an encrypted copy of your keys on our server. Protect your backup with a passphrase to keep it secure." which is quite misleadingly-written -- arguably it's more secure to not set a passphrase since then the private key never hits the server).

  • I would suggest having a (skippable) "getting started" screen for enabling e2e encryption which might explain some of the questions that @matthijskooijman mentioned above. Since the plan is to make all private rooms e2e-only we should probably work on the onboarding flow now rather than later.

    • I think there is a fairly good "intuitive" way of explaining how key backups work (you could even do it graphically to help get the point across) -- padlocks and keys. When you set up key backups, you create a new key and a large number of corresponding padlocks. You keep this key safe, and whenever you're chatting with someone and you get a new session key you put it inside a safe box and lock it with a padlock and send it to the homeserver -- you only need the recovery key when you need to do a recovery. I bet that a lot of confusion could be avoided with some picture-based explanations (something like the Keybase art style would be great IMHO).

@jryans
Copy link
Collaborator

jryans commented Apr 17, 2019

Perhaps we could restructure some of our in-app messaging to show first the simplified explanation for everyone, but then add a disclosure something like:

More details Advanced encryption explanation details go here.

so that we can still offer more detail for those who want without overwhelming those you don't.

@cyphar
Copy link

cyphar commented Apr 17, 2019

That would be a good stop-gap to at least provide some sort of information to those who should be aware of things, but I think that all the E2E messaging needs to be made layperson-friendly if we want people to turn it on (or make it the default) without being surprised when the last 12 months of chat history with their loved ones has gone up in smoke (I'm a technical user and this happened to me, but it's a painful mistake to make).

I'm aware (and really grateful) that the technical side of this is being worked on very hard (and the results speak for themselves -- the new e2e features are really top-notch), but making a good on-boarding experience with encryption (or even Matrix in general) would really help users avoid some pitfalls.

I'm fairly convinced the best shot of making it reasonable is to do it graphically (I'm really not an artist but I could knock up some mock-ups to hopefully show why graphical explanations would help). Unfortunately you'd need a designer to actually make them usable -- and they are painfully hard to find within free software communities (despite how desperately we need them in most free software projects).

@35609902357
Copy link

35609902357 commented Feb 25, 2020

@cyphar

it's more secure to not set a passphrase since then the private key never hits the server

Is this a thing embedded into the client (such as Riot) or does this happen at the protocol level (Matrix)?
Is it possible to delete the backup from the home server once the passphrase has been inserted by removing the passphrase or are there any methods to do it?

@cyphar
Copy link

cyphar commented Feb 26, 2020

Is this a thing embedded into the client (such as Riot) or does this happen at the protocol level (Matrix)?

The protocol defines how key backups work. Basically, your "recovery key" is the private half of an X25519 keypair. All new session keys get encrypted by clients using the public key (so you only need the private half when you first import your keys). Optionally, you can symmetrically encrypt the X25519 private key with a passphrase, and then that encrypted private key gets stored on the server. But since the private key is generated on your client, if you don't set a passphrase then the server never sees the private key.

Is it possible to delete the backup from the home server once the passphrase has been inserted by removing the passphrase or are there any methods to do it?

There's a "delete backup" button in the key backup settings. I don't think you can currently delete only the passphrase (while keeping the backup), but if you delete the backup and then create a new one it will contain all of your keys (and you can not add a passphrase).

@35609902357
Copy link

35609902357 commented Mar 31, 2020

@cyphar I'm having trouble understanding how it works. Whether I encrypt the backup with a passphrase or skip the passphrase and go directly for the recovery key, the backup status shows Backup key stored: not stored in both cases, but I figure it should say it's stored if I input a passphrase, thus saving it on the homeserver, why is it not the case?

Moreover when restoring a backup protected with a passphrase I get the passphrase prompt, but if I can't remember it I can skip to the recovery key without providing the passphrase, but then where does it retrieve the backup from? If it's on the device's internal storage then skipping the passphrase and inserting the recovery key on another device added for the first time should not recover any key, is this correct?

@cyphar
Copy link

cyphar commented Mar 31, 2020

@35609902357

but I figure it should say it's stored if I input a passphrase, thus saving it on the homeserver, why is it not the case?

I think it should say that too, that might be a bug, I'm not sure.

Moreover when restoring a backup protected with a passphrase I get the passphrase prompt, but if I can't remember it I can skip to the recovery key without providing the passphrase, but then where does it retrieve the backup from?

The actual session keys (the keys you are backing up) are always stored on your homeserver, encrypted using a asymmetric Curve25519 keypair. If you enter the "recovery key" you are actually entering the private key for a Curve25519 keypair (the server keeps the public half, and gives it to your devices -- so they can add entries to your key backup without storing or prompting for the private key). However if you add a passphrase then your homeserver also stores a copy of the Curve25519 private key (the "recovery key"), encrypted with the passphrase you chose.

From my understanding this design has changed somewhat since the introduction of SSSS (which is how cross-signing has been implemented), but the broad strokes should be the same.

@35609902357
Copy link

35609902357 commented Mar 31, 2020

@cyphar
So inserting a passphrase and saving the private key on the homeserver simply means adds the convenience of choosing the passphrase to restore backups instead of saving the random recovery key?

And that IF you choose a strong passphrase the history is safe, but if the passphrase is weak then you might potentially compromise the security of all your messages?

So if I lose access to all my devices, and I only have my passphrases, I add another device without any trace of previous local activities/backups, would I be able to recover all my encrypted history by only providing the recovery key? Would the only difference be inserting the recovery key by myself instead of inserting the passphrase to decrypt it and letting the homeserver insert it for me?

@cyphar
Copy link

cyphar commented Mar 31, 2020

@35609902357

So inserting a passphrase and saving the private key on the homeserver simply means adds the convenience of choosing the passphrase to restore backups instead of saving the random recovery key? And that IF you choose a strong passphrase the history is safe, but if the passphrase is weak then you might potentially compromise the security of all your messages?

Yes, though in order to exploit this you'd need to either be a homeserver owner or have the users' password to get access to the key backup.

So if I lose access to all my devices, and I only have my passphrases, I add another device without any trace of previous local activities/backups, would I be able to recover all my encrypted history by only providing the recovery key?

Yes. This is what I do, and I have lost my devices before (both before and after the introduction of key backup -- it was much easier to deal with when key backup was added).

Would the only difference be inserting the recovery key by myself instead of inserting the passphrase to decrypt it and letting the homeserver insert it for me?

Yes, though the server doesn't do the decryption of the recovery key or key backup for you (for obvious reasons) -- it's done client-side. Aside from that, that is correct.

@35609902357
Copy link

35609902357 commented Mar 31, 2020

@cyphar
Then it sounds like a silly feature to have, not only it's a waste of coding effort, but it can only endanger the security of the backup because a passphrase secure at least as the recovery key will most likely have to be stored in a password manager (or other means) anyway, and the average Joe will certainly choose a weak passphrase. So the whole thing sounds entirely like a detriment and should not be explained better, but totally removed. Or are there any upsides I'm not seeing to having it?

@ara4n
Copy link
Member

ara4n commented Apr 7, 2020

So the whole thing sounds entirely like a detriment and should not be explained better, but totally removed. Or are there any upsides I'm not seeing to having it?

It's hard to guess what you're not seeing.

The pros are:

  • You can recover all your message keys if you lose all your devices (particularly easy if you only have one device).
  • You don't have to rely on manually exporting your message keys.

The cons are:

  • If you choose a weak passphrase, someone with access to the backup (a server admin, or someone who has compromised your account password) can get all your message keys.

It's not clear why you think that storing a recovery key or recovery passphrase in a password manager is a bad thing.

@cyphar
Copy link

cyphar commented Apr 7, 2020

@ara4n To be fair, they're arguing that recovery passphrases are a "bad feature" because if the user could pick a strong passphrase they're probably using a password manager and thus could just store the recovery key. If they can't store a recovery key, then their recovery passphrase is probably weak and we shouldn't encourage them to use passwords if they're likely to be weak. Not to mention that most users (sadly) do not use password managers.

My view is that it's a trade-off which is entirely up to the user, and the pain of losing messages with your loved ones is much more likely to disenfranchise users than any system which permits weak passwords. (Not to mention losing your messages was basically guaranteed with the old key backup system, because you had to constantly keep backing it up -- something that even more technical users like myself managed to misunderstand.)

Personally I think we could do more to make authentication less vulnerable to bad passwords (such as having 2FA support 😉), but this is perhaps an example where explaining to users that the password is optional (and should be very strong) but that it requires more work on their side is probably a good idea.

@ara4n
Copy link
Member

ara4n commented Apr 7, 2020

oh, okay. yeah, we could forbid recovery passphrases and just mandate recovery keys.

@35609902357
Copy link

35609902357 commented Apr 8, 2020

@ara4n I apologize if I came across as dismissive, it was not my intention. @cyphar made an excellent job explaining my concern and providing the reason I was not able to see, that there are users who care more about preserving message history than security. Then yes, it makes sense passphrases should stay, but should be explained better to make as clear as possible the risks stemming from a weak passphrase, and above all the fact that providing the passphrase itself is not mandatory, nor added security and that it means the recovery key will touch the server if provided. It should also be the other way around, the recovery key should be provided at first with a secondary toggle to enable passphrase backup.

@bigCrash
Copy link

bigCrash commented May 20, 2020

A little bit OT, but I found no other place and it is more or less a documentation request regarding the recovery key, so here comes my question.

If I change the recovery key, do I keep my message history as long as at least 1 of my devices is logged in? Can this device "share" the history of encrypted chats with the new recovery key?

I am asking since I changed my recovery key and subsequently my other devices were classified as non-trusted (e.g. cross signing for those devices was disabled).

This does not make sense to me since I would assume that the respective encryption and cross-sigining keys are already on those other devices and only the backup of those keys (on the homeserver) would be updated with the new recovery key.

@aaronraimist
Copy link
Collaborator Author

aaronraimist commented Jul 19, 2020

Related: #14421

@LinAGKar
Copy link
Contributor

LinAGKar commented Aug 24, 2020

It sounds too complicated for normies, probably that feature can't be implemented in a way that's both safe and easy, then just forbidding the users to use the account password for key backup should at least avoid disasters in case of password leaks. Then when cross-signing will be properly implemented it will further reduce chances of losing data.

The same thing I suggested is already used by Firefox sync: https://hacks.mozilla.org/2018/11/firefox-sync-privacy/

@LinAGKar
Copy link
Contributor

LinAGKar commented Aug 25, 2020

And deriving both the account key and the recovery key from the user entered password would allow completely getting rid of recovery key management and device verification, since it could establish trust using the derived recovery key just logging in, which would significantly reduce friction for new users. Having to explicitly verify devices to establish trust could then be an advanced option most users wouldn't need.

Add TOFU to that, and most people would never to bother with device verification.

@t3chguy
Copy link
Member

t3chguy commented Aug 25, 2020

@LinAGKar it would also be incompatible with LDAP auth and SSO/SAML

@LinAGKar
Copy link
Contributor

LinAGKar commented Dec 8, 2020

Apparently they are planning to allow using a single password for both: https://github.com/vector-im/roadmap/projects/1#card-48806427

@c33s
Copy link

c33s commented Dec 8, 2020

do we really want our private keys on a server only protected by a password where the eu is planning to install backdoors in our end to end encryption?

i would really love to see good local backups or support to backup to an ssh or webdav server.

for crypto currency we use hardware wallets and say "not your keys not your coins" and in communication it is ok to auto backup the keys to the cloud without a different option?

@t3chguy
Copy link
Member

t3chguy commented Dec 8, 2020

You are not at all forced to backup your keys to the cloud. You have manual export functions inside Element. Keep in mind that due to Perfect Forward Secrecy - https://en.wikipedia.org/wiki/Forward_secrecy - you will need to backup very frequently if you do not want to lose your messages.

Element as a thing which runs inside a web environment cannot make SSH connections so that's simply out of the question.

@LinAGKar
Copy link
Contributor

LinAGKar commented Dec 8, 2020

What about automatic backup to a local file (which can then be synced/backed up some other way)?

Also, WebDAV ought to be simple for a web app, and SSH (presumably SFTP) should be doable in the desktop app, albeit not in a web browser.

@c33s
Copy link

c33s commented Dec 8, 2020

nag screen

well i felt really forced to use cloud backup with this nag screen comming up every time i started riot (where i asked for clear description in the app #13685).
i would love to have a local auto-backup which saves me from loosing access to my message history (*) because after my harddisk ran full i simply lost my keys/access to all my message history #15788 (i assume that it happend because of key rotation but why keys are getting rotated if no space is left on the device? (freeing up space and restarting the app has not brought back my messages/keys))

*: i really miss a feature to permanent decrypt a message locally. i someone is on my system where they keys reside the encrypted db wont really help

@t3chguy
Copy link
Member

t3chguy commented Dec 8, 2020

What about automatic backup to a local file (which can then be synced/backed up some other way)?

Web apps can't automatically write to your files, you can only trigger file downloads upon an explicit user action.

why keys are getting rotated if no space is left on the device

Web apps can't check how much space is left on your device.

@c33s
Copy link

c33s commented Dec 8, 2020

does both of your comments also apply to the desktop app (i only use the desktop and there is no issue tracker for the desktop app)?

Web apps can't automatically write to your files, you can only trigger file downloads upon an explicit user action.

maybe trigger onclick & onfocus & onmessage ,... with enough trigger which trigger a "cron manager" which backups the keys around every x days? of course it still can happen that no backup occurs but it is still better than no backup (or remote backup)

Web apps can't check how much space is left on your device.

oh. that's really bad but then there really should be a much better, for not tech people understandable info what no backup means and what can happen if no manual backup is made.
as it's a webapp there is also an option to allow api acces where i can trigger the download/an interaction by a local cron job.

@t3chguy
Copy link
Member

t3chguy commented Dec 8, 2020

does both of your comments also apply to the desktop app (i only use the desktop and there is no issue tracker for the desktop app)?

In the desktop app these limitations are not present but require much more additional work.

maybe trigger onclick & onfocus & onmessage ,... with enough trigger which trigger a "cron manager" which backups the keys around every x days? of course it still can happen that no backup occurs but it is still better than no backup (or remote backup)

The user experience this creates will be awful, the app randomly downloading files and your Downloads folder having a lot of random looking files.

@LinAGKar
Copy link
Contributor

LinAGKar commented Dec 8, 2020

maybe trigger onclick & onfocus & onmessage ,... with enough trigger which trigger a "cron manager" which backups the keys around every x days? of course it still can happen that no backup occurs but it is still better than no backup (or remote backup)

I'm pretty sure you can trigger a file download autmatically from JavaScript, but the user would get a file download dialog each time. It can't be hidden. The desktop app should be able to backup to a local file file automatically (or over SSH, though that seems redundant), but in a web browser the simplest option would probably be WebDAV.

If you're running in a web browser, your already have to trust whoever serves the client though, since they can just add something to it at any time.

@c33s
Copy link

c33s commented Dec 8, 2020

first of all thank you at @t3chguy, i haven't seen such a good community manager and supporter in all the years. your answers come fast and are technically backedup. your endless patience and knowhow are awesome. big applause for you here and again thank you.


to clearify i am talking about the desktop app only. i don't use the webapp in the browser.

In the desktop app these limitations are not present but require much more additional work.

it would be very good to have at least a "space left on device"-check. killing the keys because of missing device space is really a terrible UX. i don't know how the key rotation work at all but a full harddisk should never lead to the deletion of existing keys, so at least all existing messages should be accessible/decryptable.

@t3chguy
Copy link
Member

t3chguy commented Dec 8, 2020

/me is not a Community Manager but a dev - but thanks :D

I think valid concerns and limitations are listed in this issue sufficiently for the Product team to plan a solution for.

@jryans jryans removed the Z-UI/UX label Mar 9, 2021
@aaronraimist aaronraimist changed the title Explain manual and automated key backup a little better Explain manual and automated key backup better Aug 19, 2021
@kittykat kittykat added X-Needs-Community-Testing Requests testing in a community testing session. and removed X-Needs-Community-Testing Requests testing in a community testing session. labels Dec 16, 2021
@brumasribera
Copy link

brumasribera commented Jul 27, 2022

Would there be a way to simply back up the conversations and get them when you log in into new devices. I am really not concerned about security and this huge complexity I find.

I only would like to be able to keep the whole history of messages all the time, and this seems to be impossible right now to be done on a human way, not having to export files with encrypted keys now and then.

@aaronraimist
Copy link
Collaborator Author

aaronraimist commented Jul 27, 2022

@brumasribera Yes. That is what the Secure Backup feature does. It automatically backs up your keys. As long as you provide your Security Key (or Security Phrase) each time you login, your keys will be automatically backed up and restored.

@SimonBrandner SimonBrandner added O-Frequent Affects or can be seen by most users regularly or impacts most users' first experience and removed P1 [OBSOLETE LABEL] Next labels Jul 28, 2022
@brumasribera
Copy link

brumasribera commented Jul 28, 2022

@aaronraimist Backing up key means also backing up messages, I am more concerned with messages and not so much on those keys.

For me, it would be just more simple to recover the messages normally once reconnected to your account and have this extra safety layer optional.

Or at least be able to disable it, as I find it a bit cumbersome for my normal use. I am of course only talking about UX experience for the user. Thank you for your time!

@aaronraimist
Copy link
Collaborator Author

aaronraimist commented Jul 28, 2022

@brumasribera

Backing up key means also backing up messages, I am more concerned with messages and not so much on those keys.

Well that's a different problem. If you don't trust your homeserver to not delete your messages then you probably want to be using the export chat feature. Go to the Room info page and then click Export chat. There is no feature to automatically do that though, that would be #15247.

For me, it would be just more simple to recover the messages normally once reconnected to your account and have this extra safety layer optional.

Encryption makes it very difficult to automatically restore messages. If you want messages to be automatically visible with no action other than logging in, your only real option today is to not encrypt the room. In the future matrix-org/matrix-spec-proposals#3265, matrix-org/matrix-spec-proposals#2957, or matrix-org/matrix-spec-proposals#3262 may solve this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-E2EE A-E2EE-Key-Backup O-Frequent Affects or can be seen by most users regularly or impacts most users' first experience S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Something isn't working: bugs, crashes, hangs and other reported problems T-Enhancement New features, changes in functionality, performance boosts, user-facing improvements
Projects
None yet
Development

No branches or pull requests