Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Declaring End-to-end Encryption stable and turning it on by default for private rooms. #6779

Closed
ara4n opened this issue May 22, 2018 · 42 comments
Closed

Comments

@ara4n
Copy link
Member

@ara4n ara4n commented May 22, 2018

Somehow we seem to be missing a high-level tracking bug for the endgame of E2E. (The starting point was matrix-org/matrix-doc#501, but that's a spec bug and feels a bit weird to hijack it for this).

In order to declare E2E stable (and by extension Matrix), we need to:

  • Cross-sign devices at login to eliminate verification warnings (#2714)
  • Optionally incrementally backup encrypted E2E keys to your HS so if you lose all your devices you can restore your history via a recovery key that you own. (#5675)
  • Improve verification UX (comparing per-verification mnemonic passphrases rather than public keys) (#2142)
  • Hook up key sharing to let history be visible from before you joined a room (but after you were invited to it), history visibility allowing. (#2713) - mainly fixed, other than for edge case where users add devices after being invited to a E2E room
  • Have less than 0.0x% unresolved unable-to-decrypt errors reported through telemetry (#2996, #6390)
  • Fix device lists getting out of sync over federation (#6974)
  • Support full-text search within E2E rooms (#2548, solved by github.com/matrix-org/seshat)
  • Support push and in-app notifications in E2E rooms at least by locally calculating them. We could also support setting metadata on the messages to bing users, especially if we had pseudonymous MXIDs.
  • Support NotifPanel in E2E rooms (#6874)
  • Turn it on by default for private rooms. All rooms created with the intent of use as a 1:1 or a private group chat must be E2E (and homeservers would enforce that, as best they can, c.f. matrix-org/synapse#3269), although obviously a broken client could still try emitting unencrypted events into such a room). The code exists for this already over at https://github.com/matrix-org/synapse/pull/3426/files.

Ideally:

  • Provide some kind of E2E daemon/proxy to stop non-E2E clients/bots/bridges/curl-users being unceremoniously dumped out in the cold. (#6778)
  • Improve the UX for managing the trust of the devices which are in a given room (#4522). In practice if verification is working nicely this might not be so important.
  • Optionally hook up key sharing to let history be visible from before you were invited to a room (history visibility allowing) (#2286)
  • Get an end-to-end audit of crypto spanning a set of reference implementations - e.g. matrix-{web,ios,android}-sdk + synapse.
@aaronraimist
Copy link
Collaborator

@aaronraimist aaronraimist commented Oct 22, 2018

#6959 should probably be in here at least under ideally

@dbkr
Copy link
Member

@dbkr dbkr commented Feb 27, 2019

I think '* Optionally hook up key sharing to let history be visible from before you were invited to a room' needs to be non-optional, or if not then change the history visibility settings for e2e rooms to line up with what's actually possible, ie. remove 'anyone' and 'Members only (since the point in time of selecting this option)')

@ilmaisin
Copy link

@ilmaisin ilmaisin commented Mar 19, 2019

It was originally promised that e2e encryption would be enabled by default when it was out of beta. That didn't happen.

But anyway, as I understand, there are basically three things to be done before e2ee can be default: cross-signing, local search and notifications. Are there any on-going efforts to achieve those?

@ara4n
Copy link
Member Author

@ara4n ara4n commented Mar 19, 2019

yes, of course. cross-signing is in the final stages; notifications just got largely fixed on riot/web and already worked on mobile; local search development is kicking off again as of tomorrow. we also want to fix #6778 before turning it on by default.

@ilmaisin
Copy link

@ilmaisin ilmaisin commented Sep 13, 2019

Well, cross-signing has been "in the final stages" for half a year now. A honest, detailed explaination on what is really happening would be nice.

@ara4n
Copy link
Member Author

@ara4n ara4n commented Sep 13, 2019

a quick but honest detailed explanation is:

Sorry it's taken a while; turns out that this stuff is hard, and we've been juggling a lot of stuff thanks to privacy dramas etc.

@ara4n
Copy link
Member Author

@ara4n ara4n commented Sep 13, 2019

meanwhile, e2e search has been progressing first via https://github.com/matrix-org/pantalaimon (and works well; i use it daily, but it's not integrated tightly with Riot), and now by https://github.com/matrix-org/seshat, which I believe works with Riot/Desktop although I haven't tested it yet.

@ara4n
Copy link
Member Author

@ara4n ara4n commented Sep 13, 2019

turning it on by default is formally proposed as an MSC now, as part of the Canonical DMs proposal: (point 5 of https://github.com/matrix-org/matrix-doc/blob/travis/msc/immutable-dms/proposals/2199-canonical-dms.md#creating-dms)

@ara4n
Copy link
Member Author

@ara4n ara4n commented Sep 13, 2019

...and UISI errors (aka UTDs) are almost unheard of, in my experience. We fixed a major remaining cause of them back in July (matrix-org/synapse#5693) and have built out a whole new project for adding OpenTracing to Synapse so that whenever find further ones, we can pull up a full visualisation of precisely what went wrong to diagnose it and so snuff them out conclusively one by one if/when we see them: https://github.com/matrix-org/synapse/pulls?q=is%3Apr+opentracing+is%3Aclosed.

I think those were the only pending points.

@ilmaisin
Copy link

@ilmaisin ilmaisin commented Oct 24, 2019

#11125 is supposed to implement the e2e search, but it seems to have been sitting idle for two weeks now. By the way, Conversations has been doing e2ee search for ages.

The progress still looks like being really slow to me, though I absolutely may have missed something as I am not an expert on this stuff.

@ara4n
Copy link
Member Author

@ara4n ara4n commented Oct 24, 2019

You’ve missed that the e2e search is 3 layers deep: riot-web, matrix-react-sdk and seshat itself (which unlike Conversations is designed to be crossplatform, written in Rust). The react-sdk layer was active less than a week ago: https://github.com/matrix-org/matrix-react-sdk/tree/poljar/seshat-rebase and the seshat layer was active a few hours ago: https://github.com/matrix-org/seshat/tree/sqlcipher. The feature works great; i’ve been using it for a few weeks. The only issue is that it stores its data currently in plaintext on disk, which is not ideal - so we are reencrypting it when it sits on disk.

@ilmaisin
Copy link

@ilmaisin ilmaisin commented Oct 28, 2019

Is there any way to donate specifically to e2ee work? Looking at those "referenced" notices, it seems that I am not the only one who would like to have this as a much higher priority. I don't want my donations to be spent to the fiddling with emojis or any other such bells and whistles.

Should I open a bountysource entry for this issue or something?

@turt2live
Copy link
Member

@turt2live turt2live commented Oct 28, 2019

As of about 3 minutes ago the team has been given the directive to implement this as fast as safely possible. What this means is that in the coming months we should have a release that implements cross-signing and generally better e2e UX as a result.

We are aware that this statement has been made several times now, but hopefully this time we're not lying given we're about to have code to back up our statement.

@ddobrev
Copy link

@ddobrev ddobrev commented Jan 29, 2020

Hello, would you have any news?

@turt2live
Copy link
Member

@turt2live turt2live commented Jan 29, 2020

The statement above regarding our new directive is still accurate, though this time we are in fact not lying. We're extremely close to being able to show off the work we've done, though there's still a large chunk of work outstanding before we're comfortable putting it out for release.

Much of it can be tried on riot.im/develop if you're willing to risk the chances of us mixing it up :)

@jryans
Copy link
Member

@jryans jryans commented Jan 29, 2020

Cross-signing and E2E by default for DM rooms will be enabled on the develop channel (https://riot.im/develop) in the next day or so to collect feedback from early adopters.

We're hoping to release to the stable channel a few weeks after that, but as this is a huge milestone, we want to be absolutely sure it's ready before releasing to everyone.

@jryans
Copy link
Member

@jryans jryans commented Apr 16, 2020

We've identified a few more release blocking issues to resolve and will have another RC later today for more testing. Updated release target is now next Monday (2020-04-20).

@ilmaisin
Copy link

@ilmaisin ilmaisin commented Apr 16, 2020

The Android and iOS clients also need to have this implemented before it can really be the default, right? How far from completion are they from?

@jryans
Copy link
Member

@jryans jryans commented Apr 16, 2020

RiotX Android and Riot iOS are targeting the same release date as Riot Web for this work.

@rgpublic
Copy link

@rgpublic rgpublic commented Apr 17, 2020

Um, just a quick question while we are all anxiously waiting for this very cool new feature to drop on us: What does "turning it on my default" exactly mean? What happens to my existing 1:1 chats in Riot? Will they magically be encrypted as soon as I access them with the new version? Or will I have to leave/reenter them somehow? Should I recommend our users to check anything so that they won't suddenly lose access to their chat history? Perhaps making absolutely sure they have key backup enabled and?/or? having exported their E2E room keys in the user profile dialog?

@t3chguy
Copy link
Member

@t3chguy t3chguy commented Apr 17, 2020

They will not, for existing rooms the onus is on room admins to enable encryption. 1:1s are still rooms where everyone is an admin.

@rgpublic
Copy link

@rgpublic rgpublic commented Apr 17, 2020

Understood. Thanks a lot @t3chguy for the clarification!

@Ekleog
Copy link

@Ekleog Ekleog commented Apr 17, 2020

Maybe I'm missing something, but… as far as I could find in the UIs, cross-signing isn't supported yet in a released version of riot. Would it not make sense to first release cross-signing, wait for at least a few weeks for things and bugs to stabilize, and then turn on encryption by default, once it's confirmed that it actually works? Otherwise, I'd expect a lot of angry shouting at every bug that may be remaining in the cross-signing code but not yet apparent due to the low intersection between people using the development branch and people using e2e rooms

@jryans
Copy link
Member

@jryans jryans commented Apr 20, 2020

We've identified a few new performance and behaviour issues to resolve before release thanks to everyone's feedback and testing on staging. We believe there may be around 1 week of work to resolve at this point, but we'll keep evaluating every day.

@jryans
Copy link
Member

@jryans jryans commented Apr 23, 2020

We've published 1.6.0-rc.4 with some additional cross-signing fixes, so we're getting very close thanks to everyone's testing and feedback! Please do test and file issues for any feedback or problems you may see.

@QEDeD
Copy link

@QEDeD QEDeD commented Apr 23, 2020

It's good to see that you are taking the time needed to get the launch right, even though it must be rather tempting to go ahead when you are this close.

@rgpublic
Copy link

@rgpublic rgpublic commented May 5, 2020

Hooray. Party. I just got an update on my (Android) phone with a new RiotX version and E2EE. But: There is no matching update (PPA) update available for the Desktop (Linux). What should I do? Just wait?

@jryans
Copy link
Member

@jryans jryans commented May 5, 2020

There is no matching update (PPA) update available for the Desktop (Linux). What should I do? Just wait?

For issues and questions like this, please follow updates and ask questions in #riot-web:matrix.org.

@jryans
Copy link
Member

@jryans jryans commented May 5, 2020

Riot Web and Desktop v1.6.0 are now available with cross-signing and E2EE by default for DMs and private rooms enabled! 🥳 Thanks to everyone who helped test and provide feedback along the way. 😄

That wraps up the main focus of this issue. For any related concerns that seem unresolved, please file new issues.

@jryans jryans closed this May 5, 2020
@DarwinPorras
Copy link

@DarwinPorras DarwinPorras commented May 5, 2020

Hi jryans. Today I updated all my devices to Riot 1.6.0. I was able to verify all my sessions in different accounts but I can not make chats to be encrypted I also do not see the padlock closed on my app. I don't know if I forgot to setup something but I have this message on any user account

SECURITY
Messages in this room are not end-to-end encrypted.

Verify
- Riot Desktop on Windows
Hide sessions

Do you know what can I do to enable E2EE on direct messages?

@turt2live
Copy link
Member

@turt2live turt2live commented May 5, 2020

@DarwinPorras you'll have to go into the room settings. Please visit #riot-web:matrix.org for support.

@DarwinPorras
Copy link

@DarwinPorras DarwinPorras commented May 5, 2020

@turt2live Thanks!

@Bun-Bun
Copy link

@Bun-Bun Bun-Bun commented May 6, 2020

While I understand the necessity for this feature for many people, it is actually a detriment to my server.

For my purposes my server should never allow encryption ever. How do I disable encryption full stop?

@t3chguy
Copy link
Member

@t3chguy t3chguy commented May 6, 2020

For my purposes my server should never allow encryption ever. How do I disable encryption full stop?

That would be a query for your server of choice, probably https://github.com/matrix-org/synapse/

Synapse can disable both cross-signing & encryption altogether.

@Bun-Bun
Copy link

@Bun-Bun Bun-Bun commented May 6, 2020

Forgive my ignorance, but the last time I check (only a couple weeks ago) there was no official way of disabling encryption on synapse.

Having a config option on my self hosted riot to disable the encryption would be helpful. Though that only helps the web users. You're right it needs to be at the server side and all riot clients need to respect that by transparently disabling encryption and not throwing an error or other weirdness that makes users ask questions.

@ptman
Copy link
Contributor

@ptman ptman commented May 6, 2020

Indeed, matrix-org/synapse#4401 makes it sound like disabling encryption isn't yet possible.

@t3chguy
Copy link
Member

@t3chguy t3chguy commented May 6, 2020

Right.

For my purposes my server should never allow encryption ever.

But this cannot be done by a riot-web instance unless you force everyone to use that, people use things like riot android/ios too.
Ergo it needs to be done on the server.

It could be done using Nginx as a reverse proxy to intercept and fail m.room.encryption state events and also to return the right flags to claim the server doesn't support cross-signing.

@turt2live
Copy link
Member

@turt2live turt2live commented May 6, 2020

Please let's not use closed issues for support - instead, #synapse:matrix.org, #riot-web:matrix.org, or a new/already open issue's comment section is best.

@vector-im vector-im locked as resolved and limited conversation to collaborators May 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet