Making it incredibly easy and intuitive to send signed & encrypted email is not an easy task and there are not many good examples out there. I've been using GPGMail plugin for Mac Mail client which uses GPG Keychain Access- all in all, it works, but there are numerous aspect which I find quite annoying, annoying in fact enough to uninstall the plugin altogether.
The problems I've encountered with this plugin are:
Contact Based Format Choosing
Auto Encrypt When Keys Are Present For Contact
Clearly Explain What is Not Encrypted
Explain Contact Fields Better
More on this later...
Agreed on most parts. I think there is a particular change that needs to occur here regarding encryption. Most mail clients that support PGP support it as a plugin or afterthought, and therefore everything has a tendency to be "use explicitly" or sometimes "best effort" rather than "on by default" or "use implicitly". This is I think the thing we need to change.
In terms of flow, this means that where most mail clients have the user explicitly state when they want to encrypt, we would have the user explicitly state when they don't want to encrypt, although there is a fallback position that if the public key is not known for the recipient, we drop out of "encrypt" mode and highlight the reason why we did so. I see this as happening in the "to" field:
Now, as you said, there should be some signalling that encryption will/will not happen and also which bits are going to be encrypted and which arent. Some mockups on that:
... these are ugly, but you get the point. Subtle hinting and sensible defaults.
As for signatures, I've been mulling this over a bit and have come to the conclusion that we should be doing attachment-based signatures. The reason they look weird or annoying or come off like viruses is that despite 20+ years of PGP, most mail clients still don't differentiate between a detached signature and an unknown attachment. This is something we cannot really change except by brute force: If these signatures become ubiquitous, mail client developers might finally get off their arses and start understanding what they are. Heck, even Thunderbird doesn't get this one right.
(@bnvk, Let me know if you want the SVG for these; they're crap but I'm sure you can adapt, adopt and improve...)
My two biggest issues with the Mac Mail GPG plugin (and with Mac GPGTools in general):
If I don’t have someone’s key, there’s no obvious path for me to take to discover it:
Turns out “Download” actually offers a fairly helpful key-finding window:
But still, why do I have to manually search each keyserver? Why should I even have to know that the keyserver exists? The plugin knows how to search a keyserver for an email address and download the associated key, but makes the human actually do it — transferring my focus from “communicating securely with people” to “managing keys”. The more security interferes with focus on a task, the less likely people are to use it.
Is this sort of automatic key management within the scope of mailpile? It’s certainly a big UX problem at the moment.
Lots of good thoughts here @smari I think we see 100% eye to eye on what encrypted "on by default" means, but are just using different words / muxing up the UX "events" which signal this ;)
Yes @BarnabyWalters I absolutely agree about the key discovery process. Thanks for posting the screen caps. My current thinking is to try to make the key discovery / getting process feel like a "friend request" as in essence, that it is- sorta!
@barnabywalters, this is exactly why I was ranting a bit on Twitter yesterday about the screwed-uppness of HKP (HTTP Keyserver Protocol), and how it might be to blame for a lot of the brain damages of how people use PGP, and more importantly for us, how people design UIs for PGP.
As an added twist to the "on by default" approach, the program should have a way to remember preferences for how one communicates with a particular person. So I should be able to specify that even if Smári has a public key, I would rather by default send him signed messages (not encrypted) so he can access them from his mobile phone as well - unless it truly is something critical.
The app should remember for me how I prefer to communicate, and down the line, if we can implement it without creating some scary security bugs, it would be ideal if there could be autodiscovery, some way I could signal to others what my personal preferences are.
I also have some points from the workshop I attended at OHM with folks who work doing secure communication trainings, which I'm going to throw out just for posterity:
YES!!! I agree 100% with @BjarniRunar points. I especially like the notion of "The app should remember for me how I prefer to communicate" and is precisely stuff I want to build through MailPile
Also: Make a user-friendly "intuitive" UI for dealing with the aftermath of key signing parties. Because being able to use OpenPGP-encrypted email is pretty useless without having a good personal Web of Trust; and the easiest way to do that is keysigning parties.
Currently stuff for doing this is unusable (at least in an amount of time people are willing to spend) by the vast majority of even people who know what a keysigning party is; and the manual method esp. for bigger parties is a massive PITA that many just end up not doing in my experience.
So as far as I know this is current best practice (from the Ubuntu wiki):
After the Key signing
Following the key signing, you'll need to actually sign people's keys. The easiest way to do this is to use caff which is conveniently packaged in the Ubuntu signing-party package. caff lets you sign a number of keys at once, and will then email each recipient their signed key, encrypted with their key (actually, it sends one email per UID on the target key, so those people with 10 UIDs on their key will get 10 emails from caff, but that's OK - it makes sure they control that email address too). They must know their own pass phrase to retrieve their signed key, which they can then import into their gpg keyring and upload to the key server pgp.mit.edu.
caff isn't a very friendly to desktop users (it requires a local MTA or open SMTP server) who don't know Perl. If you don't mind a little light work on the command line and use KMail, there exists a quick Python script for automated keysigning with KMail. It removes the annoying manual labor bits of the procedure described below, and sends e-mails using KMail.
Thank you for starting this issue, as it is incredibly important.
@barnabywalters @brennannovak I would strongly caution against having the plugin automatically suggests keys for people based on their email address, since the email (as well as name and timestamp) associated with a key is trivially easy to forge. Even a system where the plugin automatically fetches all the keys that match a particular email and asks the user to pick one to use (based on signatures, for instance) needs careful design, since an attacker can create keys with fake identities and sign other fake keys to make them look trustworthy.
IMO the only situations where the plugin should "recommend" a key to use for an email address is when I've signed that key (in which case it should just encrypt by default with that key anyway as long as the key is still valid) and when there is a trust chain of length < n from my key to that key. The default for n should be 2 (aka, someone whose key I've signed has signed that key) but people can change it if they really want.
Pardon the sloppiness, but I made some quick sketches of what this could look like:
@diracdeltas you seem to imply that if not every recepient has a key, the message will be sent encrypted to everyone who has a key, and plain text to everyone else.
This is a great idea, but we need some really good UI features to really drive the idea home that this will happen that way. For example, with Enigmail + Thunderbird, I have not even found a way to do this. I also don't know how it would technically work while leaving header information (To, CC, BCC and so on) intact, but I just assume that someone with more knowledge than me on how eMail works can comment on that later.
My point is: If we make this the default behaviour, we need some strong indicator that this is happening, and (at least for the first time it happens) a confirmation window with the option "do not show this again, I know what I am doing".
I will also continue to bully a professor I know to finally give out a Bachelor thesis on the topic of usability of GPG / Enigmail, so we can learn from their mistakes (of which there are a bunch, as you probably know).
@malexmave Yes, that is the implication. Speaking for myself, I would want emails to trusted recipients to be encrypted by default as a matter of principle when it's not a matter of security. But this may just confuse "normal" users.
I am leaning toward showing users the same UI that they see when an email is not encrypted at all when it is sent unencrypted to at least one recipient. The point being that if we're going to convey information with UI, it should be the worst-case information from a security standpoint.
I like @smari's idea of putting a red tag next to fields that are unencrypted and a green tag next to fields that are encrypted, maybe with a broken/unbroken padlock for people who are colorblind.
@diracdeltas Correct me if I'm wrong, but from a quick glance on RFC 5321 (SMTP, esp. Section 126.96.36.199 on RCPT), I don't think that it is easy or even possible to transfer different message contents (as in, encrypted and non-encrypted) to different recipients while still maintaining the original TO and CC fields. So, sending the mail encrypted and unencrypted at the same time while having encrypted and unencrypted recipients being able to "see" each other in the TO / CC-Fields seems to be impossible inside the SMTP standard.
Then again, I am by no means an expert on this, but I think if an expert on this topic could settle this, we could continue designing the UI (if it is possible) or brainstorming alternatives (if it is impossible).
@malexmave that's right. You'd have to send out different messages, which would mean breaking the To/CC network information.
@diracdeltas many good proposals, but it feels like you're assuming we're always "fetching" keys and never retaining them in local keystores. The entire purpose of my "?" button in the above mockup was to say "we don't have a key for this person, what do you want to do?" ... clicking on the question mark would lead to some way of fetching keys.
This is also why I've been looking at HKP with an evil eye... there are problems with HKP that make trusting stuff from the keyservers a very manual process. I would like to be able to say something like, "I would like to fetch all FOAF keys". Also, I've been thinking about cover traffic on keyservers and how we might be able to make network analysis harder while increasing "automaticity"... not reached many concrete conclusions yet I'm afraid.
@smari @malexmave - actually, SMTP doesn't care about the message headers at all. You can write whatever you want in the To: and Cc: lines, that stuff is just a decoration and a hint to the receiving clients about how to implement replies. The actual work happens in the SMTP dialog where the client says "RCPT TO: xxx".
@smari Okay, I feared as much. If we really, really want this feature, the only option I could think of would be to add some lines to the beginning of the mail, stating "The following message has been sent unencrypted to email@example.com", but that would make "reply all" (yes, I know) a hassle, so it isn't really an option. See at the bottom of the comment.
I don't really see where @diracdeltas is assuming that we always fetch the keys and never retain them. I might have missed something, but I think all of his proposals would work equally well with a local keystore...
In general, I think it would be the best Idea to have two options for the fetching of new keys:
In general, there are a lot of trust and confidentiality issues when using key fetching, both on the client side ("is this key forged?") and on the server side ("but I don't want the MIT to know who I am writing to!"). The second is a general problem with all (semi)centralized services, and has, to my knowledge, not been solved in a practical fashion (except maybe using DHTs, but that would imply a whole new PKI).
And, to the fresh reply of @BjarniRunar: And the clients use the To: and Cc:-Lines when deciding who to reply to? Then it might actually be possible to implement the "dual" sending strategy of sending it encrypted to some and unencrypted to others...
Edit: Then again, it would kind of defeat the purpose, as the unencrypted recipients would reply unencrypted to the people who prefer encrypted communication, rendering the first mail's encryption pointless because it is quoted in the second mail...
One could also take a look at STEED, which may be interesting to read (even if it is not likely to take off soon, or even at all), plus there are a bunch of articles and papers linked at the bottom of the page that deal with usability of PGP (so this project can do better, where possible).
@smari Thanks. I intended to convey that we should retain keys locally once they've been fetched the first time, but I see where the confusion comes from. But this actually brings up another point: how do we sync the local keystore with the keyservers?
I think most people using GPG either manually run gpg --refresh-keys or use a daemon like Parciemonie, which has the added benefits of requesting updates over Tor with random sleep intervals.
These are the ways I can think of to deal with syncing:
1. Automatically refresh the key every time it is used. The benefit is that we will always know that we're using the most up-to-date version of the key, but this is probably the worst option, as it's almost no better than fetching keys every time and is a privacy risk, as @malexmave has mentioned.
2. Have a client-side script that regularly refreshes the entire keystore (ex: once per day). Ideally this would be done with strong anonymization like in Parcimonie.
3. Run a server that syncs with the other keyservers and pushes changes to the client. Ex: When Alice pushes a revocation certificate to pgp.mit.edu, this server sees it and pushes the revocation to all mailpile clients who have her key in their local keystore. (But this would mean that the server knows who is in everyone's keystore, unless you do something Tor-like.)
This is no longer UI-related, so perhaps we should start a new issue. :)
@diracdeltas As you proposed, I just created a new Issue for this discussion. Let's bring this Issue back into the UI Stuff and discuss GPG Key Management in #75.
@malexmave There are some good (less-bad) ways to to fetch keys from keyservers without constantly revealing who you're trying to talk to right now. parciomonie is an app which keeps your keyring up-to-date by asking your keyserver(s) for one key at a time, over Tor, with a random delay between fetches. This makes it quite difficult to guess who's making which requests to the keyserver, and gets safer the more people do it. Since it's a background activity, Tor's latency doesn't matter much.
If we have a notion of an address book or contacts list, we could use a parciomonie-like approach not only to refresh our keys, but also to look for keys from contacts where we don't already have keys. With that, we can have an upper-bound on how recently a contact could have started using GPG but still have us send them an unencrypted message. There's always the possibility of having an advanced setting or option to force a full refresh on the addresses to which we're currently composing a message.
Indeed, it's plausible that Mailpile could do some sort of Torified key fetch/refresh in the background when you start composing messages. Alternately, or in complement, the order that Mailpile chooses to backgound-fetch/refresh keys could be cunningly determined based on (for instance) how frequently you contact that UID, how often the key seems to change, whether this UID has ever had a key, and so forth.
[Having written this, I feel like these aspect of key-management might be worth dropping into a couple of other, specific tickets. Would that be the right thing for me to do?]
@diracdeltas I like what you're talking about with clear, categorised UI indicators for different key trustworthiness/validities, but I feel like you're slightly re-inventing the wheel with the states you've defined. GPG already has some notion of these states:
Was there some specific motivation for distinguishing keys that you've directly signed from those that have valid trust chains which comply with your rules?
@flamsmark We actually did start a separate ticket for key management, in which I suggested a parcimonie-ous approach. See #75 (key management) and #76 (tor integration).
So the GPG trust groupings don't make much sense to me when we're talking about implementing UI for people like my mom. IMO it would be user-safest if marginal and untrusted had the same UI (scary! don't trust me!), because people should usually not trust marginal keys. I'm open to the suggestion of making directly-signed and signed-within-a-valid-distance keys have the same UI, but I think there is a case against it.
I don't want to reinvent the GPG wheel, but I feel like GPG wasn't designed with the correct user model for Mailpile.
This issue was the main argument for me to support Mailpile. We really need an interface to help people get the connection between Contacts and Keys (or hide it in a sane way).
The UI suggestions so far did focus around the "send mail" view. I think the "address book" should also get some attention. The list should showing which contacts are verified (by whom) and maybe suggesting to verify the others - by making a phone call for example.
Stats and visualizations of my web of trust may also help to motivate me to invest time into it. ;)
To keep things simple we have to make some decisions for the user. A contact can be either verified or unverified, a message can be secure or not. Everything else would just be confusing/distracting imo.
@mjowe: I'm doing some work on contacts management this week, but we got waylaid by a decision to switch from jsontemplate to jinja as the templating engine (because more people are familiar with that syntax, it's more powerful, and it requires less work to get it where we want it).
I think contact management and key management should be the same thing, and the address book needs to be really good. I want to experiment with a few ideas, but currently need backend stuff to make that possible.
I like hte idea of stats and visualizations of the web of trust. Any specific/concrete ideas there?
Refactored a bit of the LESS for To, Subject, Message fields a bit in…
… Compose, started adding encryption UI feedback Issue #59
Most of the big design decisions about this have been made and are in various states of code implementation. Closing this thread as we move to using Github Issues for more granular specific tasks!
Is there a document which summarises the decisions? For those of us who like following along at home, is there a particular tag for the granular issues on this topic?
For @flamsmark and anyone else interested, here are the various design compositions that we are finishing development on that implement ideas discussed in this thread
Composing & Thread View
As per our spec https://github.com/pagekite/Mailpile/wiki/GnuPG-interface-security#message-part-status and the Signature & Encryption Icons
Of course, once development if finished for Alpha release, real world usability flaws with our approach may come to light, in which case we will iterate, refine, and rethink our approach!
First impressions: The difference between Invalid and Revoked is pretty hard to see, if you do not have the labels for it. The question is if the semantic difference is large enough to justify another design pass, and how an intuitive and distinguishable icon would look like.