Add support for client certificate and password authentication

[cketti]

Add support for using a client certificate and password authentication together.

[levaidaniel]

Oh yes, please!
I'd test the s***t out of this, if you guys have rc/beta releases.

[damiankaczkowski]

+1

Currently dovecot is unable to authenticate the K-9 user with client certificates authentication. Dovecot requires mail client to send AUTH command even if you explicitly select to request client certificate and to use client certificate cn as user name. If client does not send AUTH data after TLS negotiaton then dovecot fails with 'no auth attempts' error. Thunderbird got the same issue with client certificate authentication but you can workaround this and use plaintext auth with SSL socket, and configure it with username and some bogus password. Thunderbird will sends client certificate anyway during TLS negotiation if server requests it and after TLS negotiation will send AUTH data. This allows dovecot to succesfuly authenticate user.

We need to either:

• allow client certificate authentication + plaintext/encrypted authentication with username and password,
• or modify current behavior to send AUTH data with username+blank_password or try anonymous login after client cert authentication when using client cert authentication method,
• or modify client behavior to always send client certificate during TLS negotiation if server request it regardless of authentication method selected by user. This will allow to use plaintext authentication or any authentication method and the client will present its certificate if required. This is I think the current Thunderbird behavior.

Please consider implementing this feature/behavior. Thank you.

ps. This pull and comments are related to the problem too:
#506

[mhoran]

While the Dovecot authentication model does not permit cert-only auth by default, it's pretty easy to set up with a custom checkpassword script that permits auth if a valid certificate was received. This is standards compliant (K9 sends AUTH EXTERNAL) while your other proposals are not.

Sending a username and password after cert auth is permitted by the IMAP RFC, but EXTERNAL is a permitted authentication mode as well. So, ideally well end up with an either/or option for this configuration.

Happy to provide an example Dovecot configuration I'm using to make this all work.

[mhoran]

Interestingly, I'm not sure why #506 was closed. That issue pertains to SMTP, and is an issue I have as well. My SMTP server permits relay based on cert and does not require AUTH in that case. In fact, when K9 sends AUTH EXTERNAL to my SMTP server, Dovecot returns an error because it is not aware of the certificate presented to Postfix. I've backed around this by removing EXTERNAL from the SMTP handler, but a proper upstream fix would be preferred.

[damiankaczkowski]

While the Dovecot authentication model does not permit cert-only auth by default, it's pretty easy to set up with a custom checkpassword script that permits auth if a valid certificate was received. This is standards compliant (K9 sends AUTH EXTERNAL) while your other proposals are not.

Thank you for the hint. Will try checkpassword method. I am not sure how to use EXTERNAL right now but will work something out.

btw. Developers: Is there a chance that you could just enable support for mutualTLS when requested by server? As far as I know it won't break any mail RFC, will it? TLS is just a security for transport layer, should be independent from any higher level protocol including IMAP/POP/SMTP protocols. It would be nice to have full TLS support in whole product, not just in a part of it like it is now. What do you think?

Sending a username and password after cert auth is permitted by the IMAP RFC, but EXTERNAL is a permitted authentication mode as well. So, ideally well end up with an either/or option for this configuration.

This would be a great solution.

Happy to provide an example Dovecot configuration I'm using to make this all work.

If you could please. Will save me some time. Thank you in advance.

Interestingly, I'm not sure why #506 was closed. That issue pertains to SMTP, and is an issue I have as well. My SMTP server permits relay based on cert and does not require AUTH in that case. In fact, when K9 sends AUTH EXTERNAL to my SMTP server, Dovecot returns an error because it is not aware of the certificate presented to Postfix. I've backed around this by removing EXTERNAL from the SMTP handler, but a proper upstream fix would be preferred.

I have exactly the same setup, but I didn't saw that K9 sends AUTH EXTERNAL. I just saw that there is some authentication request from smtp like this for eg.:

Debug: client in: AUTH 1 PLAIN service=smtp nologin lip=192.168.0.1 rip=192.168.0.2 secured resp=A1RhbWlhbi5rY1N6a29312tpADEyMw== (previous base64 data may contain sensitive data)

Is this AUTH EXTERNAL?

[mhoran]

Here is the Dovecot checkpassword script I use, in combination with SSL client certificates:

#!/bin/sh

if [ "$AUTH_CERT" != "valid" ]; then
  exit 1
fi

passwd=$(getent passwd "$AUTH_USER")

if [ $? != 0 ]; then
  exit 1
fi

export HOME=$(printf %s $passwd|cut -d : -f 6)
export USER=$AUTH_USER
export userdb_uid=$(printf %s $passwd|cut -d : -f 3)
export userdb_gid=$(printf %s $passwd|cut -d : -f 4)
export EXTRA="userdb_uid userdb_gid"

exec $1

I also have the following in my Dovecot 10-auth.conf file, above my standard passdb configuration:

passdb {
  driver = checkpassword
  args = /usr/local/bin/checkpassword
}

Note that auth_mechanisms must include external as well. Mine is set to auth_mechanisms = plain external. Combine this with auth_ssl_username_from_cert = yes to extract $AUTH_USER from the certificate.

I had to patch K-9 to disable EXTERNAL for SMTP auth. When Postfix is configured to use Dovecot for SASL auth, Dovecot presents EXTERNAL as an auth mechanism, despite not terminating the SSL connection. This seems to be in violation of the RFC, as Dovecot should only return EXTERNAL after a valid client cert has been presented (which is not possible when used via SASL.)

I think there should be an option in K-9 to disable SASL authentication when client certificates are used (in addition to the option to provide a username and password, if so desired). In my case, password auth is not feasible in combination with certificates, as I utilize time-based one-time passwords. So I must rely on client certificate auth only -- which unfortunately I've had to hack into K-9.

[damiankaczkowski]

After discussion with @mhoran I think that the best solution (at least at the beginning) would be to just implement mutualTLS for the whole product (allow client to present client cert whenever server ask for it in TLS dialog). This would solve most problems and allow users to successfully use various authentication methods with various SMTP/IMAP/POP server combinations IMO. Currently client cert auth to postfix + dovecot is impossible (tested with newest postfix 3.0.3 and 2.11.7 and dovecot 2.2.21). At least we can't do it without patching K-9.

[sebster]

Is there any progress on this issue? I'm experiencing the exact same problem: for sending mail postfix (submission) requires mutual TLS authentication, but no further authentication after that (any valid client is allowed to send mail). However k-9 complains that AUTH EXTERNAL is not advertised. Although having a custom build is far from ideal, to get this working I'm also interested in mhoran's patch...

[otherjon]

I'm also trying to find an IMAP client that supports both client certificates and password authentication simultaneously. I couldn't quite follow what this discussion concluded -- please forgive the n00b. What is the current thinking about cert+password auth in K-9? If it won't be supported soon, do you know of another Android client that does support it?

[philipwhiuk]

As I understand reading the code, mutual TLS is exactly what happens if you use a client certificate in K-9. But we assume that case is always AUTH EXTERNAL once the TLS connection is set-up. (i.e. we use the client cert to connect and then we send "AUTH EXTERNAL username" as an IMAP command).

I'm not much interested in K-9 supporting arbitrary custom Dovecot configurations but we could for example have:

• Client certificate + no additional auth == AUTH EXTERNAL
• Client certificate + password == AUTH PLAIN / LOGIN
• Client certificate + CRAM MD5 password == AUTH CRAM_MD5
• Client certificate + XOAUTH2 == AUTH XOAUTH2

I've added this issue to the onboarding rework list so it gets considered.

[mhoran]

Hey @philipwhiuk. It's great to hear that this issue may be getting some attention. As outlined above, I have managed to get K-9 and Dovecot playing nicely with AUTH EXTERNAL via a checkpassword script. I wouldn't call this an arbitrary custom Dovecot configuration, as it's the only way (that I have found) to get Dovecot to bypass username/password requirements when relying on mutual TLS for authentication.

The issue I've had to patch K-9 to support is with mutual TLS for SMTP. When Postfix is configured with Dovecot as its SASL backend, the AUTH EXTERNAL command sent after mutual TLS communication has been established will be forwarded to the aforementioned checkpassword script. However, when using Postfix with the Dovecot SASL backend, a valid client certificate is not presented to Dovecot, and the checkpassword script fails. This makes it impossible to bypass the username/password requirement for mutual TLS over SMTP. I've patched the SmtpTransport to remove the requirement of EXTERNAL so when mutual TLS is used it does not try any additional authentication.

Thunderbird handles this configuration just fine. When configured with Connection security: "STARTTLS" and Authentication method: "No authentication", Thunderbird does not attempt any sort of EXTERNAL auth and can send messages just fine. For IMAP, it does use EXTERNAL and everything works as well. It'd be great if K-9 could officially support this configuration, as it seems standard in my understanding.

[philipwhiuk]

Right so wouldn't it be better to get rid of that script and use both the client certificate and the password? That's my proposal - that we support that out of the box in K-9 and you patch neither Postfix nor Dovecot nor K-9.

[mhoran]

I use mutual TLS in this configuration because password authentication for IMAP and SMTP must include not only my password but a one time password concatenated to it. This is intended as additional security for my webmail client, or insecure IMAP/SMTP access from an untrusted workstation.

Having to generate a one time password every time I send mail from K-9 is untenable, so I'd have to come up with some other configuration.

[mhoran]

One thing I forgot: it seems that Postfix still processes the AUTH EXTERNAL presented by K-9 even though my Postfix configuration first requires mutual TLS, then SASL. This can't be overridden, so far as I can tell. Dovecot completely refuses to run the checkpassword script, because it did not terminate the connection. Even if it did, I don't think I could make it work in any way. So I can't see any workaround I can configure other than continuing to patch K-9.

[noahhaon]

Right so wouldn't it be better to get rid of that script and use both the client certificate and the password?

FWIW that would be my ideal configuration. Thank you for your attention to this issue!

[philipwhiuk]

"This is intended as additional security for my webmail client, or insecure IMAP/SMTP access from an untrusted workstation".

It seems like you're running a very odd self-hosted configuration. Gmail and other clients use OAuth based implementations to avoid this. What piece of software mandates that every single SMTP auth must have a new OTP. Doesn't this mean you have to generate a new OTP every time the webmail client sends an email.

Frankly I don't see the value in trying to support this. It seems fundamentally broken not to have a way of trusting a client in some way.

If you don't want to use a username and password and you are using TLS client auth then AUTH EXTERNAL is the SASL standardised way of doing it: https://tools.ietf.org/html/rfc2222

Dovecot is broken if it can't handle that IMO. Consider patching Dovecot rather than K-9.

[mhoran]

OK. But let me point out that Thunderbird does, and has supported this for years. I don't see why having similar option is an issue. There is no way to configure Postfix to work with K-9 and AUTH EXTERNAL as it stands today. Dovecot simply denies the request, and if I disable SASL on a different SMTP port with SASL disabled, K-9 won't even try to connect. I raised this issue on the Dovecot mailing list in hopes of further discussion. But even though my configuration may be unique, I think you're overlooking similar applications that may be more common.

…

On February 3, 2017 7:17:12 AM EST, Philip ***@***.***> wrote: "This is intended as additional security for my webmail client, or insecure IMAP/SMTP access from an untrusted workstation". It seems like you're running a very odd self-hosted configuration. Gmail and other clients use OAuth based implementations to avoid this. What piece of software mandates that every single SMTP auth must have a new OTP. Doesn't this mean you have to generate a new OTP every time the webmail client sends an email. Frankly I don't see the value in trying to support this. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #793 (comment)

-- Sent from my mobile device.

[philipwhiuk]

On the other hand you have bugs like this in Thunderbird: https://bugzilla.mozilla.org/show_bug.cgi?id=749230

Thunderbird probably supports the un-standardised broken behaviour of Dovecot instead of the RFC.

[mhoran]

Fair enough. AUTH EXTERNAL is probably the right thing to use here, and the issue lies with Postfix and Dovecot. However, I'm not too hopeful that will get resolved, so I'll have to come up with some other solution.

[otherjon]

@philipwhiuk, your solution would work for my case, but then I'd have an additional feature request: allow the user to require password entry every time instead of storing the password. If the user presents a client cert, then that authenticates the device and prevents a malicious host from spamming the mail server. Repeating a stored password doesn't prove anything that wasn't already proven by repeating the stored cert. However, manually entering the password can prove that the good device is being used by its rightful owner, so the additional security has value.

If you like, I can open a separate feature request, but it seems very closely linked to supporting mutual TLS + password, for the reason above.

[philipwhiuk]

I don't think that's a good idea. It will utterly break Push email in all likelihood.

The way to confirm a user is in control of their device and thus K-9 is device security like fingerprints and device passwords, not having to re-enter your email password.

This is exactly the same model that every desktop client works on.

[otherjon]

Too bad, that would've been helpful over here, and I was hoping it would be easy to implement. But if you don't want to support that model and there isn't much demand for it, then I understand the decision. Thanks for the response.

[mhoran]

Hey @philipwhiuk. I'm still struggling to figure out a way forward here with K-9 and the decision to stick with AUTH EXTERNAL for SMTP. My post to the Dovecot list didn't get any responses.

I haven't been able to figure out a way to get Postfix to work with AUTH EXTERNAL for SMTP, without writing my own custom SASL method. Dovecot seems to just ignore the AUTH EXTERNAL request when it comes from Dovecot, though I've not yet been able to find where this happens in the source.

Really, I just want Postfix to ignore the SASL request when a valid TLS cert is presented. However, this does not seem to be an option.

If you have any thoughts or real world configurations that I could look at to help figure out how to get back on mainline K-9, that'd be great! I think some others subscribed to this issue would benefit from this as well.

[meierfra]

@otherjon if you still looking for a android mail client supporting client cert and password you could have a look at r2mail2. That's what I'm using at the moment, but it is has other shortcomings. So I was quiet thrilled to read that k9 mail supports client cert, just to be disappointed to find out, it is only either password or client cert for authentication, but not combined.
So I really would be interested in this feature as well. I don't get the reason why #780 was not seen as a solution. As I understand it basically keeps the client certificate dialog visible, so a client certificate can be selected for any authentication type.

[jacobzimmermann]

This issue was raised in 2015. It is now 2017 and still no fix. Yes, the onus is on K9 to fix this, not dovecot, as other email clients work just fine and only K9 is PERMANENTLY UNUSABLE in this context.

[levaidaniel]

Chill, mate. This is being worked on as a Google Summer of Code project. Just do your homework and search for it.
Also, as you may or may not know, this is an open source software backed by volunteers. So why don't you put your money where your mouth is, and send in some diffs?

[realityexists]

It's a been a year without any activity, so I guess this didn't get done during GSoC 2017. I really like K9 Mail, but having to disable client auth on the server to use it is a hard compromise to make.

[dmitrydonskih]

Ok guys, I finally got it working: K9Mail + Dovecot 2.3 = certificate authentication.
The main idea is: I have one dovecot service for common (internal) use, it supports Kerberos and plain authentication, and it does not require a cert. And, I start another one Dovecot service on another port which requires valid cert, obtains username from cert, and it has a dummy passdb with "nopassword=y":

#excerpts from dovecot-proxy.conf
base_dir = /var/run/dovecot-proxy/
service imap-login {
    inet-listener imapext {
        port = 145
    }
    inet-listener imaps {
        port = 0 # disable on 2nd dovecot
    }
}
userdb { 
    driver = sql
    args = /etc/dovecot/dovecot-sql.conf
}
passdb { 
    driver = static
    args = nopassword=y
}
auth_ssl_require_client_cert = yes
auth_ssl_username_from_cert = yes
ssl_cert_username_field = emailAddress

However, still no luck with K9Mail + Postfix cert auth...

[mhoran]

There has been some movement in Postfix to make AUTH EXTERNAL work as expected [1]. Together with the above mentioned configuration for Dovecot, that should make client certificate authentication work with Postfix/Dovecot and K9. Note that this is for configurations where a password is NOT required (my use case) but not for others (where a K9 change would be required.)

Hopefully both configurations will be supported in the (near) future!

[1] https://marc.info/?l=postfix-devel&m=154748967225540&w=2

[oreineke]

Since there seems to be no activity in solving this issue, could you please reconsider to accept #780 as a temporary solution?

[sebster]

Since there seems to be no activity in solving this issue, could you please reconsider to accept #780 as a temporary solution?

I would also really like this. I switched to R2Mail2 in 2017 after I made the previous comment here, but now R2Mail2 is no longer maintained and I have to look for a replacement. I would prefer to be able to use K9.

[toppk]

Hi, I'm just looking into this issue, and it seems like the blocker is the URI system for encoding server settings, and not the general idea of decoupling auth external from using client certificates. (auth external would be one of several valid options).

If that is true, maybe we can open a new issue to future proof the transfer settings and then we can resurrect some of the code in #506 and #780 to do the decoupling when we're ready with that.

As for the URI, I've taken a peak at it, and it seems like it does have issues, the most obvious is that ordering is important. I think both mysql and postgres have some examples of extending uri sanely that would work for imap and pop and these use cases. There are other options, like json, or creating a dictionary type for the preferences storage. I'd happy to go in further detail, and take a stab at a patch, if we're agreed on the way forward here.

pinging @cketti for feedback.

[toppk]

this enhancement has been merged to main.
#5115