LOGIN fallback doesn't occur if rejected pre-continuation

[gnu010]

Expected behavior

There should be no problem in connecting to the imap.scarlet.be server and fetch incoming mails (it works in Thunderbird and in mySecureMail). These are the required settings, from the provider, which seem just normal (provider's info in Dutch / provider's info in French):

Incoming e-mail (IMAP) : imap.scarlet.be Port : 993 (SSL enabled)
Outgoing e-mail (SMTP) : smtp.scarlet.be Port: 465 (SSL and “server requires authentication” enabled)

Actual behavior

In K-9 Mail I'm only succesful at sending email, not receiving. The following error pops up at setup:

Setup could not finish
Cannot connect to server.
(Command continuation aborted: #2# [NO, AUTHENTICATE mechanism not supported, mate])

I also get the error when, afterwards, I try small variations in the settings (such as disabling 'autodetect IMAP namespace'), in account settings > fetching mail > incoming server.

Note

I posted this issue on the community mail group as well.

Environment

K-9 Mail version: 5.207

Android version: 6.0.1

Account type (IMAP, POP3, WebDAV/Exchange): IMAP

[AEaut]

salut,

the problem is obviously not dependent on your individual k-9 installation. i have the same result with my installation on a kitkat (where imap authentication is okay in general) when i use test/invalid credentials (varius username patterns) with imap.scarlet.be.

is there staff at scarlet to answer that question?

[philipwhiuk]

It ought to be falling back to login which appears to work:

$ openssl s_client -connect imap.scarlet.be:993
CONNECTED(00000003)
depth=1 /C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Validation CA - SHA256 - G2
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=BE/ST=Brussel/L=Evere/O=Scarlet Belgium NV/CN=*.scarlet.be
   i:/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Validation CA - SHA256 - G2
 1 s:/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Validation CA - SHA256 - G2
   i:/C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFOjCCBCKgAwIBAgIMbIHRWlsamGRBbL5CMA0GCSqGSIb3DQEBCwUAMGYxCzAJ
BgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMTwwOgYDVQQDEzNH
bG9iYWxTaWduIE9yZ2FuaXphdGlvbiBWYWxpZGF0aW9uIENBIC0gU0hBMjU2IC0g
RzIwHhcNMTYwODE5MTAxNTMwWhcNMTkwODIwMTAxNTMwWjBjMQswCQYDVQQGEwJC
RTEQMA4GA1UECBMHQnJ1c3NlbDEOMAwGA1UEBxMFRXZlcmUxGzAZBgNVBAoTElNj
YXJsZXQgQmVsZ2l1bSBOVjEVMBMGA1UEAwwMKi5zY2FybGV0LmJlMIIBIjANBgkq
hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAneqljhKZa3COnYkhzfuWt1/VqV5zawfW
d9PcBf7ucRVjvq4eib4QC5SdLsLuWVXAeg11BaMKB1wOCuw6FuPuuGUUuOQwY326
uEdBzL25vc2J/uvqSllq3Nd+8BMGlYWkV1vyeWWBgSeAnfnNPujowWIBrvrcZd9z
t6qZkf0NCgg22prJ2/XIx43IMwbP5GV5tRgh6NmcAOZdzyjF/Og3dFprWjGIIv70
VJ95aIh+6kfpplya08Wgmp/AktOgSUoiWhxPSaaEZxhgD/xZRGUxaTENV4xl16F5
TU6Z5z/ANaA211LFPEKihU8NPjZ8K99C54u9lFhilA+aR5tlDptmlQIDAQABo4IB
6TCCAeUwDgYDVR0PAQH/BAQDAgWgMIGgBggrBgEFBQcBAQSBkzCBkDBNBggrBgEF
BQcwAoZBaHR0cDovL3NlY3VyZS5nbG9iYWxzaWduLmNvbS9jYWNlcnQvZ3Nvcmdh
bml6YXRpb252YWxzaGEyZzJyMS5jcnQwPwYIKwYBBQUHMAGGM2h0dHA6Ly9vY3Nw
Mi5nbG9iYWxzaWduLmNvbS9nc29yZ2FuaXphdGlvbnZhbHNoYTJnMjBWBgNVHSAE
TzBNMEEGCSsGAQQBoDIBFDA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3dy5nbG9i
YWxzaWduLmNvbS9yZXBvc2l0b3J5LzAIBgZngQwBAgIwCQYDVR0TBAIwADBJBgNV
HR8EQjBAMD6gPKA6hjhodHRwOi8vY3JsLmdsb2JhbHNpZ24uY29tL2dzL2dzb3Jn
YW5pemF0aW9udmFsc2hhMmcyLmNybDAjBgNVHREEHDAaggwqLnNjYXJsZXQuYmWC
CnNjYXJsZXQuYmUwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMB0GA1Ud
DgQWBBTuPTxW/E7uF4zJUG0W1ZSqkRV6lDAfBgNVHSMEGDAWgBSW3mHxvRwWKVMc
wMx9O4MAQOYafDANBgkqhkiG9w0BAQsFAAOCAQEAQFYl4/NXajNwZBYoBzhRA+Fj
00gi5zQBUUnwTd+pjQjfnKgv58EIGzJOQ4xaS4WR6mG6IjkLvTbB/Uet83hJ9v2X
25BnQRKASSdWVvp0rE1bFUKDzyZeOgqP0pwfw1grnQqCYc0YkdNxayP/b42O46sG
ea322mJLDEFGsGyGkx9hEKCJQiBgi3MZKlAEA9kapCwyGk958cooSdWAe6Gyikg+
QFSvnPqLyWOnDr+cAIMJha1tQv07u7ZiNAvHWVj+SIfnBdXczwpUXr7irspJf8kd
qOMIIdMQbTkF7sRZFmqdMtn1rP2F6NiwvGjhTMkC+91WtotosNLva3JsSnraEg==
-----END CERTIFICATE-----
subject=/C=BE/ST=Brussel/L=Evere/O=Scarlet Belgium NV/CN=*.scarlet.be
issuer=/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Validation CA - SHA256 - G2
---
No client certificate CA names sent
---
SSL handshake has read 2647 bytes and written 456 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES256-SHA
    Session-ID: F08FA3992432F6DD9916240B4F3B5E2651E9A1B3489A41F45EB126A89C4B2EB5
    Session-ID-ctx: 
    Master-Key: A8BD4D21907BA43B8270B22076C82A734CF760AA7C4ECFC1E59C75AEF7E318C9E6C6B8960220F449A24499E595E582D5
    Key-Arg   : None
    Start Time: 1501333672
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
* OK IMAP4 Ready vilas 00021c3f
1 OK CAPABILITY
2 AUTHENTICATE PLAIN
2 NO AUTHENTICATE mechanism not supported, mate
3 LOGIN a b
3 NO Could not determine server

I believe with valid details LOGIN username password ought to work. Are you providing the correct username & password.

The fact they advertise AUTHENTICATE PLAIN (AUTH=PLAIN) and then don't implement it is bad on their part

[AEaut]

Good evening Philip,

Should we interprete, that k-9 is capable to follow such a method fallback (for whatever reason a server may require it), and that both fslori's and my failures to LOGIN were credentials errors?

Is perhaps

3 LOGIN a b
3 NO Could not determine server

an indicator that the LOGIN username requires the pattern user@domain.xy for server (domain.xy) determination ability?

Finally, if you agree that this could have been the case, why then did k-9 both at fslori and me not display the final response but rather the intermediate response about the method fallback?

[philipwhiuk]

Yes K-9 should be doing the fallback I described (hence why I re-titled the bug and left it open). I don't have an account with imap.scarlet.be so I couldn't check the full login process. But the 3 NO Could not determine server indicated to me it probably would work if my details were correct.

I'll look into why K-9 reported the error it did today by testing it with K-9 - as the trace shows I ran that from my Mac using OpenSSL.

[gnu010]

Hi there,

Thank you both for following this up! I am not familiar enough with the aspects of server connection, but I can add that:

• the login + password credentials that I use, are correct (I tried several times, plus they work in Thunderbird and in mySecureMail - the latter being not open-source though).
• I have also tried to use the username@scarlet.be as username, no success. Normally, the username will work as such (and I think the emailaddress will not work as a username), at least that's what works in Thunderbird.
• I have made a Scarlet mailbox for you to help you solve this issue (an extra mailbox under my account). So you are able do the necessary testing. This mailbox will exist temporarily, but of course for as long as you need it. I wish not to disclose its specifics here, but you can email me at gmail.com with prefix fs859lori and I will send you back the emailaddress, user name and password.
  • I already succesfully tested this account in Thunderbird.
  • I had the same issues in k-9-mail as before.
• As AEaut proposed, I have formulated my initial k-9-mail issue at Scarlet's client support (which I did not actually expect them to 'know about'). They responded by referring me to their site (dutch - french) where the use of Android (and others) is detailed for several types of smartphones, which is not the solution. I can try to go further along this path, but then I will need to know what to ask more specifically (and therefore more technical), and hope it is considered by their technical staff. And I don't know whether they'll be willing to consider modifications (I doubt it).

[philipwhiuk]

I'm fairly sure that this issue is fixed by the PR I raised. They respond with NO to our AUTHENTICATE PLAIN request so we don't fallback to LOGIN correctly. The fix I added makes us fallback in this case. Assuming the PR is merged it should be fixed in the next release.

Either this code path is rarely used in the real world or most providers, even though they don't support AUTHENTICATE PLAIN still respond with a continuation before sending back NO

[AEaut]

Hello fslori,

you are welcome!

I'll not write you for the test credentials, because i am no k-9 staff. ... Just in case anybody should write to you now, pretending his name was "AEaut" :]

[gnu010]

@philipwhiuk: OK, thanks for making the pull request! I will await its review and merging. Perhaps I will then compile it myself with gradle (on Ubuntu Xenial) and try installing and using it already. Which I've not done before, but the K9 documentation is most helpful :-).

[cketti]

Announcing support for AUTH=PLAIN but then answering with NO sounds like a server bug to me. I'd rather not add an automatic workaround for that.

Instead, we could allow the user to specify what authentication method to use. Right now we offer the options "Normal password", "Encrypted password" and "Client certificate" during setup. Using a client certificate should complement the password authentication and thus be a separate setting (see #793). That leaves us with "Normal password" and "Encrypted password", which are not terribly useful options. In my opinion the default should be "Automatic" where we select the option we like best and use that. Additionally, we list the individual authentication methods we support. That would allow users affected by this bug to manually select the LOGIN method.

Any objections to this approach of working around the issue at hand?

[Valodim]

Is there even any point to a non automatic setting? As long as we don't transmit the password in actual plaintext, we should use whatever works and doesn't bother the user.

For the extra security, a checkable setting like "force challenge response login" sounds more understandable than a dropdown that offers "encrypted" and something else, where anyone that chooses something else feels bad about that.

[cketti]

I'd hide the setting by default. "Automatic" should work fine for most people. The option to manually specify the authentication method would basically be an expert setting and only be there to work around server bugs or do intentionally stupid things like sending a plain text password over an unencrypted connection.

[AEaut]

like vincent i don't understand why an automatic fallback in case of a "NO" response should not be implemented (since it's implemented in other clients).

if it is because of password security concerns, then how about an automatic routine trying all the standards that are somehow "safe", and if they all fail, then reporting this to the user (sort of layer/popup/...) + asking him whether the routine shall continue by trying one (or all) other "unsafe" method(s).

Is there any chance to bear in mind and prevent "fail2ban"(-like) restrictions that may (increasingly) apply after a number of login/auth attempts that failed due to invalid credentials? Can such failures be distinguished against other failures (such as due to non-support of login/auth methods)? I'm thinking about account settings to limit the number of failed login/auth attempts (during a certain period).

[cketti]

I don't like the idea of trying every supported method after the server said "no" to your authentication request the first time. A well-behaving server wouldn't change its answer.
The goal is to make it possible to use non well-behaving servers nevertheless. But having to manually select an authentication type will make it painful enough that hopefully the server implementation will be fixed at some point.

[gnu010]

A comment just to keep you all informed. Last week, I assembled @philipwhiuk 's commits in his fix2648 branch (after merging with k-9's master), using gradle. The assembled package now works very well with the above-mentioned scarlet.be server, so I am very grateful for this. Of course, I'd absolutely welcome a solution that fixes the issue in an official release.

A few months ago, I've informed staff at scarlet about the server issue, but got no response.

[kikibelux]

Hi everybody,
Bonjour/dag Fslori,

I have instaled this k-9 app and it's impossible to read mail from scarlet but sending is ok.
My scarlet account is readable by thunderbird, mymail app and aquamail app.

what can_i do touse correctly k-9 ?
thanks a lot !

[cketti]

Contact scarlet.be support and tell them to fix their server software. Right now there's no way to use it with K-9 Mail.

[kikibelux]

Maibe do you know that scarlet.be use a mailbox with alias, seem same that prtonmail.com
I hope to help you.

[cketti]

See #3933