Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL: no shared cipher #457

Open
maximiliankaul opened this issue Apr 27, 2017 · 37 comments
Open

SSL: no shared cipher #457

maximiliankaul opened this issue Apr 27, 2017 · 37 comments

Comments

@maximiliankaul
Copy link
Contributor

General informations

  • system/distribution (with version):
    Arch Linux
$ uname -r
4.10.11-1-ARCH
  • offlineimap version (offlineimap -V):
    offlineimap v7.1.0, imaplib2 v2.57 (bundled), Python v2.7.13

  • Python version:
    Python 2.7.13 and Python 3.6.1

  • server name or domain:
    mail.maximiliankaul.de

  • CLI options:
    -dALL

Configuration file offlineimaprc

# Sample minimal config file.  Copy this to ~/.offlineimaprc and edit to
# get started fast.

[general]
accounts = main

[Account main]
localrepository = main-local
remoterepository = main-remote
#status_backend = sqlite
#autorefresh = 10
ui = quiet

[Repository main-local]
type = Maildir
localfolders = ~/Maildir

[Repository main-remote]
type = IMAP
remotehost = mail.maximiliankaul.de
ssl = yes
#cert_fingerprint = 56529243061ab509ab1265b3a1a504a87138512e
#sslcacertfile = ~/.offlineimap/lets-encrypt-x3-cross-signed.pem
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
#idlefolders = ['INBOX', 'INBOX.KTH']
holdconnectionsopen = yes

[mbnames]
enabled = yes
filename = ~/.mutt/mailboxes
header = "mailboxes "
peritem = "+%(foldername)s"
sep = " "
footer = "\n"

[Account kth]
localrepository = kth-local
remoterepository = kth-remote
#status_backend = sqlite

[Repository kth-local]
type = Maildir
localfolders = /tmp/kth/

[Repository kth-remote]
type = IMAP
auth_mechanisms = LOGIN
remotehost = webmail.kth.se
remoteuser = XXX
remotepass = XXX
ssl = yes
sslcacertfile = /etc/ssl/certs/DigiCert_High_Assurance_EV_Root_CA.pem

pythonfile (if any)

None

Logs, error

OfflineIMAP 7.1.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync main:
 [thread]: Register new thread 'Account sync main' (account 'main')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account main
 Establishing connection to mail.maximiliankaul.de:993 (main-remote)
 ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)
 ['  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 281, in syncrunner\n    self.__sync()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 344, in __sync\n    remoterepos.getfolders()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 627, in acquireconnection\n    exc_info()[2])\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 542, in acquireconnection\n    af=self.af,\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 194, in __init__\n    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2183, in __init__\n    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 361, in __init__\n    self.open(host, port)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 202, in open\n    super(WrappedIMAP4_SSL, self).open(host, port)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2196, in open\n    self.ssl_wrap_socket()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket\n    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)\n', '  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket\n    ciphers=ciphers)\n', '  File "/usr/lib/python2.7/ssl.py", line 611, in __init__\n    self.do_handshake()\n', '  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake\n    self._sslobj.do_handshake()\n']
 *** Finished account 'main' in 0:00
[thread]: Unregister thread 'Account sync main'
ERROR: Exceptions occurred during the run!
ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

Traceback:
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 281, in syncrunner
    self.__sync()
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 344, in __sync
    remoterepos.getfolders()
  File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 627, in acquireconnection
    exc_info()[2])
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 542, in acquireconnection
    af=self.af,
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2183, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 361, in __init__
    self.open(host, port)
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 202, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2196, in open
    self.ssl_wrap_socket()
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket
    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)
  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake
    self._sslobj.do_handshake()

Steps to reproduce the error

  • Just run offlineimap with the above config.

Notes

  • Note that the server only supports TLS 1.2
  • Note that OpenSSL recently got updated to 1.1 (which I would guess to be the problem here)
  • Setting ssl_version in config file did not help.
  • I can connect using s_client:
$ SSL_CERT_DIR="" openssl  s_client -connect mail.maximiliankaul.de:993 -CAfile /etc/ssl/certs/ca-certificates.crt
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = mail.maximiliankaul.de
verify return:1
---
Certificate chain
 0 s:/CN=mail.maximiliankaul.de
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGEDCCBPigAwIBAgISA0tgrEO+YYG9f3tczD9XrfObMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xNzA0MjcwNTI1MDBaFw0x
NzA3MjYwNTI1MDBaMCExHzAdBgNVBAMTFm1haWwubWF4aW1pbGlhbmthdWwuZGUw
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC0B4WbF0/ypi5U2afFa8yt
+q+Rdx/q3+f0MXB9g91+RMG9hKTZNLQoxp48xksdZJ27GjaLoWeKY1ZqfoM6nWhD
+tf2C7AAZrD8y9w/wPx+OUm5/KbOR4760Z3cWvshEKkVt2nEqouAVZGJLTvd+R7J
CFUmUx9W+anXuoNlfjTJXq+HQ5Q6ZJNMxRQRm/+7I+1g08axdKlev2T0C9vANAza
nphrntRp/FgkNFdrm7xV8Fpjz5mBYaneQ9DWgd1jZn1X5JJYDFvesQPHpvo8j2MY
I84JbYXKZcnCbZnCGxvzl0ogeegNWbVAFUwt+zh1oTBZh4yz9aW+a3lWmI7HBjtO
sCeMmteSv1zJu4R2RRlvYNCgr9o8TLeBeY0mQbn2F1QrIPoua/Yd9NOzNnj0N/1b
APczl1BNF+VDZXjuwfaoDtISlqU5pXF5OBdlAazGpU7/xN0BrGsAGDT1ZCElaWyS
Z/BgXKRK9qMxU0b+D3BlLslX5UIjkLYm9cB7igHiEk6//Aj0KJzEArNtxMlByOVa
i5nlbFvXmDLBQ9jfdhQSRzYWoEo2H3Hd8pRnE7bpCgyWdZLbUec8A7X1a9oiq12k
zhqwcX3q6/5S/QCfiGFLEU9dKr4xg56/X1Plnp1Pf/R5ydUtCSA03vdjRQkFgpGv
+yWRyIJyN3PgjaSlI86kIwIDAQABo4ICFzCCAhMwDgYDVR0PAQH/BAQDAgWgMB0G
A1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1Ud
DgQWBBRC9bfmKdYnf9TYG6+IqzCs7ugiSDAfBgNVHSMEGDAWgBSoSmpjBH3duubR
ObemRWXv86jsoTBwBggrBgEFBQcBAQRkMGIwLwYIKwYBBQUHMAGGI2h0dHA6Ly9v
Y3NwLmludC14My5sZXRzZW5jcnlwdC5vcmcvMC8GCCsGAQUFBzAChiNodHRwOi8v
Y2VydC5pbnQteDMubGV0c2VuY3J5cHQub3JnLzAhBgNVHREEGjAYghZtYWlsLm1h
eGltaWxpYW5rYXVsLmRlMIH+BgNVHSAEgfYwgfMwCAYGZ4EMAQIBMIHmBgsrBgEE
AYLfEwEBATCB1jAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5v
cmcwgasGCCsGAQUFBwICMIGeDIGbVGhpcyBDZXJ0aWZpY2F0ZSBtYXkgb25seSBi
ZSByZWxpZWQgdXBvbiBieSBSZWx5aW5nIFBhcnRpZXMgYW5kIG9ubHkgaW4gYWNj
b3JkYW5jZSB3aXRoIHRoZSBDZXJ0aWZpY2F0ZSBQb2xpY3kgZm91bmQgYXQgaHR0
cHM6Ly9sZXRzZW5jcnlwdC5vcmcvcmVwb3NpdG9yeS8wDQYJKoZIhvcNAQELBQAD
ggEBAG1QEBE3xKRHpFia9DrZSW3bgVjHsm9gbgYrl3b/sQQ6O2K90BtpzulJzqsY
QGMEW6le6+L0fWE2HDc/AX7aaDceRp//P7UBr5ycyWdK1cMd+5a3gDMY114hs2jn
YqkQH8sKTpY3AO9xICGnNfKedIEW08+bMvk8Op5VbauO2UU352JJcJnEIP+EKH+h
zl/fFTPuK4UC4DKrmDjJ1dhyQeTLBplh/5fYPXOeLNYbWr+F0l1Rp9pxwR5zpObR
JZbNBMV9vHfBL3/ZZUStHh32J39/9hIAAtUKJW5Y2z3VWjBjw3dhY7AWaBrmbx7X
zcZd8FP3/EaDXeX3XUdrM+/4Ar8=
-----END CERTIFICATE-----
subject=/CN=mail.maximiliankaul.de
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-384, 384 bits
---
SSL handshake has read 3679 bytes and written 334 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: ABED71B04EF6B2D937B0F08AA271BD317BAA579E082C4F6F8F92678D95E6EF39
    Session-ID-ctx: 
    Master-Key: 4BE8725D36189F126C47EE3A4B76112A92CC6D8F7AEA6801236872A7C0BC67933FDED4BAEFBADD6C282CF01DFC8F93FF
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - f1 fe 7d f9 5f af 54 65-b0 be bf 8b e6 5c ec fc   ..}._.Te.....\..
    0010 - c9 e9 59 c4 7a 97 f6 29-b2 ef 38 16 e2 8c 48 6e   ..Y.z..)..8...Hn
    0020 - 44 d0 05 ef c9 e5 64 2a-13 d3 87 da 41 ca 2a 13   D.....d*....A.*.
    0030 - 9b 7e 07 6b dc 9a 79 76-20 b9 af ca a2 8e 44 86   .~.k..yv .....D.
    0040 - a4 aa 39 55 c8 05 0a fa-2a 59 bc ae 27 96 1c aa   ..9U....*Y..'...
    0050 - 8b 79 a5 66 18 fb 0e b9-3c 21 bb df ee a8 66 0b   .y.f....<!....f.
    0060 - a5 93 c2 27 8e d0 b1 47-55 89 63 8e a4 d9 be a9   ...'...GU.c.....
    0070 - ec 4b 4a 43 e9 cc 74 df-40 bd b5 77 e7 25 3e 48   .KJC..t.@..w.%>H
    0080 - a7 65 82 b2 5e ed 80 c7-a3 3c b2 a9 f7 83 c0 a8   .e..^....<......
    0090 - ae 3c 8c de 6c bc 21 78-73 4e 17 e4 12 54 c3 71   .<..l.!xsN...T.q

    Start Time: 1493277809
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready.
QUIT
DONE
@chris001
Copy link
Member

You should try this, and report back:

[Repository main-remote]
ssl_version = tls1_2
tls_level = tls_secure

@maximiliankaul
Copy link
Contributor Author

@chris001 No luck. Same error. I tried both tls1_2 and tls_1_2 (from here).

@chris001
Copy link
Member

@maximiliankaul Same error?

ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

Different error?

@nicolas33
Copy link
Member

The ssl_version and tls_level must be set in the remote section (type Repository). ,-)

@maximiliankaul
Copy link
Contributor Author

@chris001 same error.

@nicolas33 I’m not sure I understand. Should I set it in [Repository main-remote]? That doesn’t work.

To clearify, this is the config I’m using to test your suggestion:

# Sample minimal config file.  Copy this to ~/.offlineimaprc and edit to
# get started fast.

[general]
accounts = main

[Account main]
localrepository = main-local
remoterepository = main-remote
#status_backend = sqlite
#autorefresh = 10
ui = quiet

[Repository main-local]
type = Maildir
localfolders = ~/Maildir

[Repository main-remote]
ssl_version = tls1_2
tls_level = tls_secure
type = IMAP
remotehost = mail.maximiliankaul.de
ssl = yes
#cert_fingerprint = 56529243061ab509ab1265b3a1a504a87138512e
#sslcacertfile = ~/.offlineimap/lets-encrypt-x3-cross-signed.pem
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
#idlefolders = ['INBOX', 'INBOX.KTH']
holdconnectionsopen = yes

[mbnames]
enabled = yes
filename = ~/.mutt/mailboxes
header = "mailboxes "
peritem = "+%(foldername)s"
sep = " "
footer = "\n"

[Account kth]
localrepository = kth-local
remoterepository = kth-remote
#status_backend = sqlite

[Repository kth-local]
type = Maildir
localfolders = /tmp/kth/

[Repository kth-remote]
type = IMAP
auth_mechanisms = LOGIN
remotehost = webmail.kth.se
remoteuser = XXX
remotepass = XXX
ssl = yes
sslcacertfile = /etc/ssl/certs/DigiCert_High_Assurance_EV_Root_CA.pem

And this is the error:

$ offlineimap -dALL
OfflineIMAP 7.1.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync main:
 [thread]: Register new thread 'Account sync main' (account 'main')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account main
 Establishing connection to mail.maximiliankaul.de:993 (main-remote)
 ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)
 ['  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 281, in syncrunner\n    self.__sync()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 344, in __sync\n    remoterepos.getfolders()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 627, in acquireconnection\n    exc_info()[2])\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 542, in acquireconnection\n    af=self.af,\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 194, in __init__\n    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2183, in __init__\n    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 361, in __init__\n    self.open(host, port)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 202, in open\n    super(WrappedIMAP4_SSL, self).open(host, port)\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2196, in open\n    self.ssl_wrap_socket()\n', '  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket\n    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)\n', '  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket\n    ciphers=ciphers)\n', '  File "/usr/lib/python2.7/ssl.py", line 611, in __init__\n    self.do_handshake()\n', '  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake\n    self._sslobj.do_handshake()\n']
 *** Finished account 'main' in 0:05
[thread]: Unregister thread 'Account sync main'
ERROR: Exceptions occurred during the run!
ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

Traceback:
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 281, in syncrunner
    self.__sync()
  File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 344, in __sync
    remoterepos.getfolders()
  File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 627, in acquireconnection
    exc_info()[2])
  File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 542, in acquireconnection
    af=self.af,
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2183, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 361, in __init__
    self.open(host, port)
  File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line 202, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 2196, in open
    self.ssl_wrap_socket()
  File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket
    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)
  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake
    self._sslobj.do_handshake()

nicolas33 added a commit to nicolas33/offlineimap that referenced this issue Apr 28, 2017
Github-ref: OfflineIMAP#457
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
@nicolas33
Copy link
Member

Please try above version so we get a bit more info while in debug mode. This will help understanding what's going on.

@maximiliankaul
Copy link
Contributor Author

The error looks to be different with your branch:

$ ./offlineimap.py -dALL
OfflineIMAP 7.1.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync main:
 [thread]: Register new thread 'Account sync main' (account 'main')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account main
 Establishing connection to mail.maximiliankaul.de:993 (main-remote)
 ERROR: While attempting to sync account 'main'
  not enough arguments for format string
 ['  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner\n    self.__sync()\n', '  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync\n    remoterepos.getfolders()\n', '  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/tmp/offlineimap/offlineimap/imapserver.py", line 531, in acquireconnection\n    self.repos.getname(), self.tlslevel, self.sslversion)\n']
 *** Finished account 'main' in 0:00
[thread]: Unregister thread 'Account sync main'
ERROR: can't have text and binary mode at once
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'main'
  not enough arguments for format string

Traceback:
  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner
    self.__sync()
  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync
    remoterepos.getfolders()
  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 531, in acquireconnection
    self.repos.getname(), self.tlslevel, self.sslversion)

ERROR: can't have text and binary mode at once

Adding parantheses in offlineimap/imapserver.py line 531 I get:

./offlineimap.py -dALL
OfflineIMAP 7.1.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync main:
 [thread]: Register new thread 'Account sync main' (account 'main')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account main
 Establishing connection to mail.maximiliankaul.de:993 (main-remote)
 [imap]: main-remote SSL: level tls_secure, version tls1_2
 ERROR: While attempting to sync account 'main'
  IMAP4 protocol error: program error: <class 'TypeError'> - cannot use a bytes pattern on a string-like object
 ['  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner\n    self.__sync()\n', '  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync\n    remoterepos.getfolders()\n', '  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/tmp/offlineimap/offlineimap/imapserver.py", line 544, in acquireconnection\n    af=self.af,\n', '  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__\n    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2183, in __init__\n    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 400, in __init__\n    self.welcome = self._request_push(name=\'welcome\', tag=\'continuation\').get_response(\'IMAP4 protocol error: %s\')[1]\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 201, in get_response\n    raise typ(exc_fmt % str(val))\n']
 *** Finished account 'main' in 0:00
[thread]: Unregister thread 'Account sync main'
ERROR: can't have text and binary mode at once
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'main'
  IMAP4 protocol error: program error: <class 'TypeError'> - cannot use a bytes pattern on a string-like object

Traceback:
  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner
    self.__sync()
  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync
    remoterepos.getfolders()
  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 544, in acquireconnection
    af=self.af,
  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2183, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 400, in __init__
    self.welcome = self._request_push(name='welcome', tag='continuation').get_response('IMAP4 protocol error: %s')[1]
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 201, in get_response
    raise typ(exc_fmt % str(val))

ERROR: can't have text and binary mode at once

@nicolas33
Copy link
Member

My patch was wrong and you correctly fixed it. Now, it looks like the ssl negociation is ok. My best guess is that something changed at server side. However, I don't get why this new error, yet.

What gives offlineimap --info?

nicolas33 added a commit that referenced this issue Apr 30, 2017
Github-ref: #457
Tested-by: Maximilian Kaul <https://github.com/maximiliankaul>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
@nicolas33
Copy link
Member

I wonder the above error is with Py3. Could you stick to Py2, please?

@gebrk
Copy link

gebrk commented Apr 30, 2017

I think this may be a problem with pyopenssl on python 2.7 and latest openssl (I tested using the same versions as @maximiliankaul), I've pasted below an attempt to manually make a TLS1.2 connection - FWIW the server is Dovecot 2.2.28 also on Arch:

In [1]: import socket
In [2]: import ssl
In [3]: ssl.OPENSSL_VERSION
Out[3]: 'OpenSSL 1.1.0e  16 Feb 2017'
In [4]: sor = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
In [5]: sw = ssl.wrap_socket(sor, ssl_version=ssl.PROTOCOL_TLSv1_2)
In [6]: sw.connect(('server', 993))

SSLError                                  Traceback (most recent call last)
<ipython-input-6-89c46c317283> in <module>()
----> 1 sw.connect(('server', 993))

/usr/lib64/python2.7/ssl.pyc in connect(self, addr)
    874         """Connects to remote ADDR, and then wraps the connection in
    875         an SSL channel."""
--> 876         self._real_connect(addr, False)
    877 
    878     def connect_ex(self, addr):

/usr/lib64/python2.7/ssl.pyc in _real_connect(self, addr, connect_ex)
    865                 self._connected = True
    866                 if self.do_handshake_on_connect:
--> 867                     self.do_handshake()
    868             return rc
    869         except (OSError, ValueError):

/usr/lib64/python2.7/ssl.pyc in do_handshake(self, block)
    838             if timeout == 0.0 and block:
    839                 self.settimeout(None)
--> 840             self._sslobj.do_handshake()
    841         finally:
    842             self.settimeout(timeout)

SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

@nicolas33
Copy link
Member

@GeorgeBrooke which versions of pyopenssl and openssl work for you?

@gebrk
Copy link

gebrk commented Apr 30, 2017

@nicolas33 I have not tried downgrading yet, but I believe the last working version was openssl-1.0.2.k and python2-pyopenssl-16.2.0

However looking at the server logs I believe this is a cipher mismatch with a confusing error message - I was using Mozilla's modern ssl cipherlist from https://wiki.mozilla.org/Security/Server_Side_TLS and switching back to the default dovecot setting or Mozilla's intermediate I can connect again.

@maximiliankaul
Copy link
Contributor Author

@nicolas33 Yes, I just executed ./offlineimap.py which defaults to python 3 on Arch Linux. Sorry.

I saw you included the debug info also in the official repository “next” branch which is what I’m using below:

$ git decribe
v7.1.0-13-g22a163a
$ python2 ./offlineimap.py -dALL
OfflineIMAP 7.1.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (bundled), Python v2.7.13
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync main:
 [thread]: Register new thread 'Account sync main' (account 'main')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account main
 Establishing connection to mail.maximiliankaul.de:993 (main-remote)
 [imap]: main-remote: level 'tls_secure', version 'tls1_2'
 ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)
 ['  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner\n    self.__sync()\n', '  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync\n    remoterepos.getfolders()\n', '  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/tmp/offlineimap/offlineimap/imapserver.py", line 629, in acquireconnection\n    exc_info()[2])\n', '  File "/tmp/offlineimap/offlineimap/imapserver.py", line 544, in acquireconnection\n    af=self.af,\n', '  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__\n    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2183, in __init__\n    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 361, in __init__\n    self.open(host, port)\n', '  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 202, in open\n    super(WrappedIMAP4_SSL, self).open(host, port)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2196, in open\n    self.ssl_wrap_socket()\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket\n    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)\n', '  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket\n    ciphers=ciphers)\n', '  File "/usr/lib/python2.7/ssl.py", line 611, in __init__\n    self.do_handshake()\n', '  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake\n    self._sslobj.do_handshake()\n']
 *** Finished account 'main' in 0:00
[thread]: Unregister thread 'Account sync main'
ERROR: Exceptions occurred during the run!
ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

Traceback:
  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner
    self.__sync()
  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync
    remoterepos.getfolders()
  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 629, in acquireconnection
    exc_info()[2])
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 544, in acquireconnection
    af=self.af,
  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2183, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 361, in __init__
    self.open(host, port)
  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 202, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2196, in open
    self.ssl_wrap_socket()
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket
    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)
  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake
    self._sslobj.do_handshake()

And the output of --info:

$ python2 ./offlineimap.py --info
OfflineIMAP 7.1.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (bundled), Python v2.7.13
  imaplib2: 2.57 (bundled)
Remote repository 'main-remote': type 'IMAP'
Host: mail.maximiliankaul.de Port: None SSL: True
Establishing connection to mail.maximiliankaul.de:993 (main-remote)
Failed to connect. Reason Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)
Establishing connection to mail.maximiliankaul.de:993 (main-remote)
Traceback (most recent call last):
  File "./offlineimap.py", line 26, in <module>
    oi.run()
  File "/tmp/offlineimap/offlineimap/init.py", line 85, in run
    self.__serverdiagnostics(options)
  File "/tmp/offlineimap/offlineimap/init.py", line 516, in __serverdiagnostics
    account.serverdiagnostics()
  File "/tmp/offlineimap/offlineimap/accounts.py", line 192, in serverdiagnostics
    self.ui.serverdiagnostics(remote_repo, 'Remote')
  File "/tmp/offlineimap/offlineimap/ui/UIBase.py", line 473, in serverdiagnostics
    folders = repository.getfolders()
  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 629, in acquireconnection
    exc_info()[2])
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 544, in acquireconnection
    af=self.af,
  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2183, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 361, in __init__
    self.open(host, port)
  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 202, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2196, in open
    self.ssl_wrap_socket()
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 548, in ssl_wrap_socket
    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)
  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake
    self._sslobj.do_handshake()
offlineimap.error.OfflineImapError: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

@GeorgeBrooke: yes, I think you are correct with your assumption. Here is a log from my server (dovecot):

Apr 30 16:43:18 kaul_aws dovecot[410]: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=X.X.X.X, lip=X.X.X.X, TLS handshaking: SSL_accept() failed: error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher, session=<Jl/TU2NOPsSyTtdO>

@nicolas33
Copy link
Member

no shared cipher

So, you get your answer. I have no idea how to configure the client or the server so they can share a cipher, though.

@chris001
Copy link
Member

chris001 commented May 1, 2017

Probably it's caused by imaplib2 not using the modern ssl cipher suite.
Imaplib2 should be upgraded to use newer ciphers than the ones popular with WinXP in 2003.
Such as TLS_ECDHE_RSA_WITH_AES-128_GCM_SHA256

https://wiki.mozilla.org/Security/Server_Side_TLS

Modern compatibility

For services that don't need backward compatibility, the parameters below provide a higher level of security. This configuration is compatible with Firefox 27, Chrome 30, IE 11 on Windows 7, Edge, Opera 17, Safari 9, Android 5.0, and Java 8.

Ciphersuites: ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
Versions: TLSv1.2
TLS curves: prime256v1, secp384r1, secp521r1
Certificate type: ECDSA
Certificate curve: prime256v1, secp384r1, secp521r1
Certificate signature: sha256WithRSAEncryption, ecdsa-with-SHA256, ecdsa-with-SHA384, ecdsa-with-SHA512
RSA key size: 2048 (if not ecdsa)
DH Parameter size: None (disabled entirely)
ECDH Parameter size: 256
HSTS: max-age=15768000
Certificate switching: None

0xC0,0x2C - ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD
0xC0,0x30 - ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD
0xCC,0x14 - ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=ECDSA Enc=ChaCha20(256) Mac=AEAD
0xCC,0x13 - ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=RSA Enc=ChaCha20(256) Mac=AEAD
0xC0,0x2B - ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD
0xC0,0x2F - ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0xC0,0x24 - ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384
0xC0,0x28 - ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384
0xC0,0x23 - ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA256
0xC0,0x27 - ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256

Rationale:

AES256-GCM is prioritized above its 128 bits variant, and ChaCha20 because we assume that most modern devices support AESNI instructions and thus benefit from fast and constant time AES.
We recommend ECDSA certificates with P256 as other curves may not be supported everywhere. RSA signatures on ECDSA certificates are permitted because very few CAs sign with ECDSA at the moment.
DHE is removed entirely because it is slow in comparison with ECDHE, and all modern clients support elliptic curve key exchanges.
SHA1 signature algorithm is removed in favor of SHA384 for AES256 and SHA256 for AES128.

@nicolas33
Copy link
Member

Good point.

I don't know what the python ssl module does when we pass pre-defined ciphers. Is it required they are all supported? What if ssl is not provided by openssl?

We should check this and likely update imaplib2. We might like to expose a new configuration option up to the user if the ssl module is very strict on the ciphers passed to it.

@chris001
Copy link
Member

chris001 commented May 1, 2017

@nicolas33
Copy link
Member

This tends to convince me that our best alternative is to have the users configure the ciphers by their own.

This is a quite rare issue, though. This is the first cipher issue I'm aware, at least.

OTOH, I wonder most users would use the feature wrong. Especially since we have very poor error messages from the IMAP servers. Also, I would not be surprised that IMAP servers silently change of SSL ciphers without notification to the users. We would not have found the issue if @maximiliankaul did'n't have access to the server logs. We might need to add a big warning if we do this.

@maximiliankaul What about configuring your server with new ciphers?

@nicolas33
Copy link
Member

@GeorgeBrooke Good job at finding the culprit! This was far from easy!

@maximiliankaul
Copy link
Contributor Author

@maximiliankaul What about configuring your server with new ciphers?

Any suggestions which ciphers to use? Currently my dovecot uses

$ doveconf | grep ssl_cipher_list
ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK

This is based on the aforementioned modern cipher list from Mozilla.

I think the configuration works as indendet:

$ nmap --script ssl-enum-ciphers -p 993 mail.maximiliankaul.de

Starting Nmap 7.40 ( https://nmap.org ) at 2017-05-02 14:57 CEST
Nmap scan report for mail.maximiliankaul.de (54.93.69.112)
Host is up (0.058s latency).
rDNS record for 54.93.69.112: maximiliankaul.de
PORT    STATE SERVICE
993/tcp open  imaps
| ssl-enum-ciphers: 
|   TLSv1.2: 
|     ciphers: 
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp384r1) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (secp384r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp384r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp384r1) - A
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp384r1) - A
|     compressors: 
|       NULL
|     cipher preference: server
|_  least strength: A

Nmap done: 1 IP address (1 host up) scanned in 4.68 seconds

For what it’s worth: offlineimap was fine with this configuration for months. I last changes this setting October 2016 and have been successfully using offlineimap until recently.
What changed was OpenSSL got updated to 1.1 (on both client and server). I’m happy to provide more information / test if that helps. Just let me know if you need anything.

@chris001
Copy link
Member

chris001 commented May 2, 2017

@maximiliankaul

Any suggestions which ciphers to use?

Try these:
ECDH+HIGH:DH+HIGH:HIGH:-3DES:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!LOW

@nicolas33 nicolas33 changed the title Offlineimap tries to use SSLv3 but server only supports TLSv1.2 SSL: no shared cipher May 2, 2017
@maximiliankaul
Copy link
Contributor Author

@chris001 your cipher list works.

@chris001
Copy link
Member

chris001 commented May 2, 2017

@maximiliankaul
Excellent.
Note to @nicolas33 I created it by combining the ones recommended from the python bug 20995, reference above.

@gebrk
Copy link

gebrk commented May 2, 2017

Just to clarify @maximiliankaul you set that cipher list in dovecot so it was necessary to make server-side changes to work around this?

I think this is a python/ssl regression and therefore not an offlineimap bug, but it is a regression there as the ciphers were working correctly before.

@maximiliankaul
Copy link
Contributor Author

Just to clarify @maximiliankaul you set that cipher list in dovecot so it was necessary to make server-side changes to work around this?

That’s correct.

@chris001
Copy link
Member

chris001 commented May 3, 2017

@GeorgeBrooke It's possible also to change of cipher list, on the python (IMAP client) side.

@nicolas33 The relevant line of code is here:

self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)

Only six of the 10 parameters are passed:
self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)

This is the python 2.7 function prototype for ssl.wrap_socket.
ssl.wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, ssl_version={see docs}, ca_certs=None, do_handshake_on_connect=True, suppress_ragged_eofs=True, ciphers=None)
As you can see, the cipher list (string as referenced above, ECDH+HIGH:DH+HIGH:HIGH:-3DES:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!LOW:!EXPORT) would be passed in as the 10th parameter.

Or, the SSLContext object can be called directly, to set the cipher list:
class ssl.SSLContext(protocol)
SSLContext.set_ciphers(ciphers)
SSLContext.set_ciphers('ECDH+HIGH:DH+HIGH:HIGH:-3DES:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!LOW:!EXPORT')

@maximiliankaul
Copy link
Contributor Author

@chris001 Are you saying that specifying ciphers should fix this? It does not fix it for me:

self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version, ciphers="ECDHE-ECDSA-AES256-GCM-SHA384")

Gives the following error (note: I only changed the server cipher list to test the ones you suggested earlier, but reverted back to my old cipher list):

OfflineIMAP 7.1.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (bundled), Python v2.7.13, OpenSSL 1.1.0e  16 Feb 2017
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync main:
 [thread]: Register new thread 'Account sync main' (account 'main')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account main
 Establishing connection to mail.maximiliankaul.de:993 (main-remote)
 [imap]: main-remote: level 'tls_secure', version 'tls1_2'
 ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)
 ['  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner\n    self.__sync()\n', '  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync\n    remoterepos.getfolders()\n', '  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/tmp/offlineimap/offlineimap/imapserver.py", line 629, in acquireconnection\n    exc_info()[2])\n', '  File "/tmp/offlineimap/offlineimap/imapserver.py", line 544, in acquireconnection\n    af=self.af,\n', '  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__\n    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2186, in __init__\n    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 361, in __init__\n    self.open(host, port)\n', '  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 202, in open\n    super(WrappedIMAP4_SSL, self).open(host, port)\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2199, in open\n    self.ssl_wrap_socket()\n', '  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 551, in ssl_wrap_socket\n    ciphers="ECDHE-ECDSA-AES256-GCM-SHA384")\n', '  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket\n    ciphers=ciphers)\n', '  File "/usr/lib/python2.7/ssl.py", line 611, in __init__\n    self.do_handshake()\n', '  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake\n    self._sslobj.do_handshake()\n']
 *** Finished account 'main' in 0:00
[thread]: Unregister thread 'Account sync main'
ERROR: Exceptions occurred during the run!
ERROR: Unknown SSL protocol connecting to host 'mail.maximiliankaul.de' for repository 'main-remote'. OpenSSL responded:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)

Traceback:
  File "/tmp/offlineimap/offlineimap/accounts.py", line 281, in syncrunner
    self.__sync()
  File "/tmp/offlineimap/offlineimap/accounts.py", line 344, in __sync
    remoterepos.getfolders()
  File "/tmp/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 629, in acquireconnection
    exc_info()[2])
  File "/tmp/offlineimap/offlineimap/imapserver.py", line 544, in acquireconnection
    af=self.af,
  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2186, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 361, in __init__
    self.open(host, port)
  File "/tmp/offlineimap/offlineimap/imaplibutil.py", line 202, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 2199, in open
    self.ssl_wrap_socket()
  File "/tmp/offlineimap/offlineimap/bundled_imaplib2.py", line 551, in ssl_wrap_socket
    ciphers="ECDHE-ECDSA-AES256-GCM-SHA384")
  File "/usr/lib/python2.7/ssl.py", line 943, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 840, in do_handshake
    self._sslobj.do_handshake()

@chris001
Copy link
Member

chris001 commented May 3, 2017

@maximiliankaul You picked one specific cipher. Probably this cipher is not shared between python client and your dovecot imap server. Could you try it with the string:
ECDH+HIGH:DH+HIGH:HIGH:-3DES:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!LOW:@STRENGTH
This cipher string allows a lot more possible ciphers to try and find a matching strong cipher to use between client and server.

@nicolas33
Copy link
Member

I think exposing the ciphers to our users is an interesting feature. However, this issue is rare enough and I won't implement it myself at this time.

Changes into imaplib2 must be sent upstream.
Patches are welcome!

@chris001
Copy link
Member

chris001 commented May 3, 2017

You can run this command on your python client system to see which ciphers are available to it.
openssl ciphers -s "ECDH+HIGH:DH+HIGH:HIGH:-3DES:-PSK:-aNULL:-eNULL:-MD5:-RC4:-EXPORT:-LOW:-DES:@STRENGTH"
This is probably a good cipher string to use on the python client side for imaplib2.
Or for better future proofing, imaplib2 could just use: HIGH:@STRENGTH, this includes some anonymous ciphers excluded by aNULL, yet, it's a reasonable default setting, and you don't hard code a cipher currently considered strongest in first place, when a new cipher might rank higher than it, in the future.

@maximiliankaul
Copy link
Contributor Author

I did some more wireshark debugging and everything seems to be fine on the offlineimap end.
The offlineimap TLS handshake is fine, there are (as mentioned before) simply no shared ciphers. I’m not sure whether OpenSSL removed (or renamed) some ciphers, but making sure that there is at least one cipher (specifically TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f) or using the list @chris001 suggested) makes everything work.

I still don’t understand why it did not use 0xc030 but I’m leaving it as it is for now.
2017-05-04-112741_3840x1200_scrot

@nicolas33 Should I close this issue, or do you want to keep it open until the ciphers are exposed to the users?

@nicolas33
Copy link
Member

I think we'd better keep this issue open.

@chris001
Copy link
Member

chris001 commented May 4, 2017

IMO the sensible thing is to have imaplib2 request that openssl use only high security strong ciphers by default.
One line of code, SSLContext.set_ciphers('HIGH:@STRENGTH') added to imaplib2 should make it work.

@daniele-athome
Copy link

I have this issue with a mail-in-a-box-like configuration, but on Debian latest stable (with libssl version 1.0.2l-2)

https://github.com/mail-in-a-box/mailinabox/blob/master/setup/mail-dovecot.sh#L88

ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256

If I revert to a weaker configuration it works:

ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL

@chris001
Copy link
Member

chris001 commented Oct 23, 2017

Best is to make our imaplib2 client prefer stronger ciphers. Not make all the dovecot servers of the world use weaker ciphers...

@daniele-athome
Copy link

I know, as a matter of fact I didn't do it :-) I'm looking for a patch to apply to imaplib2 to workaround this.

@chris001
Copy link
Member

The line of code to modify is here:
https://github.com/OfflineIMAP/offlineimap/blob/maint/offlineimap/bundled_imaplib2.py#L548
No time to make a patch at the moment, maybe later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants