GSSAPI support doesn't work anymore

[lenormf]

General informations

OfflineIMAP will prompt for a password everytime I try to pull my mails, even though I have a valid kerberos ticket that should be used to avoid asking for a password.

• system/distribution (with version): Archlinux
• offlineimap version (offlineimap -V): 7.2.0
• Python version: 2.7.14

I'm using OfflineIMAP 7.1.5 in the meantime, which doesn't have this issue.

Configuration file offlineimaprc

[general]
metadata = ~/.offlineimap
accounts = abc

[mbnames]
enabled = yes
filename = ~/mail/muttrc.mailboxes
header = "mailboxes "
peritem = "+%(accountname)s/%(foldername)s"
sep = " "
footer = "\n"
incremental = no

[Account abc]
localrepository = LocalAbc
remoterepository = RemoteAbc
maxage = 365
postsynchook = notmuch new

[Repository LocalAbc]
type = Maildir
localfolders = ~/mail/abc

[Repository RemoteAbc]
type = IMAP
ipv6 = False
remotehost = imap.abc.fr
ssl = yes
cert_fingerprint = F418B0437670D1AA2DB8C1B6434001C63EAFC5D1
tls_version = tls_compat
remoteport = 993
remoteuser = fle
holdconnectionopen = yes
expunge = yes
subscribedonly = no
createfolders = False

Logs, error

$ offlineimap -d imap -a abc                                          
OfflineIMAP 7.2.0                                                                          
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)           
imaplib2 v2.57 (bundled), Python v2.7.14, OpenSSL 1.1.0h  27 Mar 2018                      
Debug mode: Forcing to singlethreaded.                                                     
Now debugging for imap: IMAP protocol debugging                                            
Now debugging for : Other offlineimap related sync messages                                
  imaplib2: 2.57 (bundled)                                                                 
[imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
Remote repository 'RemoteAbc': type 'IMAP'                                               
Host: imap.abc.fr Port: 993 SSL: True                                                
Establishing connection to imap.abc.fr:993 (RemoteAbc)                             
[imap]: RemoteAbc: level 'tls_compat', version 'None'                                    
[imap]: Attempting GSSAPI authentication                                                   
[imap]: Attempting PLAIN authentication                                                    
Enter password for account 'RemoteAbc':

[chris001]

@lenormf Which remote mail server/mail service are you using. To run tests on the same mail server.

[lenormf]

Server is Dovecot, package version 2.2.13-12~deb8u4 on Debian Jessie. Looks like the capabilities are:

AUTH=GSSAPIA0001 ENABLE IDLE more LITERAL+ post-login listed have LOGIN-REFERRALS OK Pre-login SASL-IR capabilities IMAP4rev1 AUTH=PLAIN ID

[nicolas33]

This is likely because of 8872494 from @frozencemetery.

You should downgrade to v7.1.5. If this ain't fixed I'll revert the patch.

[chris001]

From the log, it looks like it tried GSSAPI (kerberos) first, it always is supposed to try kerberos first, then it fell back to PLAIN. Something went wrong at the kerberos authentication attempt. Are you sure your local kerberos config/environment is correct/working?

[frozencemetery]

If there's nothing in the debug log then from GSSAPI's perspective, auth should have completed.

Maybe the whole thing throws an exception? Could you add a check around line 324 (imapobj.authenticate('GSSAPI', self.__gsshandler)) to see?

[lenormf]

@chris001 Yes, I'm sure. Like I said in the first message, 7.1.5 works for me, so the way kerberos is handled has changed since.

@frozencemetery I will, when I get the chance (probs not this weekend though).

[iliastsi]

@lenormf Do you have the gssapi Python module installed? OfflineIMAP requires the gssapi module now, instead of pykerberos, and it will silently fail if not available (this is probably a bug, and should be fixed).

[lenormf]

@iliastsi Like I said in the first message, 7.1.5 works for me.

[iliastsi]

@lenormf v7.1.5 depends on the pykerberos Python module, whereas v7.2.0 depends on the gssapi Python module. Could you please verify that the gssapi Python module is installed?

[nicolas33]

OfflineIMAP requires the gssapi module now, instead of pykerberos, and it will silently fail if not available (this is probably a bug, and should be fixed).

You're right. There is a design issue because of two conflicting use cases. On one hand it's good to have the dependency optional to allow a gentle fallback when GSSAPI is not used and on the other hand we don't want to fallback when GSSAPI is required.

I wonder we have no choice but require the GSSAPI for everybody or introduce yet another configuration option to let the user require it.

BTW, I wonder if gssapi is enough for kerberos to work. Does it require another package (like a kerberos implementation) to be installed? Does gssapi fail at import or runtime if no kerberos implementation/dependency is found?

[chris001]

@nicolas33 The repo for this new gssapi is here: https://github.com/pythongssapi/python-gssapi
PyPI page: https://pypi.org/project/gssapi/
You're correct, gssapi doesn't implement kerberos, it's an interface to the locally installed kerberos package.
gssapi "basic requirements" are:

• A working implementation of GSSAPI (such as from MIT Kerberos) which includes header files

• a C compiler (such as GCC)

• either the enum34 Python package or Python 3.4+

• the six and decorator python packages

• Here's what happens when you run pip install gssapi in ubuntu 16.04:

:~# pip install gssapi
Collecting gssapi
  Using cached https://files.pythonhosted.org/packages/14/d7/e04be28ea985d6e398c20fbff585069a5e45569c96b657ec9d076026d2e1/gssapi-1.5.0.tar.gz
    Complete output from command python setup.py egg_info:
    /bin/sh: 1: krb5-config: not found
    In distributed package, building from C files...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-8irlh3w2/gssapi/setup.py", line 58, in <module>
        link_args = get_output('krb5-config --libs gssapi')
      File "/tmp/pip-install-8irlh3w2/gssapi/setup.py", line 38, in _get_output
        res = subprocess.check_output(*args, shell=True, **kwargs)
      File "/usr/lib/python3.5/subprocess.py", line 626, in check_output
        **kwargs).stdout
      File "/usr/lib/python3.5/subprocess.py", line 708, in run
        output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command 'krb5-config --libs gssapi' returned non-zero exit status 127

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-8irlh3w2/gssapi/

Clearly, this demonstrates we probably should install krb5-config system package, and other missing dependencies of gssapi.

• Solution for Debian OS using Heimdal Kerberos (needs testing. EDIT: WORKS on Ubuntu 16.04. ) ( https://stackoverflow.com/questions/30896343/how-to-install-gssapi-python-module ):

sudo apt install krb5-config krb5-user #added. probably need this before making links.
#sudo ln -s /usr/bin/krb5-config.mit /usr/bin/krb5-config #already there on Debian and Ubuntu.
#sudo ln -s /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 /usr/lib/libgssapi_krb5.so #already there on Ubuntu. Seems not needed on Debian either.
sudo apt-get install python-pip libkrb5-dev
sudo pip install gssapi

[nicolas33]

So, pykerbros was implementing the required set of kerberos while gssapi is a wrapper on top of a new external dependency. I'm fine with this change but we must better advise our users. I'm assuming that distributions have the new required kerberos dependency already packaged, too.

@lenormf Does it work if you install the gssapi module? How did you upgrade offlineimap?

For the authentication fallbacks, we might like to remove 'GSSAPI' from the default auth_methods. Either we require the users to manually tune auth_methods for kerberos or we add a new configuration option like auth_try_kerberos = yes. This way, we could keep both gssapi and the new external kerberos dependency optional.

We definitely must have a better documentation about the requirements.

Please, let me know if anybody has a better approach in mind.

@OfflineIMAP/collaborators @OfflineIMAP/offlineimap-testers @OfflineIMAP/maintainers Would this be ok for you?

[nicolas33]

For the authentication fallbacks, we might like to remove 'GSSAPI' from the default auth_methods. Either we require the users to manually tune auth_methods for kerberos or we add a new configuration option like auth_try_kerberos = yes. This way, we could keep both gssapi and the new external kerberos dependency optional.

Don't mind. This would only help users relying on kerberos and not reading our changelogs to upgrade their setup. There's no needed change in this area.

[nicolas33]

The main issue is that gssapi requires much more dependencies (including install-time dependencies). This likely won't help for the users running offlineimap on tiny systems like R-Pi.

[chris001]

It would be great to get input from gssapi team @frozencemetery maybe they have script code to dynamically install a compatible default kerberos package environment on all tested OS (debian, ubuntu, mac osx, bsd, centos/fedora/redhat, arch ,etc), this would help users who want to auth with kerberos, to get up and running faster, then all we need to do is provide runtime notice, your basic config of your kerberos client login is missing, add it in this file.

[chris001]

@nicolas33 OK I've researched it and tested the correct environment OS packages required for gssapi on debian/ubuntu. It's a start. See below for the package names
.
@lenormf
You running on Arch.

1. Can you share the equivalent Arch package install commands as the following (for ubuntu/debian):

sudo apt-get install krb5-config krb5-user python-pip python3-pip libkrb5-dev
sudo pip install gssapi

2. After you install these packages (above) on your Arch, and run offlineimap (v7.2.0) again, how far does it get in the kerberos auth login? It should at least attempt kerberos auth and provide you a different error message than previously, maybe something like "missing kerberos credentials/settings", or maybe it will find your kerberos settings in your user environment and login with kerberos successfully. Could your post back your results.

[lenormf]

Package names in Archlinux:

• krb5-config krb5-user libkrb5-dev → krb5
• python-pip → python2-pip
• python3-pip → python-pip
• gssapi → python2-gssapi

I installed the GSSAPI above and got the following error:

OfflineIMAP 7.2.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (bundled), Python v2.7.14, OpenSSL 1.1.0h  27 Mar 2018
Debug mode: Forcing to singlethreaded.
Now debugging for imap: IMAP protocol debugging
Now debugging for : Other offlineimap related sync messages
Account sync abc:
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 *** Processing account abc
 Establishing connection to imap.abc.fr:993 (RemoteAbc)
 [imap]: RemoteAbc: level 'tls_compat', version 'None'
 [imap]: Attempting GSSAPI authentication
 GSSAPI authentication failed: [UNAVAILABLE] Internal error occurred. Refer to server log for more information.
 [imap]: Attempting PLAIN authentication
imap.abc.fr reader:
 [imap]:   34:08.16 imap.abc.fr reader last 20 log messages:
Account sync abc:
 PLAIN authentication failed: command: AUTHENTICATE => socket error: <type 'exceptions.IOError'> - Too many read 0
 [imap]: Attempting LOGIN authentication
 [imap]: Attempting IMAP LOGIN authentication
Enter password for account 'RemoteAbc':

Look like the imapobj.authenticate('GSSAPI', self.__gsshandler) call line 324 raises an exception (it contains the message seen above). I'll try to check the server logs to get more information about what's failing.

EDIT: it looks that even the LOGIN (i.e. with direct password input) authentication method doesn't work either:

OfflineIMAP 7.2.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (bundled), Python v2.7.14, OpenSSL 1.1.0h  27 Mar 2018
Account sync abc:
 *** Processing account abc
 Establishing connection to imap.abc.fr:993 (RemoteAbc)
 GSSAPI authentication failed: [UNAVAILABLE] Internal error occurred. Refer to server log for more information.
 PLAIN authentication failed: command: AUTHENTICATE => socket error: <type 'exceptions.IOError'> - Too many read 0
Enter password for account 'RemoteAbc':
 LOGIN authentication failed: connection closed
 ERROR: All authentication types failed:
        GSSAPI: [UNAVAILABLE] Internal error occurred. Refer to server log for more information.
        PLAIN: command: AUTHENTICATE => socket error: <type 'exceptions.IOError'> - Too many read 0
        LOGIN: connection closed
 *** Finished account 'abc' in 0:06

I got a look at the server logs, it seems that the username received by the server is sketchy:

2018-04-23 08:48:20 auth: Error: passwd(lenormf@abc.fr,10.10.10.42,<a7xFlX1r8AK+0kvE>): user not found from userdb                                                  

I think the username should be lenormf only, in stead of the whole bunch of data between parenthesis?

[frozencemetery]

• pykerberos also requieres a Kerberos implmenetation; it doesn't implement Kerberos functionality itself.
• python-gssapi is available in most linux distros (including Debian, Fedora, RHEL/CentOS, etc.) and can just be installed from the package manager with appropriate deps. (Installing on MacOS should also work.)
• It suffices to have the krb5 headers around, if you need to build using pip - the easiest way for MIT to verify this is to install whatever provides /usr/bin/krb5-config.

(Please @ me if I missed a question in there; I'm trying to follow this thread as best I can.)

[nicolas33]

@lenormf You should not edit comments because we are not notified and it's harder to follow the issue.

it looks that even the LOGIN (i.e. with direct password input) authentication method doesn't work either

I'd say that the server closes the socket after the kerberos error and we don't recover from this.

Maybe @frozencemetery has hints about the kerberos login issue.

[frozencemetery]

Is there any way we can get information from the server? If it were sending back an error message, I'd expect to see it in the debug log (line 288).

A pcap might also be interesting.

Mostly I'm curious about whether we're throwing a non-GSSAPI exception somewhere.

[chris001]

@frozencemetery
When you run 7.2.0 and connect with gssapi on a dovecot server configured to support kerberos login, does it work for you?

[lenormf]

All I could get from the server is the error I posted earlier. I can't capture a PCAP log, I don't have access to the private key of the SSL certificate.

[frozencemetery]

I never tried dovecot; I only have access to a modified postfix instance (Zimbra). I'm happy to look at any logs and such, or test instances, or what have you.

@lenormf, any chance you could try adding the check as described in #541 (comment) ?

[chris001]

@frozencemetery OK interesting. As you know, postfix is only for sending mail (SMTP protocol on ports 25, 465, 587, 2525) yet OfflineIMAP only talks to IMAP servers for reading the user mailbox (IMAP protocol on ports 143, 993). SMTP and IMAP two totally different protocols with obviously different login procedures. It'd be great if you could try running 7.2.0 to connect to your Zimbra's mailbox on IMAP port 143 or 993 and share your results? Even with the dry run flag which won't modify any data it just shows what it would modify if it were not running a dry run. For the purpose of getting a successful GSSAPI login to your Zimbra IMAP server.

[frozencemetery]

Whoops, well, shows my knowledge I guess. For what it's worth, I tested this patch before submitting it...

For 88197a7:

rharwood@seton:~/offlineimap.git/master$ kinit rharwood
Password for rharwood@TEST.COM: 
rharwood@seton:~/offlineimap.git/master$ ./offlineimap.py --dry-run -d imap
OfflineIMAP 7.2.0                                                              
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (bundled), Python v2.7.15, OpenSSL 1.1.0h  27 Mar 2018
Debug mode: Forcing to singlethreaded.
Now debugging for imap: IMAP protocol debugging
Now debugging for : Other offlineimap related sync messages
Account sync TEST:
 [imap]: Using authentication mechanisms ['GSSAPI']
 *** Processing account TEST
 Establishing connection to mail.test.com:993 (Remote)
 [imap]: Remote: level 'tls_compat', version 'None'
 [imap]: Attempting GSSAPI authentication
 []: Filtering out 'Chats'[Remote] due to folderfilter
 []: Filtering out 'Contacts'[Remote] due to folderfilter
 []: Filtering out 'Drafts'[Remote] due to folderfilter
 []: Filtering out 'Emailed Contacts'[Remote] due to folderfilter
 []: Filtering out 'Junk'[Remote] due to folderfilter
 []: Filtering out 'Sent'[Remote] due to folderfilter
 []: Filtering out 'Trash'[Remote] due to folderfilter
 []: Not creating folder '' (repository 'Remote') as it would be filtered out on that repository.
 []: Copying folder structure from Remote to Local
 []: Not syncing filtered folder 'Chats'[Remote]
 []: Not syncing filtered folder 'Contacts'[Remote]
 []: Not syncing filtered folder 'Drafts'[Remote]
 []: Not syncing filtered folder 'Emailed Contacts'[Remote]
 Syncing INBOX: IMAP -> Maildir
 Loading message list for Maildir[INBOX]
 Message list for Maildir[INBOX] loaded: 0 messages
 Loading message list for IMAP[INBOX]
 [imap]: calling imaplib2 fetch command: '1:*' (FLAGS UID INTERNALDATE)
... messages redacted ...
 Message list for IMAP[INBOX] loaded: redacted messages
 Syncing messages IMAP[INBOX] -> Maildir[INBOX]
 [DRYRUN] Copy redacted messages from INBOX[Remote] to Local
 Syncing messages Maildir[INBOX] -> IMAP[INBOX]
 []: Not syncing filtered folder 'Junk'[Remote]
 []: Not syncing filtered folder 'Sent'[Remote]
 []: Not syncing filtered folder 'Trash'[Remote]
 *** Finished account 'TEST' in 0:02
rharwood@seton:~/offlineimap.git/master$ klist
Ticket cache: KEYRING:persistent:21259:krb_ccache_rUf3eU5
Default principal: rharwood@TEST.COM

Valid starting     Expires            Service principal
14/05/18 15:48:01  15/05/18 01:47:57  imap/mail.test@TEST.COM
14/05/18 15:47:57  15/05/18 01:47:57  krbtgt/TEST.COM@TEST.COM
rharwood@seton:~/offlineimap.git/master$ 

[nicolas33]

@lenormf Would it be possible to request the server admin to check the logs? Honestly, I don't know what to think about this issue.

On one hand it works before v7.2.0 so it's a regression and on the other hand the logs say it's internal server error. So, it's possible the latest kerberos implementation is raising a new issue on this server while the client is correct.

We need to know more.

[chris001]

@nicolas33 Good point, it's probably an error on the IMAP server side, there's many possibilities, like: out of memory error, out of disk space error, misconfiguration of kerberos config error, all on the server side. It'd be great if the Dovecot Debian 8 Jessie server admin could check the logs for your kerberos login auth @lenormf

[lenormf]

@frozencemetery I think I did that check at the time, but not exceptions were caught other than the protocol error. I'll do the check again and report back.

@chris001 I had him check already, that's how he told me that the client simply sent the wrong username in the identification phase. There are other clients on the network that work just fine (I think most them are whatever Thunderbird uses), I'm the only one with the issue. Guess I could have him check again, but I don't think we'll learn anything new.

[nicolas33]

@lenormf Sorry, I've missed this edit when reading this issue again. So, we are wrong to send bad authentication data. We need a fix.

[lenormf]

@frozencemetery I checked again, no exception other than the GSSAPI one is thrown during the authentication phase (below is the patch I used).

diff --git a/offlineimap/imapserver.py b/offlineimap/imapserver.py
index 95c4d66..f552a57 100644
--- a/offlineimap/imapserver.py
+++ b/offlineimap/imapserver.py
@@ -321,7 +321,12 @@ class IMAPServer(object):
 
         self.connectionlock.acquire()
         try:
-            imapobj.authenticate('GSSAPI', self.__gsshandler)
+            try:
+                imapobj.authenticate('GSSAPI', self.__gsshandler)
+            except imapobj.error as e:
+                raise
+            except Exception as e:
+                print("EXCEPTION: %s" % e)
             return True
         except imapobj.error as e:
             self.gssapi = False

[iliastsi]

This has also been reported on Debian, so it seems that the current implementation of GSSAPI in OfflineIMAP is indeed broken.

Looking at the code, I noticed that the previous implementation was using the username specified in the config file (remoteuser option) but the current one does not. So at the very least the implementation is not backwards compatible the user is not able to specify the username as they did before which may be causing the failures. @frozencemetery how is the user supposed to specify the username? I noticed in your comment that you run kinit rharwood before running OfflineIMAP. Is this how the previous implementation of GSSAPI used to work?

[frozencemetery]

I noticed in your comment that you run kinit rharwood before running OfflineIMAP. Is this how the previous implementation of GSSAPI used to work?

kint rharwood is how I get credentials for use with GSSAPI. offlineimap won't acquire credentials itself (technically, it of course could, but this isn't the usual paradigm). So yes, I believe it is how the previous implementation worked.

Looking at the code, I noticed that the previous implementation was using the username specified in the config file (remoteuser option) but the current one does not. So at the very least the implementation is not backwards compatible the user is not able to specify the username as they did before which may be causing the failures. @frozencemetery how is the user supposed to specify the username?

Thanks @iliastsi, this turns out to be an important question. "username" is an interesting notion. GSSAPI/Kerberos has a concept of a username, which will come from the credentials. (It can also be used to determine which credentials to use, but the old code doesn't do that - it would have been an argument to kerberos.authGSSClientInit() if it did.)

IMAP also has a username, and it seems possible to me that the two don't match. (For instance, missing REALM, capitalization, etc.) I don't know how this is captured in the protocol, but what pykerberos seems to do is add another protocol message after context negotiation is complete containing some useless junk and the preferred username. (Server I'm testing against doesn't seem to care about this.) pykerberos is absolutely the wrong layer for this to be happening, of course.

Anyway, if some folks could take a look at my latest PR and let me know if it fixes the problem, I would appreciate it. (#555)

[nicolas33]

@lenormf Could you test the above patch, please?

[lenormf]

That did it, the client successfully contacted the server. I pasted the log in the PR.

[nicolas33]

Patch applied in the "next" branch. The fix will be in next release (after v7.2.0).
Thank you all.

[lenormf]

I'd like to have a release sooner rather than later, if that's possible, as this is a fairly serious bug.

[nicolas33]

Ok. @iliastsi What do you think ?

[iliastsi]

I have no way to verify the fix, but since both @lenormf and @frozencemetery agree that this is resolved, it would be nice to have a bug-fix release :)

P.S. Unfortunately, the person who reported this on Debian has not yet provided feedback on the patch.

[nicolas33]

Planned for next week-end (16th).

[lenormf]

Thanks!

[chris001]

I think this is who reported the GSSAPI (Kerberos) bug to debian:
@zedinosaur
If it's you, could you test the patch and give your feedback.

[zedinosaur]

Sorry for not responding earlier. The new version just landed in unstable and I can confirm it fixes the problem for me. Thanks!