Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automagically fetch Gmail certificates #132

Open
ildar opened this issue Nov 3, 2014 · 13 comments
Open

Automagically fetch Gmail certificates #132

ildar opened this issue Nov 3, 2014 · 13 comments

Comments

@ildar
Copy link

ildar commented Nov 3, 2014

There's the special type = Gmail and the documentation says: Specify the Gmail user name. This is the only mandatory parameter. But having configured so it complains:
$ offlineimap

OfflineIMAP 6.5.6.1-devel
Licensed under the GNU GPL v2+ (v2 or any later version)
Account sync Z2Gmail:
*** Processing account Z2Gmail
Establishing connection to imap.gmail.com:993
ERROR: No CA certificates and no server fingerprints configured. You must conf
igure at least something, otherwise having SSL helps nothing.
*** Finished account 'Z2Gmail' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: No CA certificates and no server fingerprints configured. You must confi
gure at least something, otherwise having SSL helps nothing.

Is it possible to fetch certificates automatically?

@nicolas33 nicolas33 added this to the undefined milestone Jan 9, 2015
@nicolas33 nicolas33 changed the title Gmail special type of IMAP Automagically fetch Gmail certificates Jan 9, 2015
@yang-ling
Copy link

My configuration looks like this

[Repository Remote-gmail]

type = Gmail

remoteuser = user@gmail.com

remotepasseval = get_pass("my_user")

remotehost = imap.gmail.com
remoteport = 993

ssl = yes

sslcacertfile = /etc/ssl/certs/ca-certificates.crt

########## Advanced settings

maxconnections = 2

nametrans = trans_gmail_remote

get_pass and nametrans are methods in my pythonfile

I think you need at least configure sslcacertfile

I use Archlinux. If you also use it, you can just copy my config.

@nicolas33
Copy link
Member

I think Google is taking the XOAUTH way. This latest authentication method supports fetching of tokens.

@danilobellini
Copy link

I had the same problem here with OfflineIMAP 7.0.8 on Arch Linux. Adding this line from the @yang-ling config was enough:

sslcacertfile = /etc/ssl/certs/ca-certificates.crt

There's no ssl, remotehost and remoteport in my config. As it seems, either the sslcacertfile or some other configuration still unknown for me is required, and I think the message "This is the only mandatory parameter" should be improved/fixed to include this information.

@nicolas33 nicolas33 added bug and removed feature labels Nov 4, 2016
@nicolas33 nicolas33 reopened this Nov 4, 2016
@nicolas33
Copy link
Member

It appears that XOAUTH2 with refresh token looks incompatible with starttls.

@nicolas33 nicolas33 added feature and removed bug labels Nov 5, 2016
@nicolas33
Copy link
Member

nicolas33 commented Nov 5, 2016

It appears that XOAUTH2 with refresh token looks incompatible with starttls.

This is expected behaviour. The reason is that the Gmail type enables ssl so it must be configured correctly.

I think Google is taking the XOAUTH way. This latest authentication method supports fetching of tokens.

Also, XOAUTH2 does not prevent from having SSL working.

IOW, the original request to automatically download the certificate is valid.

@chris001
Copy link
Member

chris001 commented Nov 8, 2016

@nicolas33 I think you'll be very interested in this page because it contains all of the troubleshooting steps for sending mail, in one place, including the full details on the best ways to get Gmail working and encryption. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting
Especially liking the sections on "XOAUTH2", "Encryption" and "Opportunistic TLS".
Did you know, SSL on port 465 has been deprecated by the IETF since 1998! The only applications still trying to use it are Microsoft Exchange Server and Outlook! And they shouldn't! Only STARTTLS on port 587 is the right way!

@nicolas33
Copy link
Member

Thanks! I did not add XOAUTH2 in offlineimap and I'm not much concerned by this protocol (for now, at least). I'm having hard times with this thing.

The port 993 (SSL) is hardcoded for the Gmail type. This was done @aroig (if I'm right) way before XOAUTH2 was provided by Google.

This might be why STARTTLS won't work. I didn't check what imaplib2 does and how we use it in the STARTTLS case but I wouldn't be suprised we are trying the wrong port.

@chris001
Copy link
Member

chris001 commented Nov 9, 2016

Exactly, port 587 is for sending mail out thru SMTP such as how phpmailer does it. Port 993 is the right port for secure reading IMAPS with SSL or STARTTLS.
Here's the official documentation for XOAUTH2, with TLS, and the official Python code sample:
https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThrough
https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/python/oauth2.py

@aroig
Copy link
Member

aroig commented Nov 9, 2016

I'm landing here only now due to @nicolas33 mention. I'm not sure what this issue is about anymore...

Regarding the certificate problem originally reported, yes, with ssl, we need to configure the list of Certificate Authority certificates (the sslcacertfile). I've just discovered that you can also use OS-DEFAULT as a value and offlineimap will try to find it on the common locations.

The SSL protocol fetches the certificate for the imap server, which is signed by a certification authority. Then fetches the certificate of the certification authority, which is signed by another certification authority, and keeps going up the tree until it finds a certification authority in the local list of trusted certification authorities you have provided.

This scheme protects you against someone in your network pretending to be gmail's imap server and spying on your email. You trust that every CA is legal, and only signs a certificate for which it has checked the identity, etc. A local spy on your network will have a hard time finding a CA to sign his fake gmail server. Without this chain of trust, you would instead be trusting a local DNS server giving you back the IP of gmail.

Getting gmail's certificate over the wire, or any of the CA's needed to validate it, from an untrusted server, is a bad idea. It kind of defeats the purpose of the chain of trust. If you want to use the chain of trust, there is no way around it, you need a local file with a handful of trusted CA's, obtained via a package manager with signed packages, etc. What offlineimap can do is try to find the local file in the usual places (which apparently it can do). Alternatively, we could hard-code the path at installation time. Then distros can package offlineimap with the cacerts file properly configured.

Regarding the XOAUTH discussion, I have not tried XOAUTH yet (it is somewhere on my list). I'm not sure I understood what is being discussed here. Is there a problem with gmail's backend hard-coding port 993? When I touched the gmail code, this port was already hard-coded, but of course it can be made configurable if necessary.

@nicolas33
Copy link
Member

nicolas33 commented Nov 9, 2016

Regarding the XOAUTH discussion, I have not tried XOAUTH yet (it is somewhere on my list). I'm not sure I understood what is being discussed here. Is there a problem with gmail's backend hard-coding port 993? When I touched the gmail code, this port was already hard-coded, but of course it can be made configurable if necessary.

Ok, thanks. I was wrong, we are fine with port 993.

Getting gmail's certificate over the wire, or any of the CA's needed to validate it, from an untrusted server, is a bad idea.

Thank you. I'm not sure we want to support automagically fetching the certificate. Though, I would not be opposed to a configuration option like "automagically_fetch_certificate" as long as we explain this is less secured than certificate checks.

Now, about STARTTLS.

Port 993 is the right port for secure reading IMAPS with SSL or STARTTLS.

STARTTLS allows to enable SSL/TLS tunnels while starting in clear text mode. IOW, this elevates the security from clear text.

Here, I wonder XOAUTH2 and STARTTLS are exclusive or not in the first place. The reason I want to check this is because STARTTLS does NOT requires checking the certificate. IOW, this would be another way to "bypass" the certificate checks and not having the sslcacertfile configuration option set.
I think both are NOT compatible because STARTTLS and XOAUTH2 are meant for very different purposes.

BUT in imapserver.py, we have:

        # Authentication routines, hash keyed by method name
        # with value that is a tuple with
        # - authentication function,
        # - tryTLS flag,
        # - check IMAP capability flag.
        auth_methods = {
          "GSSAPI": (self.__authn_gssapi, False, True),
          "XOAUTH2": (self.__authn_xoauth2, True, True),
          "CRAM-MD5": (self.__authn_cram_md5, True, True),
          "PLAIN": (self.__authn_plain, True, True),
          "LOGIN": (self.__authn_login, True, False),
        }

where tryTLS is about STARTTLS. I'm not sure trying STARTTLS with XOAUTH2 is correct in the first place.

Perhaps the XOAUTH2 authors can help, here. @cscorley @Frizlab @Roguelazer could you confirm that trying STARTTLS with XOAUTH2 is intended? Would this work?

@nicolas33
Copy link
Member

I'm not sure trying STARTTLS with XOAUTH2 is correct in the first place.

Again, while this would be surprising (both are meant for different use cases) this limitation might not apply for all the IMAP servers out there. If Gmail (correctly, IMHO) does not allow STARTTLS with XOAUTH2 we might like to try STARTTLS with XOAUTH2 for other IMAP servers.

@chris001
Copy link
Member

chris001 commented Nov 9, 2016

Here's the official gmail python code sample for XOAUTH2 IMAP4 login. Note how the login is 4 simple lines of code. There's no mention of STARTTLS for the IMAP login.
https://github.com/google/gmail-oauth2-tools/blob/master/python/oauth2.py#L255
The SMTP sample however does use STARTTLS and port 587 smtp submission.

def TestImapAuthentication(user, auth_string):
  """Authenticates to IMAP with the given auth_string.
  Prints a debug trace of the attempted IMAP connection.
  Args:
    user: The Gmail username (full email address)
    auth_string: A valid OAuth2 string, as returned by GenerateOAuth2String.
        Must not be base64-encoded, since imaplib does its own base64-encoding.
  """
  print
  imap_conn = imaplib.IMAP4_SSL('imap.gmail.com')
  imap_conn.debug = 4
  imap_conn.authenticate('XOAUTH2', lambda x: auth_string)
  imap_conn.select('INBOX')


def TestSmtpAuthentication(user, auth_string):
  """Authenticates to SMTP with the given auth_string.
  Args:
    user: The Gmail username (full email address)
    auth_string: A valid OAuth2 string, not base64-encoded, as returned by
        GenerateOAuth2String.
  """
  print
  smtp_conn = smtplib.SMTP('smtp.gmail.com', 587)
  smtp_conn.set_debuglevel(True)
  smtp_conn.ehlo('test')
  smtp_conn.starttls()
  smtp_conn.docmd('AUTH', 'XOAUTH2 ' + base64.b64encode(auth_string))

We should consider adding this oauth2.py official gmail code, ship it with offlineimap as a library for offlineimap use, and rely on it for all XOAUTH2 auth and login, this could close any bug related to gmail secure oauth2 login.

nicolas33 added a commit that referenced this issue Nov 20, 2016
Github-ref: #132
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
@thekix
Copy link
Member

thekix commented Oct 27, 2020

Hi,

probably this patch from offlineimap (python 3) could help here:

OfflineIMAP/offlineimap3#17

Regards,
kix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants