Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot use remotepasseval with Python3 #373

Open
darnir opened this issue Aug 14, 2016 · 11 comments

Comments

@darnir
Copy link
Member

@darnir darnir commented Aug 14, 2016

General informations

  • system/distribution (with version): Arch Linux
  • offlineimap version (offlineimap -V): offlineimap v7.0.5, imaplib2 v2.55 (bundled)
  • Python version: Python 3.5.2
  • server name or domain:
  • CLI options: -o

Configuration file offlineimaprc

remotepasseval = get_pass('UdS')

pythonfile (if any)

#! /usr/bin/env python
from subprocess import check_output


def get_pass(account):
    return check_output(["pass", "Mail/" + account]).splitlines()[0]

Logs, error

OfflineIMAP 7.0.5
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
*** Processing account UdS
Establishing connection to a.remote.server (UdS-remote)
ERROR: While attempting to sync account 'UdS'
  'bytes' object has no attribute 'encode'
*** Finished account 'UdS' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'UdS'
  'bytes' object has no attribute 'encode'

Traceback:
  File "/usr/lib/python3.5/site-packages/offlineimap/accounts.py", line 271, in syncrunner
    self.__sync()
  File "/usr/lib/python3.5/site-packages/offlineimap/accounts.py", line 334, in __sync
    remoterepos.getfolders()
  File "/usr/lib/python3.5/site-packages/offlineimap/repository/IMAP.py", line 444, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/lib/python3.5/site-packages/offlineimap/imapserver.py", line 519, in acquireconnection
    self.__authn_helper(imapobj)
  File "/usr/lib/python3.5/site-packages/offlineimap/imapserver.py", line 422, in __authn_helper
    if func(imapobj):
  File "/usr/lib/python3.5/site-packages/offlineimap/imapserver.py", line 351, in __authn_plain
    imapobj.authenticate('PLAIN', self.__plainhandler)
  File "/usr/lib/python3.5/site-packages/offlineimap/bundled_imaplib2.py", line 705, in authenticate
    typ, dat = self._simple_command('AUTHENTICATE', mechanism.upper())
  File "/usr/lib/python3.5/site-packages/offlineimap/bundled_imaplib2.py", line 1692, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/usr/lib/python3.5/site-packages/offlineimap/bundled_imaplib2.py", line 1418, in _command
    literal = literator(data, rqb)
  File "/usr/lib/python3.5/site-packages/offlineimap/bundled_imaplib2.py", line 2283, in process
    ret = self.mech(self.decode(data))
  File "/usr/lib/python3.5/site-packages/offlineimap/imapserver.py", line 216, in __plainhandler
    passwd = self.__getpassword()
  File "/usr/lib/python3.5/site-packages/offlineimap/imapserver.py", line 169, in __getpassword
    self.password = self.repos.getpassword() or \
  File "/usr/lib/python3.5/site-packages/offlineimap/repository/IMAP.py", line 384, in getpassword
    return self.localeval.eval(passwd).encode('UTF-8')

Steps to reproduce the error

@darnir

This comment has been minimized.

Copy link
Member Author

@darnir darnir commented Aug 14, 2016

Setting the pythonfile to be:


def get_pass(account):
    return check_output(["pass", "Mail/" + account]).splitlines()[0].decode()

does not work since offlineimap expects it to be a byte object, not a string. But Python3 byte objects do not have an encode attribute

@drnedm

This comment has been minimized.

Copy link

@drnedm drnedm commented Sep 18, 2016

I had the same issue and found the following fixes it. However, using python3, offlineimap tries to authenticate using method=PLAIN first, which fails. Server log:

python3:
Sep 18 17:34:49 mail imapd-ssl[2327]: Connection, ip=[1.2.3.4]
Sep 18 17:34:50 mail imapd-ssl[2327]: LOGIN FAILED, method=PLAIN, ip=[1.2.3.4]
Sep 18 17:34:55 mail imapd-ssl[2327]: LOGIN, user=myUsername, ip=[1.2.3.4], port=[35990], protocol=IMAP

python2:
Sep 18 17:36:15 mail imapd-ssl[2327]: Connection, ip=[1.2.3.4]
Sep 18 17:36:16 mail imapd-ssl[2327]: LOGIN, user=myUsername, ip=[1.2.3.4], port=[35998], protocol=IMAP

No idea why that happens, but after an initial
PLAIN authentication failed: Login failed.

offlineimap works nicely with python2 and python3.

Hope that helps,

Cheers,
B

offlineimapStrByteFix.patch.txt

@nicolas33

This comment has been minimized.

Copy link
Member

@nicolas33 nicolas33 commented Sep 18, 2016

@BeerSerc Could send your signoff so I can merge? The chunks on imaplib2 must be sent to the imaplib2 project.

@drnedm

This comment has been minimized.

Copy link

@drnedm drnedm commented Sep 18, 2016

As I said, it still triggers the behaviour that OI now tries this PLAIN auth to the server, so I think it might be good if somebody else could have a look at it. Not a python3 expert myself ...

And as I'm new to github, how would I sign off an attached patch like this? Or do I need to fork and send a pull request? Or do you just want a signed-off line with an email address?

@dolohow

This comment has been minimized.

Copy link
Member

@dolohow dolohow commented Sep 18, 2016

And as I'm new to github, how would I sign off an attached patch like this?

When doing git commit add switch --signoff.

Or do I need to fork and send a pull request? Or do you just want a signed-off line with an email address?

@nicolas33 accepts both, but the commits or email patches need to be signed off.

@nicolas33

This comment has been minimized.

Copy link
Member

@nicolas33 nicolas33 commented Sep 18, 2016

I'll remove the chunk about UTF-8 encoding, too. This is the wrong place to do this. In IMAP.py the value of IMAPRepository.getuser() must be UTF-8 encoded, already..

@drnedm

This comment has been minimized.

Copy link

@drnedm drnedm commented Sep 19, 2016

This is all about python2 str vs python3 bytes. So if you remove the encode/decode stuff, we are back to zero where offlineimap does not work with python3 if you use remotepasseval.

@nicolas33

This comment has been minimized.

Copy link
Member

@nicolas33 nicolas33 commented Sep 19, 2016

I was talking the encode call in your patch. It's wrong to have two chained UTF-8 encode calls. With the patch they are for some case. If fixing one use case breaks other use cases, the patch is wrong.

@drnedm

This comment has been minimized.

Copy link

@drnedm drnedm commented Sep 29, 2016

Sorry for the delay. What I wanted to say is: My patch is nothing but encodes and decodes, because for some reason in 7.0.6/HEAD this is in some places mixed, what results in the problems the original reporter (and me) had.
If you remove all encodes/decodes, there is literally no patch left.
I don't know, where precisely this has to be fixed, as it's the first time I am looking at offlineimap code. And it's also why I didn't create a pull-request for it or signed it off, as I am not sure the patch is sane, just saying it fixes the problem and thus might help somebody with more knowledge than me to find the correct place to fix the string/bytes handling.

@nicolas33

This comment has been minimized.

Copy link
Member

@nicolas33 nicolas33 commented Sep 29, 2016

@BeerSerc Got it! Thanks!

@drnedm

This comment has been minimized.

Copy link

@drnedm drnedm commented Sep 29, 2016

I traced it down a little further. I am pretty convinced that the problem is the _quote() function in bundled_imaplib2.py +1648.

it reads

def _quote(self, arg):
return '"%s"' % arg.replace('', '\').replace('"', '"')

which fails if arg is your password as a byte string.
Unfortunately, if I change it to work with bytestrings, I get no backtrace anymore, but

ERROR: All authentication types failed:
PLAIN: Login failed.
LOGIN: Error in IMAP command received by server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.