Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128) #20

Closed
0x64746b opened this issue Aug 18, 2011 · 16 comments
Closed

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128) #20

0x64746b opened this issue Aug 18, 2011 · 16 comments
Milestone
0.11

Comments

@0x64746b
Copy link
Collaborator

alot dies reproducibly when trying to open a certain thread.

$ alot

Traceback (most recent call last):
  File "/usr/local/bin/alot", line 20, in <module>
    main()
  File "/usr/local/lib/python2.6/dist-packages/alot/init.py", line 90, in main
    args.colours,
  File "/usr/local/lib/python2.6/dist-packages/alot/ui.py", line 74, in __init__
    self.mainloop.run()
  File "/usr/local/lib/python2.6/dist-packages/urwid-1.0.0-py2.6-linux-i686.egg/urwid/main_loop.py", line 271, in run
    self.screen.run_wrapper(self._run)
  File "/usr/local/lib/python2.6/dist-packages/urwid-1.0.0-py2.6-linux-i686.egg/urwid/raw_display.py", line 228, in run_wrapper
    return fn()
  File "/usr/local/lib/python2.6/dist-packages/urwid-1.0.0-py2.6-linux-i686.egg/urwid/main_loop.py", line 292, in _run
    self.event_loop.run()
  File "/usr/local/lib/python2.6/dist-packages/urwid-1.0.0-py2.6-linux-i686.egg/urwid/main_loop.py", line 667, in run
    self._loop()
  File "/usr/local/lib/python2.6/dist-packages/urwid-1.0.0-py2.6-linux-i686.egg/urwid/main_loop.py", line 705, in _loop
    self._watch_files[fd]()
  File "/usr/local/lib/python2.6/dist-packages/urwid-1.0.0-py2.6-linux-i686.egg/urwid/main_loop.py", line 343, in _update
    self.process_input(keys)
  File "/usr/local/lib/python2.6/dist-packages/urwid-1.0.0-py2.6-linux-i686.egg/urwid/main_loop.py", line 432, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/usr/local/lib/python2.6/dist-packages/alot/ui.py", line 42, in keypress
    self.ui.apply_command(cmd)
  File "/usr/local/lib/python2.6/dist-packages/alot/ui.py", line 349, in apply_command
    cmd.apply(self)
  File "/usr/local/lib/python2.6/dist-packages/alot/command.py", line 88, in apply
    sb = buffer.ThreadBuffer(ui, self.thread)
  File "/usr/local/lib/python2.6/dist-packages/alot/buffer.py", line 192, in __init__
    self.rebuild()
  File "/usr/local/lib/python2.6/dist-packages/alot/buffer.py", line 232, in rebuild
    bars_at=bars)
  File "/usr/local/lib/python2.6/dist-packages/alot/widgets.py", line 213, in __init__
    self.sumline = self._build_sum_line()
  File "/usr/local/lib/python2.6/dist-packages/alot/widgets.py", line 235, in _build_sum_line
    self.sumw = MessageSummaryWidget(self.message, even=self.even)
  File "/usr/local/lib/python2.6/dist-packages/alot/widgets.py", line 361, in __init__
    sumstr = self.__str__()
  File "/usr/local/lib/python2.6/dist-packages/alot/widgets.py", line 366, in __str__
    return self.message.__str__()
  File "/usr/local/lib/python2.6/dist-packages/alot/message.py", line 58, in __str__
    return "%s (%s)" % (aname, self.get_datestring())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

Please let me know if you need further, more specific information on the mails.
@pazz
Copy link
Owner

pazz commented Aug 18, 2011

is that on master or development?

@pazz
Copy link
Owner

pazz commented Aug 18, 2011

it also might be related to you running python 2.6: it could be that formating unicode strings has changed.
Please try the following:

#/usr/bin/python
from datetime import datetime
now = datetime.now()
print '%s (%s)' % (u'\u9896', now)

another possibility is that aname contains a utf8 string that somehow gets handed to a unicode constructor,
causing a aname.decode('ascii'), which fails..

@pazz
Copy link
Owner

pazz commented Aug 18, 2011

another possibility:

import email
email.Utils.parseaddr(u'Patrick pazz@here.com')

Does this return a tuple of unicode strings?
(u'Patrick', u'pazz@here.com')
If it does, we're ok and you don't get unicode strings from notmuch.database.Message.get_header()
which means you should pull the new (2days?) bindings from notmuch master.

@0x64746b
Copy link
Collaborator Author

It's on master

dtk@blackboxx:/media/devel/notmuch/git/alot$ git branch
  fix_docu
* master

And it happens with python 2.7 as well

dtk@blackboxx:/media/devel/notmuch/git/alot$ python2.7 bin/alot 

Traceback (most recent call last):
  File "bin/alot", line 20, in <module>
    main()
  File "/media/devel/notmuch/git/alot/alot/init.py", line 90, in main
    args.colours,
  File "/media/devel/notmuch/git/alot/alot/ui.py", line 74, in __init__
    self.mainloop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 271, in run
    self.screen.run_wrapper(self._run)
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/raw_display.py", line 228, in run_wrapper
    return fn()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 292, in _run
    self.event_loop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 667, in run
    self._loop()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 705, in _loop
    self._watch_files[fd]()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 343, in _update
    self.process_input(keys)
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 432, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/media/devel/notmuch/git/alot/alot/ui.py", line 42, in keypress
    self.ui.apply_command(cmd)
  File "/media/devel/notmuch/git/alot/alot/ui.py", line 349, in apply_command
    cmd.apply(self)
  File "/media/devel/notmuch/git/alot/alot/command.py", line 88, in apply
    sb = buffer.ThreadBuffer(ui, self.thread)
  File "/media/devel/notmuch/git/alot/alot/buffer.py", line 192, in __init__
    self.rebuild()
  File "/media/devel/notmuch/git/alot/alot/buffer.py", line 232, in rebuild
    bars_at=bars)
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 213, in __init__
    self.sumline = self._build_sum_line()
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 235, in _build_sum_line
    self.sumw = MessageSummaryWidget(self.message, even=self.even)
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 361, in __init__
    sumstr = self.__str__()
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 366, in __str__
    return self.message.__str__()
  File "/media/devel/notmuch/git/alot/alot/message.py", line 58, in __str__
    return "%s (%s)" % (aname, self.get_datestring())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
dtk@blackboxx:/media/devel/notmuch/git/alot$

@0x64746b
Copy link
Collaborator Author 

dtk@blackboxx:~$ ipython
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39) 
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: from datetime import datetime

In [2]: now = datetime.now()

In [3]: print '%s (%s)' % (u'\u9896', now)
颖 (2011-08-18 14:35:06.762814)

In [4]:

@pazz
Copy link
Owner

pazz commented Aug 18, 2011

how about the second suggestion?

@0x64746b
Copy link
Collaborator Author

email.Utils.parseaddr(u'Patrick pazz@here.com')

Does this return a tuple of unicode strings?

well, kinda ^^

In [1]: import email

In [2]: email.Utils.parseaddr(u'Patrick pazz@here.com')
Out[2]: ('', u'Patrickpazz@here.com')

It returns a tuple of unicode strings, but the first one's empty as it doesn't seperate the name from the address ^^
Now that's nasty ^^

@0x64746b
Copy link
Collaborator Author 

$ cat email_test.py 
#!/usr/bin/python2.7

import email
import platform

print platform.python_version()
print email.Utils.parseaddr(u'Patrick pazz@here.com')
$ python2.7 email_test.py 
2.7.0+
('', u'Patrickpazz@here.com')
$

@pazz
Copy link
Owner

pazz commented Aug 18, 2011

haha: thats a c/p mistake with github:)
Its supposed to be
"Patrick patrick@here.com"
there

@0x64746b
Copy link
Collaborator Author 

In [1]: import email

In [2]: email.Utils.parseaddr(u'Patrick <pazz@here.com>')
Out[2]: (u'Patrick', u'pazz@here.com')

sweet as apple pie. will update notmuch.

@0x64746b
Copy link
Collaborator Author

Am I missing something?

dtk@blackboxx:/media/devel/notmuch/git/notmuch$ git fetch
remote: Counting objects: 13, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 7 (delta 5), reused 0 (delta 0)
Unpacking objects: 100% (7/7), done.
From git://notmuchmail.org/git/notmuch
   ee4579a..f10ec87  master     -> origin/master
dtk@blackboxx:/media/devel/notmuch/git/notmuch$ git status
# On branch master
# Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
#
nothing to commit (working directory clean)
dtk@blackboxx:/media/devel/notmuch/git/notmuch$ git merge origin
Updating ee4579a..f10ec87
Fast-forward
 bindings/python/notmuch/tag.py    |    2 +-
 bindings/python/notmuch/thread.py |   10 ++++++++--
 2 files changed, 9 insertions(+), 3 deletions(-)
dtk@blackboxx:/media/devel/notmuch/git/notmuch$ cd bindings/python/
dtk@blackboxx:/media/devel/notmuch/git/notmuch/bindings/python$ sudo python2.7 setup.py install
running install
running build
running build_py
copying notmuch/thread.py -> build/lib.linux-i686-2.7/notmuch
copying notmuch/database.py -> build/lib.linux-i686-2.7/notmuch
copying notmuch/filename.py -> build/lib.linux-i686-2.7/notmuch
copying notmuch/__init__.py -> build/lib.linux-i686-2.7/notmuch
copying notmuch/globals.py -> build/lib.linux-i686-2.7/notmuch
copying notmuch/tag.py -> build/lib.linux-i686-2.7/notmuch
copying notmuch/message.py -> build/lib.linux-i686-2.7/notmuch
running install_lib
copying build/lib.linux-i686-2.7/notmuch/thread.py -> /usr/local/lib/python2.7/dist-packages/notmuch
copying build/lib.linux-i686-2.7/notmuch/tag.py -> /usr/local/lib/python2.7/dist-packages/notmuch
byte-compiling /usr/local/lib/python2.7/dist-packages/notmuch/thread.py to thread.pyc
byte-compiling /usr/local/lib/python2.7/dist-packages/notmuch/tag.py to tag.pyc
running install_egg_info
Removing /usr/local/lib/python2.7/dist-packages/notmuch-0.6.egg-info
Writing /usr/local/lib/python2.7/dist-packages/notmuch-0.6.egg-info
dtk@blackboxx:/media/devel/notmuch/git/notmuch$ cd -
/media/devel/notmuch/git/alot
dtk@blackboxx:/media/devel/notmuch/git/alot$ python2.7 bin/alot 

Traceback (most recent call last):
  File "bin/alot", line 20, in <module>
    main()
  File "/media/devel/notmuch/git/alot/alot/init.py", line 90, in main
    args.colours,
  File "/media/devel/notmuch/git/alot/alot/ui.py", line 74, in __init__
    self.mainloop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 271, in run
    self.screen.run_wrapper(self._run)
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/raw_display.py", line 228, in run_wrapper
    return fn()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 292, in _run
    self.event_loop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 667, in run
    self._loop()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 705, in _loop
    self._watch_files[fd]()
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 343, in _update
    self.process_input(keys)
  File "/usr/local/lib/python2.7/dist-packages/urwid-1.0.0-py2.7-linux-i686.egg/urwid/main_loop.py", line 432, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/media/devel/notmuch/git/alot/alot/ui.py", line 42, in keypress
    self.ui.apply_command(cmd)
  File "/media/devel/notmuch/git/alot/alot/ui.py", line 349, in apply_command
    cmd.apply(self)
  File "/media/devel/notmuch/git/alot/alot/command.py", line 88, in apply
    sb = buffer.ThreadBuffer(ui, self.thread)
  File "/media/devel/notmuch/git/alot/alot/buffer.py", line 192, in __init__
    self.rebuild()
  File "/media/devel/notmuch/git/alot/alot/buffer.py", line 232, in rebuild
    bars_at=bars)
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 213, in __init__
    self.sumline = self._build_sum_line()
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 235, in _build_sum_line
    self.sumw = MessageSummaryWidget(self.message, even=self.even)
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 361, in __init__
    sumstr = self.__str__()
  File "/media/devel/notmuch/git/alot/alot/widgets.py", line 366, in __str__
    return self.message.__str__()
  File "/media/devel/notmuch/git/alot/alot/message.py", line 58, in __str__
    return "%s (%s)" % (aname, self.get_datestring())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
dtk@blackboxx:/media/devel/notmuch/git/alot$

@pazz
Copy link
Owner

pazz commented Aug 18, 2011

sorry, can't reproduce.. I'm usually using git pull, but fetch and merge should be the same thing.
could you fwd me the mail?

@0x64746b
Copy link
Collaborator Author

actually, it's m4z' "Heads-up! Diplomarbeit im Anflug." thread. U received that one?
Thing is, I'm not sure whether you'll need the whole (large -.-) thread to reproduce, or which individual mail, including attachments, ...
And whether it will be reproducible after having been fwd'ed.

@pazz
Copy link
Owner

pazz commented Aug 18, 2011

I received only todays mail. anything special in the authors headers of the prev. ones?

@0x64746b
Copy link
Collaborator Author

I used thunderbird's feature to forward a whole thread as an attachment. so... u got mail ;)
hth.

@0x64746b
Copy link
Collaborator Author

you fixed it. somehow. somewhere. in one of the modules, I guess. I realized it worked when I set my PYTHONPATH to use the modules in the git (instead of the installed ones) in order to do some debugging. The old, installed master had the problem, develop didn't have it anymore, but I just updated master, and it seems to be working there as well now (I only got a change in USAGE, so it must have been in an earlier, not yet installed merge).
Sry for all the trouble
dtk

@ghost ghost mentioned this issue Mar 19, 2012
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.11
Development

No branches or pull requests
2 participants
@pazz @0x64746b