Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'NoneType' object has no attribute 'get_newest_date' #46

Closed
lmacken opened this issue Aug 30, 2011 · 14 comments
Closed

Comments

@lmacken
Copy link
Contributor

lmacken commented Aug 30, 2011

I started with a fresh setup with the latest notmuch & alot, and now I'm getting this exception.

Traceback (most recent call last):
  File "bin/alot", line 20, in <module>
    main()
  File "/home/lmacken/code/github.org/alot/alot/init.py", line 95, in main
    args.colours,
  File "/home/lmacken/code/github.org/alot/alot/ui.py", line 75, in __init__
    self.mainloop.run()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 169, in run
    self.screen.run_wrapper(self._run)
  File "/usr/lib64/python2.7/site-packages/urwid/raw_display.py", line 233, in run_wrapper
    return fn()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 180, in _run
    self.draw_screen()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 360, in draw_screen
    canvas = self.widget.render(self.screen_size, focus=True)
  File "/usr/lib64/python2.7/site-packages/urwid/widget.py", line 134, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 568, in render
    focus and self.focus_part == 'body')
  File "/home/lmacken/code/github.org/alot/alot/buffer.py", line 41, in render
    return self.body.render(size, focus)
  File "/usr/lib64/python2.7/site-packages/urwid/widget.py", line 134, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/lib64/python2.7/site-packages/urwid/listbox.py", line 327, in render
    (maxcol, maxrow), focus=focus)
  File "/usr/lib64/python2.7/site-packages/urwid/listbox.py", line 214, in calculate_visible
    self._set_focus_complete( (maxcol, maxrow), focus )
  File "/usr/lib64/python2.7/site-packages/urwid/listbox.py", line 475, in _set_focus_complete
    (maxcol,maxrow), focus)
  File "/usr/lib64/python2.7/site-packages/urwid/listbox.py", line 445, in _set_focus_first_selectable
    (maxcol, maxrow), focus=focus)
  File "/usr/lib64/python2.7/site-packages/urwid/listbox.py", line 217, in calculate_visible
    focus_widget, focus_pos = self.body.get_focus()
  File "/home/lmacken/code/github.org/alot/alot/walker.py", line 34, in get_focus
    return self._get_at_pos(self.focus)
  File "/home/lmacken/code/github.org/alot/alot/walker.py", line 63, in _get_at_pos
    widget = self._get_next_item()
  File "/home/lmacken/code/github.org/alot/alot/walker.py", line 72, in _get_next_item
    next_widget = self.containerclass(next_obj, **self.kwargs)
  File "/home/lmacken/code/github.org/alot/alot/widgets.py", line 37, in __init__
    self.rebuild()
  File "/home/lmacken/code/github.org/alot/alot/widgets.py", line 44, in rebuild
    newest = self.thread.get_newest_date()
AttributeError: 'NoneType' object has no attribute 'get_newest_date'
@pazz
Copy link
Owner

pazz commented Sep 5, 2011

hmpf. for some reason my comment did not show up here.. might have to do with me being in a train. anyhow:

I can't reproduce this one, there must be something special with a particular thread in your initial search result.
Could you add a ui.logger.debug(tid) just before line 44 in widgets.py and look for particularities in the
thread?
thanks,
/p

@pazz
Copy link
Owner

pazz commented Oct 4, 2011

Traceback (most recent call last):
  File "/usr/local/bin/alot", line 20, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/alot/init.py", line 114, in main
    args.colours,
  File "/usr/local/lib/python2.7/dist-packages/alot/ui.py", line 79, in __init__
    self.mainloop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 274, in run
    self.screen.run_wrapper(self._run)
  File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py", line 229, in run_wrapper
    return fn()
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 307, in _run
    self.event_loop.run()
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 1159, in wrapper
    rval = f(*args,**kargs)
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 358, in _update
    self.process_input(keys)
  File "/usr/local/lib/python2.7/dist-packages/urwid/main_loop.py", line 447, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/usr/local/lib/python2.7/dist-packages/alot/ui.py", line 44, in keypress
    urwid.Frame.keypress(self, size, key)
  File "/usr/local/lib/python2.7/dist-packages/urwid/container.py", line 641, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/usr/local/lib/python2.7/dist-packages/alot/buffer.py", line 53, in keypress
    return self.body.keypress(size, key)
  File "/usr/local/lib/python2.7/dist-packages/urwid/listbox.py", line 776, in keypress
    return self._keypress_page_down((maxcol, maxrow))
  File "/usr/local/lib/python2.7/dist-packages/urwid/listbox.py", line 1177, in _keypress_page_down
    widget, pos = self.body.get_next(pos)
  File "/usr/local/lib/python2.7/dist-packages/alot/walker.py", line 39, in get_next
    return self._get_at_pos(start_from + 1)
  File "/usr/local/lib/python2.7/dist-packages/alot/walker.py", line 61, in _get_at_pos
    widget = self._get_next_item()
  File "/usr/local/lib/python2.7/dist-packages/alot/walker.py", line 70, in _get_next_item
    next_widget = self.containerclass(next_obj, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/alot/widgets.py", line 38, in __init__
    self.rebuild()
  File "/usr/local/lib/python2.7/dist-packages/alot/widgets.py", line 45, in rebuild
    newest = self.thread.get_newest_date()
AttributeError: 'NoneType' object has no attribute 'get_newest_date'

@lmacken
Copy link
Contributor Author

lmacken commented Oct 19, 2011

This issue is still preventing me from using alot. I cannot figure out how to view the thread that is causing problems. I'm running 'notmuch show ' and it's not displaying anything. Any ideas?

@lmacken
Copy link
Contributor Author

lmacken commented Oct 19, 2011

Ah, I'm starting to track this issue down now...

@lmacken
Copy link
Contributor Author

lmacken commented Oct 19, 2011

Ok, so the core of the problem here is with emails that have timestamps that are in the future.

Traceback (most recent call last):
  File "/home/lmacken/code/github.org/alot/alot/db.py", line 160, in get_thread
    return Thread(self, query.search_threads().next())
  File "/home/lmacken/code/github.org/alot/alot/db.py", line 200, in __init__
    self.refresh(thread)
  File "/home/lmacken/code/github.org/alot/alot/db.py", line 211, in refresh
    self._oldest_date = datetime.fromtimestamp(ts)
ValueError: year is out of range

In notmuch.vim, this message timestamp is displayed as (the future).

@pazz
Copy link
Owner

pazz commented Oct 20, 2011

i knew the problem would solve itselve if we wait long enough :)
Seriously, this looks rather ugly. First of: I can't see the connection between your two tracebacks:
if alot.db.Thread.i__init__ raises a ValueError, the code in IteratorWalkter should pass that along as its
only testing for StopIteration. (thats _get_next_item). It should rather fail with
'next_widget' referenced before assignment than returning None.

Regarding the timestamp issue: Could you provide such a problematic timestamp?
the datetime.fromtimestamp docs say

Return the local date corresponding to the POSIX timestamp, such as is returned by time.time(). This may raise ValueError, 
if the timestamp is out of the range of values supported by the platform C localtime() function. 
It’s common for this to be restricted to years from 1970 through 2038.

but on my system (ubuntu 11.04) i get

In [42]: datetime.datetime.fromtimestamp(2217928174)
Out[42]: datetime.datetime(2040, 4, 13, 12, 9, 34)

Seemingly i can go as high up as the year 10000 before i get these errors.
Maybe this has somethig to do with me being on 64bit?
notmuch.thread.get_oldest_timestamp's docs say:

Return type:    c_unit64

So: a fix should look like this:

  1. find out your local maximal value for timestamps
  2. catch ValueErrors, and set the datetimeobject to represent your maximal date.

This way we won't have to hardcode that value. moreover, the threadline widget could test for exactly
this value and display "invalid" or so instead.

@pazz
Copy link
Owner

pazz commented Oct 20, 2011

datetime doesnt seem to use time.localtime, as this works perfectly well here:

In [61]: time.localtime(503017928174)
Out[61]: time.struct_time(tm_year=17910, tm_mon=1, tm_mday=2, tm_hour=18, tm_min=16, tm_sec=14, tm_wday=6, tm_yday=2, tm_isdst=0)

@pazz pazz closed this as completed Oct 20, 2011
@pazz pazz reopened this Oct 20, 2011
@pazz
Copy link
Owner

pazz commented Oct 20, 2011

datetime.datetime.max might be our friend

@lmacken
Copy link
Contributor Author

lmacken commented Oct 20, 2011

Ok, so my last traceback is caused by me adding a 'logging.exception' when the Thread.init fails, so that is probably why you were confused.

When running 'notmuch show thread:ID', the date in the header is as follows: Date: 1314866652
Yet the thread.get_oldest_date() is returning 41493121656249600

The following patch works around the issue, but I have a feeling this is a bug somewhere else.

--- a/alot/db.py
+++ b/alot/db.py
@@ -204,8 +204,14 @@ class Thread(object):
         self._authors = thread.get_authors()
         self._subject = thread.get_subject()
         ts = thread.get_oldest_date()
-        self._oldest_date = datetime.fromtimestamp(ts)
-        self._newest_date = datetime.fromtimestamp(thread.get_newest_date())
+        try:
+            self._oldest_date = datetime.fromtimestamp(ts)
+        except ValueError: # year is out of range
+            self._oldest_date = datetime.min
+        try:
+            self._newest_date = datetime.fromtimestamp(thread.get_newest_date())
+        except ValueError: # year is out of range
+            self._newest_date = datetime.max
         self._tags = set([t for t in thread.get_tags()])
         self._messages = {}  # this maps messages to its children
         self._toplevel_messages = []
diff --git a/alot/message.py b/alot/message.py
index e00d62d..2fb41e0 100644
--- a/alot/message.py
+++ b/alot/message.py
@@ -43,7 +43,10 @@ class Message(object):
         self._id = msg.get_message_id()
         self._thread_id = msg.get_thread_id()
         self._thread = thread
-        self._datetime = datetime.fromtimestamp(msg.get_date())
+        try:
+            self._datetime = datetime.fromtimestamp(msg.get_date())
+        except ValueError: # year is out of range
+            self._datetime = datetime.max
         self._filename = msg.get_filename()
         self._from = msg.get_header('From')
         self._email = None  # will be read upon first use

@pazz
Copy link
Owner

pazz commented Oct 20, 2011

@logging.exception: ahyes, ok makes sense, thx.

quick question: does fromtimestamp also throw a ValueError if the timestamp is below datetime.min ? if so,
returnung max here is not what we want.
It might be a better idea to jyst set the value to None, indicating 'invalid' (and adjusting the widgets of course).
thoughts?

Unfortunately, this might well be an error in the notmuch lib.

One more thing: could you have a look at the source of that message and post the date-header here?
That way, I could write this into a dummy mail for debugging. In case the timestamp is well formated,
this issue looks like a problem in the bindings/lib.

Is it only one single mail that causes the problem?

@pazz
Copy link
Owner

pazz commented Oct 20, 2011

oh wait: Date: 1314866652 is the raw string in the mail? thats not valid!

@lmacken
Copy link
Contributor Author

lmacken commented Oct 20, 2011

Yup, it's definitely not valid, and the spam checker headers in that message also mention that it is invalid.

@pazz
Copy link
Owner

pazz commented Oct 21, 2011

ok: i'd favor something like this:

  • set the datetime value to None,
  • adjust the widgets that display this (Threadline, MessageHeader I believe)

Moreover we must take some care when designing the new™ Messagelist buffer that will replace threadbuffer
as to None values in dates when sorting.

It would be cool if we had some sort of validity checker for header (is there something in stdlib?). Then we could
highlight invalid headerlines in the MessageHeaderWidget.

I don't have much time atm, but will hack on a fix here soon if you don't beat me to it :)

pazz added a commit that referenced this issue Oct 21, 2011
@pazz
Copy link
Owner

pazz commented Oct 21, 2011

found the time :)

this should fix the issue for you. Strangely, I still cant reproduce it as a call to notmuch.database.thread.get_newest_date() on a thread with your datestring
will return datetime.datetime(1970, 1, 1, 0, 59, 59) for me.

pazz added a commit that referenced this issue Oct 21, 2011
@pazz pazz closed this as completed Oct 25, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants