Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

WebStat: wrong date handling on certain days #898

Open
jeromecaffaro opened this Issue · 1 comment

3 participants

@jeromecaffaro
Collaborator

Originally on 2012-02-07

The following exception is currently produced when trying to view statistics (key statistics and custom events) for "the last 3 months" with format "flot" (due to February having only 28th days? This one 29th? Or every month with less than 31 days?):

* 2012-02-07 09:50:36 -> ValueError: day is out of range for month (webstat_engine.py:1922:create_graph_trend_flot)

-* User details
                                     agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:10.0) Gecko/20100101 Firefox/10.0
                                     group: []
                                     guest: 0
                                  language: en
                              login_method: Local
                                  nickname: admin
precached_permitted_restricted_collections: []
                        precached_useadmin: True
                       precached_usealerts: True
                      precached_useapprove: True
                      precached_usebaskets: True
                       precached_usegroups: True
                        precached_useloans: True
                     precached_usemessages: True
             precached_usepaperattribution: True
                   precached_usepaperclaim: True
                        precached_usestats: True
                   precached_viewclaimlink: False
                 precached_viewsubmissions: True
                                   referer: </stats/comments_frequency?ln=en&collection=All&format=flot&timespan=last+month&s_date=02%2F07%2F2012+00%3A00&f_date=02%2F07%2F2012+09%3A50&action_gen=Generate>
                               remote_host: 
                                 remote_ip: 127.0.0.1
                                   session: 68b0ae9eeefbec1db979872c7d942cbb
                                       uid: 1
                                       uri: </stats/comments_frequency?ln=en&collection=All&format=flot&timespan=last+three+months&s_date=02%2F07%2F2012+00%3A00&f_date=02%2F07%2F2012+09%3A50&action_gen=Generate>
                   websearch_group_records: 10
                         websearch_helpbox: 1
                       websearch_latestbox: 1

-* Traceback details 

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/invenio/webinterface_handler_wsgi.py", line 462, in application
    ret = invenio_handler(req)
  File "/usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py", line 360, in _profiler
    return _handler(req)
  File "/usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py", line 407, in _handler
    return root._traverse(req, path, False, guest_p)
  File "/usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py", line 237, in _traverse
    return obj._traverse(req, path, do_head, guest_p)
  File "/usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py", line 250, in _traverse
    result = _check_result(req, obj(req, form))
  File "/usr/lib/python2.6/dist-packages/invenio/webstat_webinterface.py", line 238, in comments_frequency
    body=perform_display_keyevent('comments frequency', argd, req, ln=ln),
  File "/usr/lib/python2.6/dist-packages/invenio/webstat.py", line 1214, in perform_display_keyevent
    os.path.basename(filename), settings, ln=ln) + sql
  File "/usr/lib/python2.6/dist-packages/invenio/webstat.py", line 1520, in _perform_display_event
    create_graph_trend(data, path, settings)
  File "/usr/lib/python2.6/dist-packages/invenio/webstat_engine.py", line 1687, in create_graph_trend
    create_graph_trend_flot(trend, path, settings)
  File "/usr/lib/python2.6/dist-packages/invenio/webstat_engine.py", line 1922, in create_graph_trend_flot
    next = current.replace(day=31)
ValueError: day is out of range for month

-* Stack frame details

Frame application in /usr/lib/python2.6/dist-packages/invenio/webinterface_handler_wsgi.py at line 479
-------------------------------------------------------------------------------
       476             else:
       477                 req.flush()
       478         except:
---->  479             register_exception(req=req, alert_admin=True)
       480             if not req.response_sent_p:
       481                 req.status = HTTP_INTERNAL_SERVER_ERROR
       482                 req.headers_out['content-type'] = 'text/html'
-------------------------------------------------------------------------------
          start_response =  '<built-in method start_response of mod_wsgi.Adapter object at 0xb76819f8>'
                 environ =  [...]
                     req =  '<invenio.webinterface_handler_wsgi.SimulatedModPythonRequest object at 0x23edf94c>'
        possible_handler =  'None'
         possible_module =  'None'

Frame _profiler in /usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py at line 360
-------------------------------------------------------------------------------
       357                     raise Exception('Debugging requested, but no debugger registered: "%s"' % args['debug'])
       358             return _handler(req)
       359         else:
---->  360             return _handler(req)
       361 
       362     def _handler(req):
       363         """ This handler is invoked by mod_python with the apache request."""
-------------------------------------------------------------------------------
                _handler =  '<function _handler at 0x23edeb1c>'
                    args =  "{'timespan': ['last three months'], 'ln': ['en'], 'format': ['flot'], 'f_date': ['02/07/2012 09:50'], 'collection': ['All'], 'action_gen': ['Generate'], 's_date': ['02/07/2012 00:00']}"
                     req =  '<invenio.webinterface_handler_wsgi.SimulatedModPythonRequest object at 0x23edf94c>'

Frame _handler in /usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py at line 449
-------------------------------------------------------------------------------
       446 
       447             ## as suggested in
       448             ## <http://www.python.org/doc/2.3.5/lib/module-gc.html>
---->  449             del gc.garbage[:]
       450 
       451     return _profiler
       452 
-------------------------------------------------------------------------------
         allowed_methods =  "('GET', 'POST', 'HEAD', 'OPTIONS')"
                       g =  'None'
                 guest_p =  '0'
                     req =  '<invenio.webinterface_handler_wsgi.SimulatedModPythonRequest object at 0x23edf94c>'
                     uri =  "'/stats/comments_frequency'"
                    path =  "['stats', 'comments_frequency']"
                    root =  '<invenio.webinterface_layout.WebInterfaceInvenio object at 0x23edf26c>'
                bad_msie =  'None'

Frame _traverse in /usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py at line 237
-------------------------------------------------------------------------------
       234         # renderer. We even pass it the parsed form arguments.
       235         if path:
       236             if hasattr(obj, '_traverse'):
---->  237                 return obj._traverse(req, path, do_head, guest_p)
       238             else:
       239                 raise apache.SERVER_RETURN, apache.HTTP_NOT_FOUND
       240 
-------------------------------------------------------------------------------
                     obj =  '<invenio.webstat_webinterface.WebInterfaceStatsPages object at 0x23edf14c>'
                    name =  "'stats'"
                 guest_p =  '0'
                     req =  '<invenio.webinterface_handler_wsgi.SimulatedModPythonRequest object at 0x23edf94c>'
               component =  "'stats'"
                 do_head =  'False'
                    path =  "['comments_frequency']"
                    self =  '<invenio.webinterface_layout.WebInterfaceInvenio object at 0x23edf26c>'

Frame _traverse in /usr/lib/python2.6/dist-packages/invenio/webinterface_handler.py at line 250
-------------------------------------------------------------------------------
       247                 req.uri not in CFG_NO_LANG_RECOGNITION_URIS:
       248             ln = get_preferred_user_language(req)
       249             form.add_field('ln', ln)
---->  250         result = _check_result(req, obj(req, form))
       251         return result
       252 
       253     def __call__(self, req, form):
-------------------------------------------------------------------------------
                     obj =  '<bound method WebInterfaceStatsPages.comments_frequency of <invenio.webstat_webinterface.WebInterfaceStatsPages object at 0x23edf14c>>'
                    name =  "'comments_frequency'"
                    form =  "{'timespan': [Field('timespan', 'last three months')], 'ln': [Field('ln', 'en')], 'format': [Field('format', 'flot')], 'f_date': [Field('f_date', '02/07/2012 09:50')], 'collection': [Field('collection', 'All')], 'action_gen': [Field('action_gen', 'Generate')], 's_date': [Field('s_date', '02/07/2012 00:00')]}"
                 guest_p =  '0'
                     req =  '<invenio.webinterface_handler_wsgi.SimulatedModPythonRequest object at 0x23edf94c>'
               component =  "'comments_frequency'"
                 do_head =  'False'
                    path =  '[]'
                    self =  '<invenio.webstat_webinterface.WebInterfaceStatsPages object at 0x23edf14c>'

Frame comments_frequency in /usr/lib/python2.6/dist-packages/invenio/webstat_webinterface.py at line 238
-------------------------------------------------------------------------------
       235                 ln=ln)
       236 
       237         return page(title="Comments frequency",
---->  238                     body=perform_display_keyevent('comments frequency', argd, req, ln=ln),
       239                     navtrail="""<a class="navtrail" href="%s/stats/%s">Statistics</a>""" % \
       240                     (CFG_SITE_URL, (ln != CFG_SITE_LANG and '?ln=' + ln) or ''),
       241                     description="CDS, Statistics, Comments frequency",
-------------------------------------------------------------------------------
                auth_msg =  "'Authorization granted'"
                    form =  "{'timespan': [Field('timespan', 'last three months')], 'ln': [Field('ln', 'en')], 'format': [Field('format', 'flot')], 'f_date': [Field('f_date', '02/07/2012 09:50')], 'collection': [Field('collection', 'All')], 'action_gen': [Field('action_gen', 'Generate')], 's_date': [Field('s_date', '02/07/2012 00:00')]}"
                      ln =  "'en'"
                    self =  '<invenio.webstat_webinterface.WebInterfaceStatsPages object at 0x23edf14c>'
                     req =  '<invenio.webinterface_handler_wsgi.SimulatedModPythonRequest object at 0x23edf94c>'
               user_info =  "{'uid': 1, 'agent': 'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:10.0) Gecko/20100101 Firefox/10.0', 'uri': '/stats/comments_frequency?ln=en&collection=All&format=flot&timespan=last+three+months&s_date=02%2F07%2F2012+00%3A00&f_date=02%2F07%2F2012+09%3A50&action_gen=Generate', 'precached_usegroups': True, 'session': '68b0ae9eeefbec1db979872c7d942cbb', 'precached_useadmin': True, 'group': [], 'login_method': 'Local', 'websearch_latestbox': 1, 'precached_usepaperattribution': True, 'precached_usepape [...]
               auth_code =  '0'
                    argd =  "{'f_date': '02/07/2012 09:50', 'ln': 'en', 'format': 'flot', 'timespan': 'last three months', 'collection': 'All', 's_date': '02/07/2012 00:00', 'sql': 0}"

Frame perform_display_keyevent in /usr/lib/python2.6/dist-packages/invenio/webstat.py at line 1214
-------------------------------------------------------------------------------
      1211     else:
      1212         sql = ''
      1213     return out + _perform_display_event(data,
----> 1214                         os.path.basename(filename), settings, ln=ln) + sql
      1215 
      1216 
      1217 def perform_display_customevent(ids=[], args={}, req=None, ln=CFG_SITE_LANG):
-------------------------------------------------------------------------------
                  t_args =  "{'xtic_format': '%b', 't_format': '%Y-%m-%d', 't_fullname': 'Last three months', 't_start': '2011-12-01', 'collection': 'All', 'granularity': 'month', 't_end': '2012-02-07'}"
                    args =  "{'f_date': '02/07/2012 09:50', 'ln': 'en', 'format': 'flot', 'timespan': 'last three months', 'collection': 'All', 's_date': '02/07/2012 00:00', 'sql': 0}"
                   force =  'False'
                settings =  '{\'multiple\': None, \'format\': \'flot\', \'xtic_format\': \'%b\', \'title\': \'Comments frequency in collection "All"\', \'xlabel\': \'Last three months (month)\', \'ylabel\': \'Number of comments\'}'
                      ln =  "'en'"
                    data =  "[('2011-12-01 00:00:00', 18L), ('2012-01-01 00:00:00', 5L), ('2012-02-01 00:00:00', 0)]"
                     req =  '<invenio.webinterface_handler_wsgi.SimulatedModPythonRequest object at 0x23edf94c>'
                    list =  'False'
                   param =  "'collection'"
                filename =  "'webstat_comments_frequency_All_last_three_months'"
                event_id =  "'comments frequency'"
                 options =  "{'timespan': ('combobox', 'Time span', [('today', 'Today', '2012-02-07', '2012-02-08', 'hour', '%Y-%m-%d', '%H'), ('this week', 'This week', '2012-02-06', '2012-02-08', 'day', '%Y-%m-%d', '%a'), ('last week', 'Last week', '2012-01-30', '2012-02-06', 'day', '%Y-%m-%d', '%a'), ('this month', 'This month', '2012-02-01', '2012-02-08', 'day', '%Y-%m-%d', '%d'), ('last month', 'Last month', '2012-01-01', '2012-02-01', 'day', '%Y-%m-%d', '%d'), ('last three months', 'Last three months', '2011-12-01',  [...]
                    parr =  "'The comments frequency is the amount of comments written for all the documents.'"
                     sql =  "''"
                 choosed =  "{'timespan': 'last three months', 's_date': '02/07/2012 00:00', 'f_date': '02/07/2012 09:50', 'collection': 'All', 'format': 'flot'}"
                gatherer =  '<function <lambda> at 0x23ee9454>'
                   order =  "['collection', 'format', 'timespan']"
           allow_refresh =  'True'
                     out =  '\'<p>The comments frequency is the amount of comments written for all the documents.</p><form method="get">\\n        <input type="hidden" name="ln" value="en" /><table id="keyevent_table" class="searchbox">\\n                    <thead>\\n                        <tr><th class="searchboxheader">Collection</th><th class="searchboxheader">Output format</th><th class="searchboxheader">Time span</th><th class="searchboxheader"></th></tr>\\n                </thead>\\n                <tbody><tr valig [...]

Frame _perform_display_event in /usr/lib/python2.6/dist-packages/invenio/webstat.py at line 1520
-------------------------------------------------------------------------------
      1517         create_graph_table(data, path, settings)
      1518         return TEMPLATES.tmpl_display_event_trend_text(settings["title"], path, ln=ln)
      1519 
----> 1520     create_graph_trend(data, path, settings)
      1521     if settings["format"] == "asciiart":
      1522         out = TEMPLATES.tmpl_display_event_trend_ascii(
      1523             settings["title"], path, ln=ln)
-------------------------------------------------------------------------------
                      ln =  "'en'"
                    path =  "'/opt/cds-invenio/var/www/img/tmp_webstat_comments_frequency_All_last_three_months'"
                    data =  "[('2011-12-01 00:00:00', 18L), ('2012-01-01 00:00:00', 5L), ('2012-02-01 00:00:00', 0)]"
                    name =  "'webstat_comments_frequency_All_last_three_months'"
                settings =  '{\'multiple\': None, \'format\': \'flot\', \'xtic_format\': \'%b\', \'title\': \'Comments frequency in collection "All"\', \'xlabel\': \'Last three months (month)\', \'ylabel\': \'Number of comments\'}'

Frame create_graph_trend in /usr/lib/python2.6/dist-packages/invenio/webstat_engine.py at line 1687
-------------------------------------------------------------------------------
      1684     elif settings["format"] == 'gnuplot':
      1685         create_graph_trend_gnu_plot(trend, path, settings)
      1686     elif settings["format"] == "flot":
----> 1687         create_graph_trend_flot(trend, path, settings)
      1688 
      1689 
      1690 def create_graph_trend_ascii_art(trend, path, settings):
-------------------------------------------------------------------------------
                   trend =  "[('2011-12-01 00:00:00', 18L), ('2012-01-01 00:00:00', 5L), ('2012-02-01 00:00:00', 0)]"
                    path =  "'/opt/cds-invenio/var/www/img/tmp_webstat_comments_frequency_All_last_three_months'"
                settings =  '{\'multiple\': None, \'format\': \'flot\', \'xtic_format\': \'%b\', \'title\': \'Comments frequency in collection "All"\', \'xlabel\': \'Last three months (month)\', \'ylabel\': \'Number of comments\'}'

Frame create_graph_trend_flot in /usr/lib/python2.6/dist-packages/invenio/webstat_engine.py at line 1922
-------------------------------------------------------------------------------
      1919         current =  _to_datetime(maxx, '%Y-%m-%d %H:%M:%S')
      1920         next = current + granularity_td
      1921         if (granularity_td.seconds + granularity_td.days * 24 * 3600) > 2592000:
----> 1922             next = current.replace(day=31)
      1923         tics += 'xaxis: { mode:"time",min:parseDate%s("%s"),max:parseDate%s("%s")},'\
      1924             % (title, _to_datetime(minx, '%Y-%m-%d %H:%M:%S'), title, next)
      1925 
-------------------------------------------------------------------------------
                    size =  "['500', '400']"
                    minx =  "'2011-12-01 00:00:00'"
                    maxx =  "'2012-02-01 00:00:00'"
                    rows =  '3'
                   title =  "'Commentsfr'"
                   trend =  "[('2011-12-01 00:00:00', 18L), ('2012-01-01 00:00:00', 5L), ('2012-02-01 00:00:00', 0)]"
                settings =  '{\'multiple\': None, \'format\': \'flot\', \'xtic_format\': \'%b\', \'title\': \'Comments frequency in collection "All"\', \'xlabel\': \'Last three months (month)\', \'ylabel\': \'Number of comments\'}'
                    next =  'datetime.datetime(2012, 3, 3, 0, 0)'
                 current =  'datetime.datetime(2012, 2, 1, 0, 0)'
                    tics =  "'yaxis: {\\n                tickDecimals : 0\\n        },'"
          granularity_td =  'datetime.timedelta(31)'
                    path =  "'/opt/cds-invenio/var/www/img/tmp_webstat_comments_frequency_All_last_three_months'"
                   first =  '1'
                     out =  '\'<!--[if IE]><script language="javascript" type="text/javascript"\\n                    src="/js/excanvas.min.js"></script><![endif]-->\\n              <script language="javascript" type="text/javascript" src="/js/jquery.flot.min.js"></script>\\n              <script language="javascript" type="text/javascript" src="/js/jquery.flot.selection.min.js"></script>\\n              <script id="source" language="javascript" type="text/j [...]
                     row =  '2'
@tiborsimko
Owner

Originally on 2014-02-26

See also #1715

@lnielsen lnielsen added this to the v1.x milestone
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.