Trac-github suddenly stops working

[mattab]

Hello,

First of all thank you for this great plugin. We've been using it for a few weeks on http://dev.piwik.org and it is great!

However, today we have a problem that the plugin seems disabled, and we can't find a way to enable it again. There has been no change in the files. Do you know, how or why the plugin could stop working and appear disabled, very suddenly ? It was working 24 hours ago: http://dev.piwik.org/trac/ticket/3668#comment:2

We have tried reinstalling via;
$ cd /path/www/dev.piwik.org/trac/plugins
$ pip install -e git://github.com/aaugustin/trac-github.git#egg=TracGitHub-dev

To test I use:
$ curl --data "test=1" http://dev.piwik.org/trac/github
But that returns the "Error handler not found".

Also our timeline: http://dev.piwik.org/trac/timeline is not showing git commits anymore.

We compared the whole filesystem to backup and nothing changed in the last 24 hours.

Thank you very much for your help!

Matt

[Jellyfrog]

What does the log says?

Check your logging settings, http://dev.piwik.org/trac/admin/general/logging
Start with log level "warning" maybe

[mattab]

Well, this is certainly incredible. It is working again !

Now I do:

$ curl --data "test=1" http://dev.piwik.org/trac/github
Invalid payload

(as expected)

It's working again. I don't get it... we havent changed anything!

But thank you for the tip for the logging I will remember for next time to check there. Keep it up! Btw you can maybe put a Flattr on your page we would be glad to flattr you :)

[mattab]

FYI Looking at logs I found an error, that may be of interest:

2013-02-22 03:50:51,291 Trac[main] ERROR: Internal Server Error: 
Traceback (most recent call last):
  File "www/dev.piwik.org/trac/venv/lib/python2.6/site-packages/trac/web/main.py", line 497, in _dispatch_request
    dispatcher.dispatch(req)
  File "www/dev.piwik.org/trac/venv/lib/python2.6/site-packages/trac/web/main.py", line 170, in dispatch
    if handler.match_request(req):
  File "www/dev.piwik.org/trac/venv/src/tracgithub/tracext/github.py", line 36, in match_request
    req.args['rev'] = rev
  File "www/dev.piwik.org/trac/venv/lib/python2.6/site-packages/trac/web/api.py", line 316, in __getattr__
    value = self.callbacks[name](self)
  File "www/dev.piwik.org/trac/venv/lib/python2.6/site-packages/trac/web/api.py", line 299, in <lambda>
    'args': lambda req: arg_list_to_args(req.arg_list),
  File "www/dev.piwik.org/trac/venv/lib/python2.6/site-packages/trac/web/api.py", line 316, in __getattr__
    value = self.callbacks[name](self)
  File "www/dev.piwik.org/trac/venv/lib/python2.6/site-packages/trac/web/api.py", line 656, in _parse_arg_list
    value = unicode(value.value, 'utf-8')
UnicodeDecodeError: 'utf8' codec can't decode byte 0xce in position 1: invalid continuation byte

Maybe this exception should be caught in the plugin...

[aaugustin]

That happens when a request contains broken UTF-8.

I know Django contains code to deal with this problem, but I don't remember what it does exactly. The request isn't processable anyway.

The exception could be caught at the HTTP layer of Trac ie. trac.web. It's clearly outside of the scope of trac-github.

[rjollos]

Occasionally the post-commit hook doesn't work and I see the following in the logs:

2016-07-12 13:07:06,356 Trac[github] ERROR: Payload contains unknown commit 55cd9488ccd897bb9b75450852c100d13cf0df02

Next time it happens I will try to gather more info, but I think that the changeset isn't being pushed to the remote. Running a fetch on the repository seems to fix the issue.

[aaugustin]

Yes, we've seen that on Django as well. It appears to be a race condition where Trac is notified but the git fetch doesn't get the commit. I never figured out when it happens exactly.

[rjollos]

I wonder if we could reorder the code and implement a retry of git remote update --prune when an unknown commit is encountered. Have you given any thought to possible fixes/workarounds?

[aaugustin]

I remember trying various things, but that was several years ago, so unfortunately I don't remember the details.