New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Base web hook fails with non ascii characters in payload #3321

Closed
Frodox opened this Issue Jun 14, 2017 · 3 comments

Comments

Projects
None yet
3 participants
@Frodox
Member

Frodox commented Jun 14, 2017

buildbot-0.9.7

...buildbot-contrib/master/contrib$ ./post_build_request.py -w "привет привет не работает" -H bb-master -r kernel -P kernel -c "не asci комментарий" -R "test"

and in master/twisted.log

2017-06-07 14:28:38+0300 [_GenericHTTPChannelProtocol,10,127.0.0.1] Attempting to load module buildbot.www.hooks.base
2017-06-07 14:28:38+0300 [_GenericHTTPChannelProtocol,10,127.0.0.1] Got the following changes [{'category': None, 'codebase': None, 'repository': 'kernel', 'author': '\xd0\xbf\xd1\x80\xd0\xb8\xd0\xb2\xd0\xb5\xd1\x82 \xd0\xbf\xd1\x80\xd0\xb8\xd0\xb2\xd0\xb5\xd1\x82 \xd0\xbd\xd0\xb5 \xd1\x80\xd0\xb0\xd0\xb1\xd0\xbe\xd1\x82\xd0\xb0\xd0\xb5\xd1\x82', 'when': None, 'comments': u'\u043d\u0435 asci \u043a\u043e\u043c\u043c\u0435\u043d\u0442\u0430\u0440\u0438\u0439', 'project': 'kernel', 'branch': None, 'revlink': None, 'properties': {}, 'files': [], 'revision': 'test'}]
2017-06-07 14:28:38+0300 [_GenericHTTPChannelProtocol,10,127.0.0.1] Payload: {'project': ['kernel'], 'revision': ['test'], 'repository': ['kernel'], 'comments': ['\xd0\xbd\xd0\xb5 asci \xd0\xba\xd0\xbe\xd0\xbc\xd0\xbc\xd0\xb5\xd0\xbd\xd1\x82\xd0\xb0\xd1\x80\xd0\xb8\xd0\xb9'], 'author': ['\xd0\xbf\xd1\x80\xd0\xb8\xd0\xb2\xd0\xb5\xd1\x82 \xd0\xbf\xd1\x80\xd0\xb8\xd0\xb2\xd0\xb5\xd1\x82 \xd0\xbd\xd0\xb5 \xd1\x80\xd0\xb0\xd0\xb1\xd0\xbe\xd1\x82\xd0\xb0\xd0\xb5\xd1\x82']}
2017-06-07 14:28:38+0300 [_GenericHTTPChannelProtocol,10,127.0.0.1] WARNING: change source is using deprecated self.master.addChange method; this method will disappear in Buildbot-1.0.0
2017-06-07 14:28:38+0300 [_GenericHTTPChannelProtocol,10,127.0.0.1] adding changes from web hook
        Traceback (most recent call last):
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py", line 1447, in unwindGenerator
            return _inlineCallbacks(None, gen, Deferred())
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py", line 1301, in _inlineCallbacks
            result = g.send(result)
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/buildbot/www/change_hook.py", line 158, in submitChanges
            change = yield self.master.addChange(src=src, **chdict)
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py", line 1447, in unwindGenerator
            return _inlineCallbacks(None, gen, Deferred())
        --- <exception caught here> ---
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py", line 1301, in _inlineCallbacks
            result = g.send(result)
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/buildbot/master.py", line 485, in addChange
            kwargs[k] = ascii2unicode(kwargs[k])
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/buildbot/util/__init__.py", line 239, in ascii2unicode
            return bytes2unicode(x, encoding='ascii', errors=errors)
          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/buildbot/util/__init__.py", line 235, in bytes2unicode
            return text_type(x, encoding, errors)
        exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)
@tardyp

This comment has been minimized.

Show comment
Hide comment
@tardyp

tardyp Jun 14, 2017

Member

we should use bytes2unicode instead of ascii2unicode. Would you like to make a patch?

Member

tardyp commented Jun 14, 2017

we should use bytes2unicode instead of ascii2unicode. Would you like to make a patch?

@Frodox

This comment has been minimized.

Show comment
Hide comment
@Frodox

Frodox Jun 14, 2017

Member

Where exactly..? I do not see anything in master/buildbot/changes/base.py
but see

$ grep -nRi 'ascii2unicode' .|wc -l
85

in a lot of other places

Member

Frodox commented Jun 14, 2017

Where exactly..? I do not see anything in master/buildbot/changes/base.py
but see

$ grep -nRi 'ascii2unicode' .|wc -l
85

in a lot of other places

@tardyp

This comment has been minimized.

Show comment
Hide comment
@tardyp

tardyp Jun 14, 2017

Member

well, you can start with the place in the stack trace. maybe you will get more

          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/buildbot/master.py", line 485, in addChange
            kwargs[k] = ascii2unicode(kwargs[k])
Member

tardyp commented Jun 14, 2017

well, you can start with the place in the stack trace. maybe you will get more

          File "/home/builder/buildbot/sandbox/lib/python2.7/site-packages/buildbot/master.py", line 485, in addChange
            kwargs[k] = ascii2unicode(kwargs[k])

Frodox added a commit to Frodox/buildbot that referenced this issue Jun 14, 2017

@Frodox Frodox referenced this issue Jun 14, 2017

Merged

Fix non ascii payload handling in base web hook #3325

1 of 1 task complete

rodrigc added a commit to Frodox/buildbot that referenced this issue Jun 28, 2017

@tardyp tardyp closed this in #3325 Jun 28, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment