Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a unicode-related crash in output buffering (-B) on Python 2.6 #130

Closed
wants to merge 1 commit into from
Closed

Fix a unicode-related crash in output buffering (-B) on Python 2.6 #130

wants to merge 1 commit into from

Conversation

gergelyerdelyi
Copy link

This pull request is likely a fix for #116

Python 2.6 does not honour the encoding attribute of sys.std{out|err},
when calling .write(), causing it to fail with a UnicodeEncodeError
if a unicode object containing non-ASCII characters is passed in.

See http://bugs.python.org/issue4947 for more details.

This patch works around the problem by always encoding according to
.encoding, optionally falling back to UTF-8 if it is None.

Note: this problem can be easily reproduced by running the following
test snippet with: nose2 -B unicode_test

#!/usr/bin/python
# -*- coding: utf-8 -*-
def test():
    print u"árvíztűrő tükörfúrógép"
    assert False
Traceback (most recent call last):
  File "/usr/bin/nose2", line 9, in <module>
    load_entry_point('nose2==0.4.6', 'console_scripts', 'nose2')()
  File "/usr/lib/pymodules/python2.6/nose2/main.py", line 281, in discover
    return main(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/nose2/main.py", line 97, in __init__
    super(PluggableTestProgram, self).__init__(**kw)
  File "/usr/lib/pymodules/python2.6/unittest2/main.py", line 98, in __init__
    self.runTests()
  File "/usr/lib/pymodules/python2.6/nose2/main.py", line 257, in runTests
    self.result = runner.run(self.test)
  File "/usr/lib/pymodules/python2.6/nose2/runner.py", line 57, in run
    self.session.hooks.afterTestRun(event)
  File "/usr/lib/pymodules/python2.6/nose2/events.py", line 214, in __call__
    result = getattr(plugin, self.method)(event)
  File "/usr/lib/pymodules/python2.6/nose2/plugins/result.py", line 101, in afterTestRun
    self._reportSummary(event)
  File "/usr/lib/pymodules/python2.6/nose2/plugins/result.py", line 156, in _reportSummary
    self._printErrorList('FAIL', failures, evt.stream)
  File "/usr/lib/pymodules/python2.6/nose2/plugins/result.py", line 171, in _printErrorList
    stream.writeln("%s" % err)
  File "/usr/lib/pymodules/python2.6/nose2/util.py", line 281, in writeln
    self.stream.write(arg)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 197: ordinal not in range(128)

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.15%) when pulling 74d3eac on gergelyerdelyi:buffering-unicode-fix into 7d233a7 on nose-devs:master.

@thedrow
Copy link
Member

thedrow commented Oct 19, 2013

Your workaround fails for Python 3.x.
Also, could you please add the test case you provided and turn it into a functional test case? You can see examples in the tests/functional directory.

@coveralls
Copy link

Coverage Status

Coverage decreased (-2.04%) when pulling f0679f8 on gergelyerdelyi:buffering-unicode-fix into 7d233a7 on nose-devs:master.

Python 2.6 does not honour the encoding attribute of sys.std{out|err},
when calling .write(), causing it to fail with a UnicodeEncodeError
if a unicode object containing non-ASCII characters is passed in.

See http://bugs.python.org/issue4947 for more details.

This patch works around the problem by always encoding according to
.encoding, optionally falling back to UTF-8 if it is None.
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.05%) when pulling 6477d93 on gergelyerdelyi:buffering-unicode-fix into 7d233a7 on nose-devs:master.

@gergelyerdelyi
Copy link
Author

@thedrow: sorry, I did not realize at first that the same source is supposed to run on 3.x as well. Updated the patch with an explicit limit on Python version. The problem only affects cPython <= 2.6, and none of the newer versions, nor pypy, which also reports 2.7.x nowadays.

As for the test case, since the problem triggers only in a failing case and only if the result is actually printed to the real stdout/err, I have not been able to devise a meaningful test that would not pollute the test output screen for all runs. Since the problem is specific to cPython 2.6 anyhow, I am not sure how far is it worth to go with this.

@thedrow
Copy link
Member

thedrow commented Oct 20, 2013

@gergelyerdelyi Give me 2 to 3 days and I'll show you what exactly should be done.

@thedrow
Copy link
Member

thedrow commented Nov 11, 2013

I'll add the test case.

@thedrow
Copy link
Member

thedrow commented Feb 19, 2014

Superseded by #141 and possibly #150. Thank you for your contribution though.

@thedrow thedrow closed this Feb 19, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants