Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 2.7 and "from __future__ import unicode_literals" causes UnicodeEncodeError #2276

derterz opened this issue May 18, 2018 · 2 comments


Copy link

derterz commented May 18, 2018

I have some very strange UnicodeEncodeError in my application. I could reduce the problem to the following example:

from __future__ import unicode_literals
deg = u'\u00B0'
'%s%s' % ('15', deg)

When I compile this example with cython>=0.28.0 and run is with Python 2.7.14 I get the following error:

Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:25:58) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import crash
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "crash.pyx", line 3, in init crash
    '%s%s' % ('15', deg)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position 0: ordinal not in range(128)

Using cython==0.27.3 there is no such error and the example works fine.

By looking at the code it seems the "PyObject_Str" statement causes the problem. However I have no idea how to fix the issue except for using cython 0.27.3 for python 2.7.

  /* "crash.pyx":3
 * from __future__ import unicode_literals
 * deg = u'\u00B0'
 * '%s%s' % ('15', deg)             # <<<<<<<<<<<<<<
  __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_deg); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error)
  __pyx_t_2 = __Pyx_PyObject_FormatSimpleAndDecref(PyObject_Str(__pyx_t_1), __pyx_empty_unicode); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
  __pyx_t_1 = __Pyx_PyUnicode_Concat(__pyx_kp_u_15, __pyx_t_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error)
  __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
  __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
Copy link

scoder commented May 18, 2018

Thanks for the report, it's really a Py2 issue.
Fixed here: ac6b858

@scoder scoder closed this as completed May 18, 2018
Copy link

derterz commented May 18, 2018

I recompiled my application with the latest cython version and I can confirm the problem is gone.
Thanks a lot for fixing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet

No branches or pull requests

2 participants