Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] cython -2 optimizes repeated strings "b'inappropriately'" #3951

Alphadelta14 opened this issue Dec 21, 2020 · 0 comments

[BUG] cython -2 optimizes repeated strings "b'inappropriately'" #3951

Alphadelta14 opened this issue Dec 21, 2020 · 0 comments


Copy link

Describe the bug
When building against Python 2 (I know, we are running into this because we migrated all of our build tools to Python 3) using Cython in Python 3 (cython -2), some string optimizations like "-"*5 get expanded to "b'-----'" instead of the expected "-----".

To Reproduce
Files to reproduce:

from __future__ import print_function
pip2 install Cython
cython -2 -o expand_cy2_3.0a6_2_good.c
gcc -shared -fPIC "-I$(python2 -c "import sysconfig; print(sysconfig.get_config_var('INCLUDEPY'))")" -o expand_cy2_3.0a6_2_good.c
python2 -c "import expand"
# Out: -----

pip3 install Cython
cython -2 -o expand_cy3_3.0a6_2_bad.c

gcc -shared -fPIC "-I$(python2 -c "import sysconfig; print(sysconfig.get_config_var('INCLUDEPY'))")" -o 
python2 -c "import expand"
# Out: b'-----'

Expected behavior
As outlined above, expected behavior is that this is still compiled to "-----"

Environment (please complete the following information):

  • OS: Linux
  • Python version: Python 3 compiler, Level 2 cython with Python 2 interpreter
  • Cython version: 0.29.21 (and 0.28.3 and 3.0a6)

Additional context
We're in the process of getting rid of all of the Python 2 stuff... it's a bit sticky if our build tools can't be migrated (our python 2 cython was supporting both python 2 and 3 before)

Alphadelta14 added a commit to Alphadelta14/cython that referenced this issue Dec 22, 2020
Ensure that when StrNode is a BytesLiteral, that we don't coerce it to unicode

Fixes cython#3951
@scoder scoder added this to the 0.29.22 milestone Jan 26, 2021
@scoder scoder removed this from the 0.29.22 milestone Feb 19, 2021
@scoder scoder closed this as completed in cd6e29d Apr 12, 2021
@scoder scoder added this to the 0.29.23 milestone Apr 14, 2021
scoder pushed a commit that referenced this issue Apr 14, 2021
* Handle constant folding for LanguageLevel 2 on Python 3. Ensure that when StrNode is a BytesLiteral, that we don't coerce it to unicode.
* Add test for string multiplication bug. Needed to change the TreePath slightly to allow bytes-to-str comparison.

Fixes #3951
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet

Successfully merging a pull request may close this issue.

2 participants