Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

% string formatting with multiple inputs via of tuples doesn't match python3.5 behavior #3092

Open
dannyjeck opened this issue Aug 22, 2019 · 2 comments

Comments

@dannyjeck
Copy link

commented Aug 22, 2019

I'm new to cython but this seems like incorrect behavior.

Python code put in cytest/test.py:

def test_func_no_tuples(a, b):
  print((a.__class__, b.__class__))
  print((a, b))
  s = 'string being formatted=%d' % a
  print(s)


def test_func_with_tuple(a,b):
  print((a.__class__, b.__class__))
  print((a, b))
  s = 'string being formatted=%d-%d' % (a,b)
  print(s)

test_func_no_tuples(50.,50.)

test_func_with_tuple(50.,50.)

Before compilation the output of python3 -c "from cytest import test" is:

python3 -c "from cytest import test"
(<class 'float'>, <class 'float'>)
(50.0, 50.0)
string being formatted=50
(<class 'float'>, <class 'float'>)
(50.0, 50.0)
string being formatted=50-50

After a pip install of cython==0.29.13 I compile with:

PYTHONLIB=/usr/include/python3.5
CFLAGS="-shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I${PYTHONLIB}"
cython --no-docstrings -3 cytest/test.py -o cytest/test.c
gcc $CFLAGS -o cytest/test.so cytest/test.c

Now the output is:

(<class 'float'>, <class 'float'>)
(50.0, 50.0)
string being formatted=50
(<class 'float'>, <class 'float'>)
(50.0, 50.0)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cytest/test.py", line 16, in init cytest.test
    test_func_with_tuple(50.,50.)
  File "cytest/test.py", line 11, in cytest.test.test_func_with_tuple
    s = 'string being formatted=%d-%d' % (a,b)
ValueError: Unknown format code 'd' for object of type 'float'

This is the error I would get if did 1.2.__format__('d'), but it appears that in the other case cython isn't using that function.

The relevant c code output by cython is:

  /* "cytest/test.py":4
 *   print((a.__class__, b.__class__))
 *   print((a, b))
 *   s = 'string being formatted=%d' % a             # <<<<<<<<<<<<<<
 *   print(s)
 * 
 */
  __pyx_t_3 = __Pyx_PyUnicode_FormatSafe(__pyx_kp_u_string_being_formatted_d, __pyx_v_a); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 4, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __pyx_v_s = ((PyObject*)__pyx_t_3);
  __pyx_t_3 = 0;

  /* "cytest/test.py":11
 *   print((a.__class__, b.__class__))
 *   print((a, b))
 *   s = 'string being formatted=%d-%d' % (a,b)             # <<<<<<<<<<<<<<
 *   print(s)
 * 
 */
  __pyx_t_3 = PyTuple_New(4); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 11, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_3);
  __pyx_t_4 = 0;
  __pyx_t_5 = 127;
  __Pyx_INCREF(__pyx_kp_u_string_being_formatted);
  __pyx_t_4 += 23;
  __Pyx_GIVEREF(__pyx_kp_u_string_being_formatted);
  PyTuple_SET_ITEM(__pyx_t_3, 0, __pyx_kp_u_string_being_formatted);
  __pyx_t_2 = __Pyx_PyObject_Format(__pyx_v_a, __pyx_n_u_d); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 11, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_2);
  __pyx_t_5 = (__Pyx_PyUnicode_MAX_CHAR_VALUE(__pyx_t_2) > __pyx_t_5) ? __Pyx_PyUnicode_MAX_CHAR_VALUE(__pyx_t_2) : __pyx_t_5;
  __pyx_t_4 += __Pyx_PyUnicode_GET_LENGTH(__pyx_t_2);
  __Pyx_GIVEREF(__pyx_t_2);
  PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_t_2);
  __pyx_t_2 = 0;
  __Pyx_INCREF(__pyx_kp_u_);
  __pyx_t_4 += 1;
  __Pyx_GIVEREF(__pyx_kp_u_);
  PyTuple_SET_ITEM(__pyx_t_3, 2, __pyx_kp_u_);
  __pyx_t_2 = __Pyx_PyObject_Format(__pyx_v_b, __pyx_n_u_d); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 11, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_2);
  __pyx_t_5 = (__Pyx_PyUnicode_MAX_CHAR_VALUE(__pyx_t_2) > __pyx_t_5) ? __Pyx_PyUnicode_MAX_CHAR_VALUE(__pyx_t_2) : __pyx_t_5;
  __pyx_t_4 += __Pyx_PyUnicode_GET_LENGTH(__pyx_t_2);
  __Pyx_GIVEREF(__pyx_t_2);
  PyTuple_SET_ITEM(__pyx_t_3, 3, __pyx_t_2);
  __pyx_t_2 = 0;
  __pyx_t_2 = __Pyx_PyUnicode_Join(__pyx_t_3, 4, __pyx_t_4, __pyx_t_5); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 11, __pyx_L1_error)
  __Pyx_GOTREF(__pyx_t_2);
  __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  __pyx_v_s = ((PyObject*)__pyx_t_2);
  __pyx_t_2 = 0;

It looks like the __Pyx_PyUnicode_FormatSafe function is being used only in the first case and not in the second, which may or may not be relevant. I am using Python 3.5.2.

If there is any other information needed let me know

@dannyjeck

This comment has been minimized.

Copy link
Author

commented Aug 22, 2019

I should also mention I'm compiling on ubuntu 16.04

@scoder

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2019

Looks like this is a difference between %-formatting and format()/f-strings. Cython optimises some %-formatting cases into f-strings, that's why this occurs.

I think the right fix would be to generate a bit of additional code that checks if the format value is an integer, and if not, passes it through PyNumber_Int(). Tests will show if the behaviour is the same in Py2 and Py3, and if both need this adaptation.

@cython cython deleted a comment from BoBoThein Sep 9, 2019

@cython cython deleted a comment from DirectAcces Sep 10, 2019

@cython cython deleted a comment from DirectAcces Sep 10, 2019

@cython cython deleted a comment from nickwilson330 Sep 10, 2019

@cython cython deleted a comment from blankraza Sep 10, 2019

@cython cython deleted a comment from wangjing123-wj Sep 10, 2019

@cython cython deleted a comment from omsunman Sep 10, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.