Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overflow error in doctests #1911

Closed
toddrme2178 opened this issue Oct 4, 2017 · 7 comments · Fixed by #3634
Closed

Overflow error in doctests #1911

toddrme2178 opened this issue Oct 4, 2017 · 7 comments · Fixed by #3634

Comments

@toddrme2178
Copy link
Contributor

I am trying to build Cython 0.27 and 0.27.1 for openSUSE and I am getting an OverFlow error in the docstests when I run python3 runtests.py -vv (but not for python2) under an x86_64 architecture. This works fine up to Cython 0.26.1, but fails for 0.27 and 0.27.1 in the same way. We are using python 3.6.2 and 2.7.13 and gcc 7.2.1.

======================================================================
FAIL: int_abs (builtin_abs)
Doctest: builtin_abs.int_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.int_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in int_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.int_abs
Failed example:
    int_abs(-max_int-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -2147483648


======================================================================
FAIL: long_abs (builtin_abs)
Doctest: builtin_abs.long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_abs
Failed example:
    long_abs(-max_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808


======================================================================
FAIL: long_long_abs (builtin_abs)
Doctest: builtin_abs.long_long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/c/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_long_abs
Failed example:
    long_long_abs(-max_long_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808


======================================================================
FAIL: int_abs (builtin_abs)
Doctest: builtin_abs.int_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.int_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in int_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.int_abs
Failed example:
    int_abs(-max_int-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -2147483648


======================================================================
FAIL: long_abs (builtin_abs)
Doctest: builtin_abs.long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_abs
Failed example:
    long_abs(-max_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808


======================================================================
FAIL: long_long_abs (builtin_abs)
Doctest: builtin_abs.long_long_abs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for builtin_abs.long_long_abs
  File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line unknown line number, in long_long_abs

----------------------------------------------------------------------
File "/home/abuild/rpmbuild/BUILD/Cython-0.27.1/TEST_TMP/run/cpp/builtin_abs/builtin_abs.cpython-36m-x86_64-linux-gnu.so", line ?, in builtin_abs.long_long_abs
Failed example:
    long_long_abs(-max_long_long-1)     #doctest: +ELLIPSIS
Expected:
    Traceback (most recent call last):
        ...
    OverflowError: ...
Got:
    -9223372036854775808
@robertwb
Copy link
Contributor

I was unable to reproduce this on sles-11-sp4-v20170621 with Cython 0.27.1, Python 3.6.2, and gcc gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux).

@robertwb
Copy link
Contributor

I also tried SUSE Linux Enterprise Server 12 SP3, can't get it to fail.

@kiwifb
Copy link

kiwifb commented Oct 15, 2017

Also reported in Gentoo https://bugs.gentoo.org/632519 - I can reproduce it with gcc-6.4.0. I am guessing that is because gcc now default to C99 (I think) which wouldn't happen with the compilers you tried.

@kiwifb
Copy link

kiwifb commented Mar 14, 2018

I don't have this anymore with 0.28. I have another strange issue though that may or may not be related to this.

@mgorny
Copy link
Contributor

mgorny commented Apr 29, 2018

The problem seems to be caused by -fstrict-overflow (implied by -O2). Evidence:

CFLAGS='-O1' ./runtests.py  builtin_abs  # -> pass
CFLAGS='-O2' ./runtests.py  builtin_abs  # -> fail
CFLAGS='-O2 -fno-strict-overflow' ./runtests.py  builtin_abs  # -> pass
CFLAGS='-O1 -fstrict-overflow' ./runtests.py  builtin_abs  # -> fail

This is with gcc-6.4.0. I don't have older versions of gcc to test.

mgorny added a commit to mgorny/gentoo that referenced this issue Apr 29, 2018
Pass -fno-strict-overflow to work around test failures.

Bug: cython/cython#1911
@gsnedders
Copy link
Contributor

This is still occurring (gcc 9.3.0, python2-config --cflags equal to -I/usr/include/python2.7 -I/usr/include/python2.7 -fno-strict-aliasing -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -DNDEBUG -march=x86-64 -mtune=generic -O2 -pipe -fno-plt; I think the O3 is the only relevant part here!).

The code we're producing in all these cases is:

__pyx_t_1 = abs(__pyx_v_a);
if (unlikely(__pyx_t_1 < 0)) {
  PyErr_SetString(PyExc_OverflowError, "value too large");
  __PYX_ERR(0, 60, __pyx_L1_error);
}

Calling abs with a value that cannot be converted to an absolute value is undefined behaviour, . it's return value is guaranteed to be non-negative for all defined cases, so resultantly gcc removes the branch as its dead code under C semantics. -fno-strict-overflow or -fwrapv stop this under gcc, but clang appears to always elide the conditional statement (which is probably a bug!).

@scoder
Copy link
Contributor

scoder commented Apr 25, 2020

Yes, that looks wrong. I think we have to test the input argument upfront, not the result.

The check is generated in ExprNodes.py, SimpleCallNode.generate_result_code(). I think it should use a dedicated expression node instead, to be inserted into the tree in analyse_c_function_call(). The fact that the safety check is currently implemented directly in the code generation of SimpleCallNode is rather a hack.

serge-sans-paille added a commit to serge-sans-paille/cython that referenced this issue May 25, 2020
serge-sans-paille added a commit to serge-sans-paille/cython that referenced this issue May 26, 2020
serge-sans-paille added a commit to serge-sans-paille/cython that referenced this issue May 26, 2020
serge-sans-paille added a commit to serge-sans-paille/cython that referenced this issue May 26, 2020
serge-sans-paille added a commit to serge-sans-paille/cython that referenced this issue May 26, 2020
@scoder scoder added this to the 0.29.19 milestone May 26, 2020
@scoder scoder modified the milestones: 0.29.19, 0.29.20 May 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants