Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: silence Cython warnings about changes dtype/ufunc size. #432

Merged
merged 1 commit into from Sep 11, 2012

Conversation

rgommers
Copy link
Member

@rgommers rgommers commented Sep 8, 2012

These warnings are visible whenever you import scipy (or another package) that
was compiled against an older numpy than is installed. For example compiled
against 1.5.1, like current scipy binaries are, and used with 1.7.0.

These warnings aren't useful; if numpy would really break its ABI it would be
noticed in no time without these warnings.

These warnings are visible whenever you import scipy (or another package) that
was compiled against an older numpy than is installed.  For example compiled
against 1.5.1, like current scipy binaries are, and used with 1.7.0.

These warnings aren't useful; if numpy would really break its ABI it would be
noticed in no time without these warnings.
@rgommers
Copy link
Member Author

rgommers commented Sep 8, 2012

Should be backported to 1.7.x too I think. Note that I proposed this on the list already (without feedback): http://article.gmane.org/gmane.comp.python.numeric.general/51323

@njsmith
Copy link
Member

njsmith commented Sep 8, 2012

...wouldn't it make more sense to just remove the code in cython? It just seems wrong to have two chunks of code in Cython and in Numpy that have no purpose except to cancel each other out.

@rgommers
Copy link
Member Author

rgommers commented Sep 8, 2012

This size check is deemed to be useful by the Cython guys in general; it's just that for numpy it is quite unhelpful.

The other option would be to special-case numpy, as discussed at http://grokbase.com/t/python/cython-devel/121ygkvfhc/cython-upcoming-issues-with-numpy-deprecated-apis-and-cythons-sizeof-checks. This hasn't been done yet I think, and we've seen these warnings now for several years.

@njsmith
Copy link
Member

njsmith commented Sep 8, 2012

Ah, I see. I sent a note to cython-dev in case they want to speak up, but it makes more sense to me now.

Is there a reason you don't filter warnings for ndarray?

@rgommers
Copy link
Member Author

rgommers commented Sep 8, 2012

I'm not seeing any for ndarray. But it can't hurt to filter for those too.

@kisielk
Copy link

kisielk commented Sep 9, 2012

Wouldn't the correct solution in a case like this be to rebuild scipy so that it's built against the correct numpy version? Not sure why you'd want to silence such warnings.

@rgommers
Copy link
Member Author

rgommers commented Sep 9, 2012

That's not a feasible solution, and certainly shouldn't be necessary. Scipy (as many other packages) is compatible with a number of versions of numpy. So when we distribute scipy binaries, we build them against the lowest supported numpy version (1.5.1 as of now) and they work with 1.6.x, 1.7.x and numpy master as well.

The real correct would be for Cython only to issue warnings when the size of dtypes/ufuncs has changes in a way that breaks the ABI, and be silent otherwise.

@njsmith
Copy link
Member

njsmith commented Sep 11, 2012

Cython folks seem to be in favor, so let's do this.

njsmith added a commit that referenced this pull request Sep 11, 2012
MAINT: silence Cython warnings about changes dtype/ufunc size.
@njsmith njsmith merged commit a17875b into numpy:master Sep 11, 2012
@rgommers
Copy link
Member Author

rgommers commented Sep 11, 2012

Thanks for merging. For reference the Cython thread: http://thread.gmane.org/gmane.comp.python.cython.devel/14352/focus=14354

@rgommers rgommers deleted the silence-cython-warnings branch Apr 4, 2015
rgommers added a commit to rgommers/numpy that referenced this pull request Apr 5, 2015
…n noise.

See numpygh-432 for details.  Motivation for adding this now is the discussion
on numpygh-5343.
@c-Monster
Copy link

c-Monster commented Jul 26, 2018

+1

@vfr292
Copy link

vfr292 commented Sep 9, 2020

Hi,

I just installed Numpy 1.19.1 and scikit-learn-0.21.3 (on a mac) from Pip and I'm getting this warning when I import a RandomForestClassifier from scikit. I thought this warning would have been silenced?

I thought I would point out that this seems to be popping up. Maybe the Numpy in Scikit pre-dates the silencing?

This is a little over my head programming-wise, but I'm happy to provide more information if that helps. Thank you for maintaining a great library! :)

@seberg
Copy link
Member

seberg commented Sep 9, 2020

@vfr292 yes, they should be, I think. Could you paste the exact warning message? I wonder if the text changed and that causes the warning not to be filtered. Unless filtering in NumPy is not enough and it actually has to be filtered in the model using NumPy?

@vfr292
Copy link

vfr292 commented Sep 9, 2020

Hi @seberg, here is the exact message:

numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject

Below is the full error message if that helps.

---------------------------------------------------------------------------
RuntimeWarning                            Traceback (most recent call last)
<ipython-input-19-62f767cfc1ca> in <module>()
      1 print("Current Time: {}".format(datetime.now()))
----> 2 from sklearn.ensemble import RandomForestClassifier
      3 
      4 # # per https://stackoverflow.com/questions/40845304/runtimewarning-numpy-dtype-size-changed-may-indicate-binary-incompatibility
      5 # warnings.filterwarnings("ignore", message="numpy.dtype size changed")

~/Documents/Python/Car_Listing_Analysis/venv_car_listing_analysis/lib/python3.6/site-packages/sklearn/__init__.py in <module>()
     78     from . import _distributor_init  # noqa: F401
     79     from . import __check_build  # noqa: F401
---> 80     from .base import clone
     81     from .utils._show_versions import show_versions
     82 

~/Documents/Python/Car_Listing_Analysis/venv_car_listing_analysis/lib/python3.6/site-packages/sklearn/base.py in <module>()
     19 from . import __version__
     20 from ._config import get_config
---> 21 from .utils import _IS_32BIT
     22 from .utils.validation import check_X_y
     23 from .utils.validation import check_array

~/Documents/Python/Car_Listing_Analysis/venv_car_listing_analysis/lib/python3.6/site-packages/sklearn/utils/__init__.py in <module>()
     20 from scipy.sparse import issparse
     21 
---> 22 from .murmurhash import murmurhash3_32
     23 from .class_weight import compute_class_weight, compute_sample_weight
     24 from . import _joblib

sklearn/utils/murmurhash.pyx in init sklearn.utils.murmurhash()

RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject`

@seberg
Copy link
Member

seberg commented Sep 9, 2020

Hmmm, the numpy __init__.py still includes the warning filters, so I am not sure why they are not be filtered here.

I suppose sklearn could probably disable the warning entirely in their Cython code (with the knowledge that we may extend structures, and hopefully that can't be a problem for them).
I am now wondering whether the actual reason is that the numpy.pxd file that Cython compiled against misses the check-size ignore option in: ctypedef class numpy.ufunc [object PyUFuncObject, check_size ignore].

@mattip
Copy link
Member

mattip commented Sep 9, 2020

Which python are you using? The wheel building repo of scikit-learn is pinned to old versions of numpy on 3.6, 3.7, and I don't see what version of cython they use. Maybe @glemaitre could lend some insights? The wheels should be built with at least cython 0.29 to pick up the check-size option.

@glemaitre
Copy link

glemaitre commented Sep 9, 2020

We have a minimum of 0.28.5 but I can see that 0.29.14 was used to build the wheels of the last release.
I will check a bit more because it does seem the last available release but I don't think that we pin it.
FYI it was for scikit-learn 0.23.2 and I have the impression that the warning is raised in an older version (0.21)?
(Sorry I try to catch up with the issue).

@glemaitre
Copy link

glemaitre commented Sep 9, 2020

@vfr292
Copy link

vfr292 commented Sep 9, 2020

HI @mattip, below is my python version:

3.6.8 (v3.6.8:3c6b436a57, Dec 24 2018, 02:04:31)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]

Also, I was mistaken, I just installed scikit-learn-0.23.2, I uninstalled scikit-learn-0.21.3 to install the new version. Sorry or the mixup :(

This is all a little over my head, is there something you would like me to do? Uninstall or reinstall anything?

Thank you all for looking into this! :). I really appreciate it!

@mattip
Copy link
Member

mattip commented Sep 9, 2020

Thanks @glemaitre. But it seems it is out of scikit-learn's hands.

It turns out even the 3.0 alpha release does not have the check_size ignore directive. master does. Maybe worth pinging cython to see if they can include it in 3.0 to prevent this in the future.

In numpy 1.19 we began shipping our own __init__.pxd with the directive, but that won't help here.

The change that causes the warning happened in the release of NumPy 1.16. So it seems the best option for @vfr292 is the one @seberg started with: a filter for that warning.

@vfr292
Copy link

vfr292 commented Sep 9, 2020

Thank you @mattip. I can filter the warning. For anyone else who reads this, the snipped below worked for me:

warnings.filterwarnings("ignore", message="numpy.dtype size changed")
warnings.filterwarnings("ignore", message="numpy.ufunc size changed")

@seberg
Copy link
Member

seberg commented Sep 9, 2020

Oh, I thought cython had those ignores, and it does, but I did not realize it happened so recently!

I am still a bit unsure why the warning filters set up in the NumPy __init__.py do not kick in though. Although, I guess filters being global there may be more things going on here.

scoder added a commit to cython/cython that referenced this pull request Sep 13, 2020
…e the useless size warnings about them.

See numpy/numpy#432 (comment)
Also remove the useless "extern" modifiers from cdef classes declared in "extern" sections.
@scoder
Copy link
Contributor

scoder commented Sep 13, 2020

It turns out even the 3.0 alpha release does not have the check_size ignore directive.

@mattip I've added them to Cython's 0.29.x branch. The next maintenance release will have them.

@mattip
Copy link
Member

mattip commented Sep 13, 2020

@scoder thanks!

sairam4123 added a commit to sairam4123/cython that referenced this pull request Nov 4, 2020
* Fix test in Py2.

* Check for exceptions also when @returns() is used, not only for "->" return type annotations. (cythonGH-3664)

When you use Python type annotations, it would be weird if you lost Python exception propagation semantics along the way, just by compiling the code. So the default behaviour is "except? -1" here for C integer types.

Arguably, this would also be a better default for the decorator case.

See cython#3625 (comment)

* Add tests that exception propagation works with the "@returns()" decorator.

* Update changelog.

* Allow selecting specific doctests in test modules with "-k pattern", instead of always running all tests.

* Correct the positions reported for errors in f-strings.
Closes cython#3674

* Fix f-string error positions in nogil test.

* Clarify backwards incompatible special method change in changelog.

* Fix many indentation and whitespace issues throughout the code base (cythonGH-3673)

… and enforce them with pycodestyle.

* Allow selecting specific doctests in test modules with "-k pattern", instead of always running all tests.

* Prepare release of 0.29.20.

* Update changelog.

* Avoid calling PyUnicode_FromUnicode() in Py3.
Closes cython#3677

* Update changelog.

* Minor code simplification.

* Always bind Cython functions

* Update changelog.

* Avoid an "unused variable" warning for code that gets compiled out in Py3.

* Minor code simplification.

* Revert "Always bind Cython functions"

This reverts commit 6677326.

* Update changelog.

* Revert "Limited API updates and cleanup for cython#2056. cythonGH-3635)"

This reverts commit 02bb311.

* Revert "Invoke binop super method via direct slot access."

This reverts commit bcb9387.

* Revert "Add support for pow operator."

This reverts commit d849fb2.

* Revert "Python-style binary operation methods."

This reverts commit e6a8124.

* Re-add "c_api_binop_methods" directive for backwards compatibility after reverting cython#3633 and force it to "False".
Closes cython#3688.

* Change the default of the "c_api_binop_methods" directive to False. (cythonGH-3644)

This is a backwards incompatible change that enables Python semantics for special methods by default, while the directive allows users to opt out of it and go back to the previous C-ish Cython semantics.

See cython#2056

* Always bind Cython functions in Py3 (cythonGH-3683)

Rebased 1bb26b9 for cython 0.29.x,
and made conditional to Python 3.  This does not solve the original staticmethod problem
in Python 2 but it does resolve classmethod problems in Python 3.9.  Therefore, it unbreaks
other packages on Python 3.9, improves overall state for Python 3 and does not introduce
regressions in Python 2.

* Limit the scope of a local variable in a helper function.

* Avoid a C compiler warning about a constant condition.

* Remove dead code.

* Update changelog.

* Add support for C++ scoped enums with "enum class" and "enum struct" (cythonGH-3640)

Closes cython#1603.

* Add missing "PyUnicode_GET_LENGTH" to unicode.pxd (cythonGH-3692)

* Add missing "PyUnicode_GET_LENGTH" to unicode.pxd (cythonGH-3692)

* Fix usage of deprecated Py_UNICODE API.

* Add safety fix to avoid reading a character from the empty string.

* Fix prefix of internal function name.

* Fix definition of "__Pyx_CyFunction_USED", which should only be #defined and not have a value.

* Avoid using "tp_name" when CYTHON_COMPILING_IN_LIMITED_API (cythonGH-3693)

* exec() did not allow recent Python syntax features in Py3.8+ due to https://bugs.python.org/issue35975
Closes cython#3695

* Avoid using the "tp_iternext" slot when CYTHON_USE_TYPE_SLOTS is disabled.

* Give the "__Pyx_PyObject_GetIterNext" helper macro a more explanatory name.

* Avoid unused variable in PyPy etc.

* Avoid a call to PyTuple_GET_ITEM() to get the item array pointer if CYTHON_ASSUME_SAFE_MACROS is disabled.
See cython#3701

* Enable travis for all branches.

* Fix indentation counter for module init function.

* Fix FunctionState handling for module cleanup function.

* Validate that all temps were correctly released at the end of a function.

* Keep reference to module dict around also in limited mode (cythonGH-3707)

`PyModule_GetDict` is part of the limited API so we can keep a reference
to the module dict around regardless of limited mode being enabled or not.

* Validate and fix temp releasing (cythonGH-3708)

* Fix a temp leak in the type init code.
* Fix temp leaks in fused types initialisation code.
* Correctly release the buffer index temps allocated for index calculations.
* Make tests fails hard if a temp variable is not released at the end of a generated function.
* Fix temp leak in switch statement code.
* Make end-to-end tests fail on refnanny output.
* Fix result temp leak in PyTypeTestNode.
* Fix result temp leak in external type/function import code and enable the refnanny check for them.
* Fix temp leak when try-return-finally is used in generators.
* Make it explicit when an allocated temp is not meant to be reused.
* Fix temp leak when assigning to the real/imag attributes of complex numbers.
* Fix temp leak when assigning to a memoryview slice.
* Clean up "num_threads" result temp in parallel section, not only in prange loop.
* Fix temp leak in Pythran buffer setitem code.
* Simplify NumPyMethodCallNode since it does not need the Python function anymore. Previously, it generated code that needlessly looked up the Python function without actually using it.
* Fix temp leak when deleting C++ objects.
* Add a test that actually reusing temps when deleting C++ objects works correctly.

* Test runner: disable keep-alive output in --debug mode to keep a potential pdb console clean.

* Fix argument name usage in finally blocks (cythonGH-3713)

Fixes cython#3712 (hopefully) by reverting a small part of bbef4d7

* Fix argument name usage in finally blocks (cythonGH-3713)

Fixes cython#3712 (hopefully) by reverting a small part of bbef4d7

* Document version-tagged pxd files (cythonGH-3587)

* Implement generic optimized loop iterator with indexing and type inference for memoryviews (cythonGH-3617)

* Adds bytearray iteration since that was not previously optimised (because it allows changing length during iteration).
* Always set `entry.init` for memoryviewslice.

* Update changelog.

* Make end-to-end tests fail on refnanny output.

* Fix FunctionState handling for module cleanup function.

* Update change log.

* Really only use PyUnicode_FromUnicode() when needed (cythonGH-3697)

* Really only use PyUnicode_FromUnicode() for strings that contain lone surrogate, not for normal non-BMP strings and not for surrogate pairs on 16bit Unicode platforms.

See cython#3678

* Extend buildenv test to debug a MacOS problem.
* Add a test for surrogate pairs in Unicode strings.
* Limit PyUnicode_FromUnicode() usage to strings containing lone surrogates.
* Accept ambiguity of surrogate pairs in Unicode string literals when generated on 16bit Py2 systems.

* Disable testing against NumPy 1.19+ in the 0.29.x branch, which removed C-API features.

* Disable testing against NumPy 1.19.0 in the 0.29.x branch, which breaks a C-API call.

* Validate and fix temp releasing (cythonGH-3708) (cythonGH-3717)

* Validate and fix temp releasing (cythonGH-3708)
Backports 92147ba.

    * Fix a temp leak in the type init code.
    * Fix temp leaks in fused types initialisation code.
    * Correctly release the buffer index temps allocated for index calculations.
    * Make tests fails hard if a temp variable is not released at the end of a generated function.
    * Fix temp leak in switch statement code.
    * Make end-to-end tests fail on refnanny output.
    * Fix result temp leak in PyTypeTestNode.
    * Fix result temp leak in external type/function import code and enable the refnanny check for them.
    * Fix temp leak when try-return-finally is used in generators.
    * Make it explicit when an allocated temp is not meant to be reused.
    * Fix temp leak when assigning to the real/imag attributes of complex numbers.
    * Fix temp leak when assigning to a memoryview slice.
    * Clean up "num_threads" result temp in parallel section, not only in prange loop.
    * Fix temp leak in Pythran buffer setitem code.
    * Simplify NumPyMethodCallNode since it does not need the Python function anymore. Previously, it generated code that needlessly looked up the Python function without actually using it.
    * Fix temp leak when deleting C++ objects.
    * Add a test that actually reusing temps when deleting C++ objects works correctly.

* Improve syntax feature support of Cython CodeWriter (cythonGH-3514)

* Disable Py_UNICODE fallback for __Pyx_UnicodeContainsUCS4() in Py3.9 since Py_UNICODE is deprecated and PEP-393 unicode is practically required.

* Avoid calling PyUnicode_FromUnicode() in Py3 (cythonGH-3721)

See cython#3677

* Add missing name substitution in __Pyx_PyUnicode_Substring() utility code.

* Update changelog.

* Prepare release of 0.29.21.

* Really only use PyUnicode_FromUnicode() when needed (cythonGH-3697)

* Really only use PyUnicode_FromUnicode() for strings that contain lone surrogate, not for normal non-BMP strings and not for surrogate pairs on 16bit Unicode platforms.

See cython#3678

* Extend buildenv test to debug a MacOS problem.
* Add a test for surrogate pairs in Unicode strings.
* Limit PyUnicode_FromUnicode() usage to strings containing lone surrogates.
* Accept ambiguity of surrogate pairs in Unicode string literals when generated on 16bit Py2 systems.

* Disable Py_UNICODE fallback for __Pyx_UnicodeContainsUCS4() in Py3.9 since Py_UNICODE is deprecated and PEP-393 unicode is practically required.

* Fix test in 16-bit Unicode deployments.

* Update changelog.

* Always consider 0-sized arrays as C- and F-contiguous (cythonGH-3728)

Fixes cython#2093

* Always consider 0-sized arrays as C- and F-contiguous (cythonGH-3728)

Fixes cython#2093

* Update changelog.

* Add missing unordered_map template defaults (cythonGH-3686)

* Add missing unordered_map template defaults (cythonGH-3686)

* Update changelog.

* Improve test output in case of failures.

* Using Py_UNICODE to store lone surrogates makes Py3 join surrogate pairs on 16-bit Unicode platforms (Windows) when reading them back in, although we correctly processed them before.
Instead, we now use the "unicode_escape" codec to store byte strings, because it can return surrogate characters (which the other codecs cannot).

* Disable test in Py2.6.

* Update changelog.

* Use the more appropriate CYTHON_USE_TYPE_SLOTS guard for accessing the binop number slot.

* Fix typo in error message.

* Update the documentation on the arithmetic special methods and issue a "backwards compatibility" warning when the reversed method is not implemented.

See cython#2056

* Add the new "c_api_binop_methods" directive to the documentation.

* PyPy does not support PyType_GetSlot(). Use type slots instead.

* Reformat doc paragraph.

* Document C inline properties.

* Beautify example output.

* Use inline properties on the "PyComplex" builtin type declared in "cpython.complex" to provide C level access to the "real" and "imag" attributes (which Cython provides anyway for the 'undeclared' builtin type).

* Prevent compile error when the result of repr() is assigned to a "unicode" LHS with language_level=3.
Closes cython#3736

* Handle `for x in cpp_function_call()` (cythonGH-3667)

Fixes cython#3663

This ensures that rvalues here are saved as temps, while keeping the
existing behaviour for `for x in deref(vec)`, where the pointer for vec
is copied, meaning it doesn't crash if vec is reassigned.

The bit of this change liable to have the biggest effect is that I've
changed the result type of dereference(x) and x[0] (where x is a c++
type) to a reference rather than value type. I think this is OK because
it matches what C++ does. If that isn't a sensible change then I can
probably inspect the loop sequence more closely to try to detect this.

* Clarify changelog entry on ways to deal with the incompatible binop method change.

* Remove Google Analytics script from documentation to avoid tracking our users.

* Add donation banner to documentation.

* Create bug template

* Update issue templates

* Delete unused custom ticket template

* Add handshake emoji to donations banner to make it more visible.

* Don't create CReference in C (only c++) (cythonGH-3746)

* Fixed reference types being passed to getitemint (cythonGH-3755)

* Reorder test module to restore the "invalid - valid" order.

* Update changelog.

* Prepare release of 3.0a6.

* Add test comments on how "memslice.pyx" and "memoryview.pyx" relate.

* Print test dependency versions to help with test failure debugging.

* Add CI builds for different CPU architectures on travis.

* Update issue templates

* Add type "Py_hash_t" in pure Python mode.

* Update changelog.

* Clarify changelog entry.

* Remove dead code and dead comments from "numpy/__init__.pxd".

* Change "Py_intptr_t" declaration in numpy.pxd to what CPython uses as fallback. "int" seems overly lazy if it tends to be larger on many systems.

* Readability improvements in "numpy.pxd".

* Add a comment that numpy.pxd is maintained by the NumPy project.

* Try to fix NumPy test failures by not setting the "NPY_NO_DEPRECATED_API" #define for NumPy 1.19.[01].

* Use NumPy 1.18.x for testing on travis as long as 1.19.[01+] ships a numpy.pxd that is incompatible with Cython 3.0.

* Remove useless "extern" modifiers from cdef classes declared in "extern" sections.

* Do not depend on the default type inference in "cpython/array.pxd".

* Avoid unused C variable warning by moving declaration and usage under the same condition.
Closes cython#3763

* Use a generic shape accessor for pythranized array (cythonGH-3762)

This is a follow up to cython#3307

* Silence a C compiler warning about a constant condition.

* Remove an unused function that had been added for the now-deleted "__getbuffer__" implementation.

* Support PEP-560 ("__class_getitem__") for extension classes (cythonGH-3765)

* Add "make checks" target to run code checks.

* In bug template, ask for Python version in addition to Cython version

* Support simple, non-strided views of "cython.array".
Closes cython#3775

* Remove unused cimports.

* Update changelog.

* Set PYTHONHOME in embedding test to fix compilation issues in Py3.8/macOS.

* Fix unrelated test after changing MemoryView.pyx.

* Revert "Set PYTHONHOME in embedding test to fix compilation issues in Py3.8/macOS."

This reverts commit a333d6a.

The change did not resolve the test issue in travis.

* Fix Python 3.4 + MSVC issue with elaborated type specifier for enum (cythonGH-3782)

* Add more cimport_from_pyx tests (cythonGH-3786)

There's now a fairly wide range of valid syntax for declaring things in pyx files and it should all be supported when cimporting from them.

* Split known types into separate lines to let them have their own VCS history.

* Add a note on PayPal fees for small payments.

* Avoid merged-in code picking up directives from main module (cythonGH-3785)

Fixes cython#1071

* Fix cygdb (cythonGH-3542)

* Cython debugger documentation: Added link to an installation script.
* Got a new libpython.py from the cpython source distribution.
* Default language level in tests is now 3 instead of 2
* Migrated codefile from python 2 to python 3.
* Added testcase for the cy list command in cygdb.
* Temporarily removing test case that freezes gdb.
* Fixed a bug that broke several Cygdb tests.

The cython_debug/cython_debug_info_* files map the names of the C-functions generated by the Cython compiler to the names of the functions in the *.pyx source. If the function was defined using "def" (and not "cpdef" or "cdef") in the *.pyx source file, the C-function named in cython_debug/cython_debug_info_* used to be __pyx_pw_*, which is the name of the wrapper function and now it is __pyx_f_*, which is the name of the actual function. This makes some Cygdb tests pass that did not pass before.

* Better error messages: If a cygdb command raises, a traceback will be printed.
* Fixed a bug in cygdb.

The following now works:
1. Start cygdb
2. Type "cy exec" and hit enter
3. Type some other lines
4. Type "end" and hit enter.
-> These "other lines" will get executed

* Fixed a bug in cygdb: cy list now works outside of functions.
* Added print_hr_allmarkers function for easier debugging.
* Fixed a bug that broke cygdb:

cy break did not work if you put the breakpoint outside of a function if there was e.g. the following somewhere in your *.pyx file:
cdef class SomeClass():
    pass

* Added a Cygdb test for printing global variables.
* Fixing cygdb: Replaced cy print with a simple, working solution.
* If an exception in Cygdb occurs, a stacktrace will be printed.
* Fixed a bug that broke cy break -p
* Bugfix: The compiler now writes out correctly which cython linenumber and path corresponds to which c linenumber.
* Set language_level=2 in runtests.py

* Update changelog.

* Add a "gdb" test tag that depends on being able to run gdb.

* Update CPython "test_fstring" copy to Py3.9.

* Improved documentation for annotation typing (cythonGH-3799)

Mainly by moving it to a separate section to make it easier
to find, however also added a small amount of extra information
about some of the obvious limitations.

* Fix `return None` in CodeWriter. (cythonGH-3795)

* Rename "GCC_DIAGNOSTIC" macro to make it Cython specific and make it available to other utility code sections.

* Add a test for unpacking large PyLong values.

* Silence gcc diagnostics whenever -1 is cast to something user provided. (cythonGH-3803)

Fixes cythonGH-2749.

* Define extern `PyBUF_MAX_NDIM` (cythonGH-3811)

Ensure that Cython exposes `PyBUF_MAX_NDIM` from Python as part of
`cpython.buffer` to allow access to developers.

* Change test to use only integer calculations to prevent platform specific rounding issues, while keeping a reasonable distribution of integers across the PyLong digit ranges.

* Created a glossary and added one new entry (cythonGH-3810)

* Call destructors for structs in C++ (cythonGH-3228)

In C++ mode, structs can contain C++ classes. Therefore structs
should have their destructors called to ensure any class
contained is also destructed.

Also, a bit more thorough about ensuring constructor is generated where necessary.

Closes cythonGH-3226.

* Rename "GCC_DIAGNOSTIC" macro to make it Cython specific and make it available to other utility code sections.

* Silence gcc diagnostics whenever -1 is cast to something user provided. (cythonGH-3803)

Fixes cythonGH-2749.

* Allow cast to ctuple (cythonGH-3812)

* Make asyncio.iscoroutinefunction() recognise Cython compiled coroutines. (cythonGH-3427)

Python's asyncio.coroutines uses an object to tag objects as coroutine functions. We now read this object and use it to tag Cython compiled coroutines as well.

It also includes tests to make sure `asyncio.iscoroutinefunction()` works as expected.
This doesn't fix `inspect.iscouroutinefunction()` (which uses a flag that can trigger undesirable behaviour for cython functions).

Closes cython#2273

* Do not cover lines that were excluded in the coveragerc config file (cythonGH-3682)

Closes cython#3680.

* Add doc support for cpdef enum (cythonGH-3813)

* Update changelog.

* Add "check_size ignore" to all NumPy.pxd class declarations to silence the useless size warnings about them.
See numpy/numpy#432 (comment)
Also remove the useless "extern" modifiers from cdef classes declared in "extern" sections.

* Update changelog.

* Update changelog.

* Support utility code in headers (cythonGH-3779)

Specifically this allows public packed structs but may also allow other public declarations that require small amounts of utility code.

* Prevent overflowing hash values for "Machines.Node" due to arbitrarily large unsigned pointer values.
Closes cython#3840

* Extract "error_type" handling from the type creation functions.

* Remember in the AnnotationsWriter when a serialised expression contained unknown/unsupported nodes.

* Do not include "u" string prefix in annotations since tools that process them probably expect Py3 string semantics anyway.

* Keep AnnotationNode instead of the bare annotation expression in "entry.annotation" to get access to the string representation.

* Fix test after removing the 'u' prefix from unicode string annotation values.

* Add support for PEP 526 `__annotations__` in class body. (cythonGH-3829)

Closes cython#2552

* Some more glossary entries (cythonGH-3836)

* Rename test file to avoid ambiguity with the C "const" modifier.

* Join '*' and '**' parsing in declarators to avoid differences for 'const' parsing etc.

* Set language level in Cython's speed-up .pxd files since it no longer depends on the .py files that they correspond to.

* Declare "scanner.sy" as "unicode" string to optimise its usage.

* Use more recent C-API functions on tear-down of the embedding code.

* Improve some wordings in README.rst (cythonGH-3852)

* Restore Py2 compatibility in test.

* Replace useless comment.

* Fixed assorted crashes with fused types (cythonGH-3853)

Show a more detailed error for unspecializable fused return types.

* Add a more common and more versatile example to the Verbatim C-Code documentation.

* Fix RST typo.

* Make C code C89 again.

* Add support for forwarding references (cythonGH-3821)

See, for example, https://isocpp.org/blog/2012/11/universal-references-in-c11-scott-meyers

Closes cython#3814

* Avoid C compiler warnings about unused variables in test.

* Avoid C compiler warning about unused variable in test.

* Test Py3.7 and Py3.8 in C and C++ mode under appveyor.
Run both in the same job since the machines are quite fast with parallel processes, but setting one up is slow.

* Fix test compile failure in MSVC.

* Allow creation of wrappers for cdef functions with memoryviews (cythonGH-3856)

Fixes cython#3843

* Clean up and test type identifier escaping.
- hash() hashing lead to unpredictable random prefixes for long names across multiple runs
- use a single regex run instead of repeated calls to replace()

* Move a memoryview test out of the "run" test directory since memoryview tests tend to be slow and have their own test directory.

* Fix test compile failure in MSVC.

* Clarify the section on exception return values.

* Be a bit more paranoid about macro usage in the refnanny code.

* Looks like the usual macro on Windows is "_WIN32" and not "WIN32". Let's support both, to be on the safe side.

* Try to fix platform specific test once more.
"synchapi.h" is not supposed to be included directly, and "windows.h" can break ... other stuff.
Let's see what we can do.

* Fix ReST typo.

* docs: fix typos, minor clarification.

* Minor docs clarification on error return values. (cythonGH-3859)

* Replace deprecated Element.getiterator() with Element.iter(). (cythonGH-3864)

`xml.etree.ElementTree.Element.getiterator()` was deprecated in Python
2.7 & 3.2 and removed in the freshly released Python 3.9.

* Prefer Element.iter() over the deprecated Element.getiterator(). (cythonGH-3865)

`xml.etree.ElementTree.Element.getiterator()` was deprecated in Python
2.7 & 3.2 and removed in Python 3.9.

* Split a combined assert in two to avoid an unused C temp variable when assertions are disabled. (cythonGH-3870)

* Split a combined assert in two to avoid an unused C temp variable when assertions are disabled. (cythonGH-3870)

* Disable a test that fails in Py2 in 0.29.x since 'str' is 'unicode' in cython3, also in Python 2.

* Rename test file to avoid ambiguity with the C "const" modifier.

* Use more recent C-API functions on tear-down of the embedding code.

* Make type identifier escaping deterministic: hash() hashing lead to unpredictable random prefixes for long names across multiple Python runs.

* Clarify the section on exception return values.

* Fix ReST typo.

* docs: fix typos, minor clarification.

* Minor docs clarification on error return values. (cythonGH-3859)

* Add Python 3.9 to appveyor build.

* Resolve merge conflict.

* Removed `same_as` methods from Plex.Actions.Action (cythonGH-3847)

It seems to be unused and it looked like the existing implementation was faulty.

* Fix test after changing the hash method used for escaping long type descriptions.

* Add warning for common user mistake and fix crash relating to annotated attributes. (cythonGH-3850)

Closes cython#3830

* Improve error reporting when users mistakenly write "&&" or "||" instead of Python's "and" and "or" operators. (cythonGH-3858)

* Disable the embedding test on MacOS-X to get the builds green again. Too many PRs depend on the travis tests to give a green light.

* Simplify the output stream capturing for the C compiler runs by using a temp file instead of threads.

* Help reporters see the first paragraph of the bug report template.

* Detect _MSC_VER for __PYX_STD_MOVE_IF_SUPPORTED to support MSVC support even when "__cplusplus" is not set appropriately (cythonGH-3792)

* Fix memoryview casts involving fused types (cythonGH-3882)

I think this approach is more satisfactory than the old way it used to "work", where "fused_to_specific" was permanently added to the module scope containing the fused type (in this case the Cython scope), was applied in "Scope.lookup_type", but continued to have an effect on the scope forever.

Closes cython#3881

* Add O_DIRECT to posix/fcntl.pxd (cythonGH-3894)

Closes cython#3242

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Jeroen Demeyer <jeroen.k.demeyer@gmail.com>
Co-authored-by: Robert Bradshaw <robertwb@gmail.com>
Co-authored-by: Michał Górny <mgorny@gentoo.org>
Co-authored-by: Ashwin Srinath <3190405+shwina@users.noreply.github.com>
Co-authored-by: Thomas A Caswell <tcaswell@gmail.com>
Co-authored-by: Matthias Braun <matze@braunis.de>
Co-authored-by: da-woods <dw-git@d-woods.co.uk>
Co-authored-by: Tao He <sighingnow@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Clemens <chofreither@gmail.com>
Co-authored-by: will <timwilloney@gmail.com>
Co-authored-by: serge-sans-paille <serge.guelton@telecom-bretagne.eu>
Co-authored-by: Yuan <pyslvs@gmail.com>
Co-authored-by: Volker-Weissmann <39418860+Volker-Weissmann@users.noreply.github.com>
Co-authored-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: cf-natali <cf.natali@gmail.com>
Co-authored-by: jakirkham <jakirkham@gmail.com>
Co-authored-by: Tasha "Ren" Chin <tasha.s.chin@gmail.com>
Co-authored-by: Pedro Marques da Luz <pmdaluz@gmail.com>
Co-authored-by: Mathias Laurin <Mathias.Laurin+github.com@gmail.com>
Co-authored-by: matham <moiein2000@gmail.com>
Co-authored-by: Sean <stewart.d.sean@gmail.com>
Co-authored-by: Rajat Dash <49777622+DARKPROGRAMMER16@users.noreply.github.com>
Co-authored-by: ptype <ptype@users.noreply.github.com>
Co-authored-by: Nick Pope <nick.pope@flightdataservices.com>
Co-authored-by: Jeppe Dakin <jeppe_dakin@hotmail.com>
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Feb 21, 2021
0.29.22 (2021-02-20)
====================

Features added
--------------
* Some declarations were added to the provided pxd includes.

Bugs fixed
----------
* A crash when calling certain functions in Py3.9 and later was resolved.

* ``const`` memory views of structs failed to compile.

* ``const`` template declarations could not be nested.

* The declarations in the ``cpython.pycapsule`` module were missing their
  ``const`` modifiers and generated incorrect C code.

* Casts to memory views failed for fused dtypes.

* ``repr()`` was assumed to return ``str`` instead of ``unicode`` with ``language_level=3``.

* Calling ``cpdef`` functions from cimported modules crashed the compiler.

* Cython no longer validates the ABI size of the NumPy classes it compiled against.
  See the discussion in numpy/numpy#432

* A C compiler warning about enum value casting was resolved in GCC.

* Coverage reporting in the annotated HTML file failed in Py3.9.

* The embedding code now reports Python errors as exit status.

* Long type declarations could lead to (harmless) random changes in the
  C file when used in auto-generated Python wrappers or pickled classes.

Other changes
-------------
* Variables defined as ``cpdef`` now generate a warning since this
  is currently useless and thus does not do what users would expect.
Athanaseus pushed a commit to kernsuite-debian/pybind11 that referenced this pull request Mar 23, 2021
to ignore "RuntimeWarning: numpy.ufunc size changed, may indicate
binary incompatibility" during pytest runs, as already done for
numpy.dtype.

Triggered by numpy upgrades when scipy is not built against the new
numpy, not considered harmful. See discussions at
numpy/numpy#432
scipy/scipy#3863

Closes: #917722.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants