Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] __Pyx_PyObject_CallOneArg() doesn't support Python 3.9 METH_METHOD flags #3917

Closed
vstinner opened this issue Nov 25, 2020 · 9 comments
Closed

Comments

@vstinner
Copy link
Contributor

The PEP 573 was accepted and implemented in Python 3.9. It adds a new METH_METHOD flag which pass the "defining class".

For example, the zlib.Decompress.decompress() function gets this flag by using the new defining_class parameter type in its Argument Clinic definition in Python 3.10:

/*[clinic input]
zlib.Decompress.decompress

    cls: defining_class
    data: Py_buffer
        The binary data to decompress.
    /
    max_length: Py_ssize_t = 0
        The maximum allowable length of the decompressed data.
        Unconsumed input data will be stored in
        the unconsumed_tail attribute.

(...)
[clinic start generated code]*/

It generates the following calling convention:


#define ZLIB_DECOMPRESS_DECOMPRESS_METHODDEF    \
    {"decompress", (PyCFunction)(void(*)(void))zlib_Decompress_decompress, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Decompress_decompress__doc__},

static PyObject *
zlib_Decompress_decompress(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
    (...)
}

Notice the second parameter: PyTypeObject *cls.

It seems like __Pyx_PyObject_CallOneArg() doesn't support this new calling convention and simply treat the function as the "fastcall" calling convention. It does not pass the defining type.

As a consequence, zlib_Decompress_decompress() does crash when parsing its arguments.

See https://bugzilla.redhat.com/show_bug.cgi?id=1898157 for a concrete example in scipy.

I tested Cython 0.29.21 (python3-Cython-0.29.21-4.1.fc34.x86_64) on Fedora Rawhide with Python 3.10.

@vstinner
Copy link
Contributor Author

The PEP 573 was accepted and implemented in Python 3.9.

FYI it was implemented in https://bugs.python.org/issue38787

cc @encukou

@hroncok
Copy link
Contributor

hroncok commented Nov 25, 2020

I tested Cython 0.29.21 (python3-Cython-0.29.21-4.1.fc34.x86_64) on Fedora Rawhide with Python 3.10.

Note that our Cython is patched with #3877 -- that should not make a difference.

@scoder
Copy link
Contributor

scoder commented Nov 26, 2020

I think the best fix for 0.29.x is to switch to using PyCFunction_CheckExact() here.

@scoder
Copy link
Contributor

scoder commented Nov 26, 2020

Ah, it's actually only an issue in Cython 0.29.x. Cython 3 has a different implementation alltogether here.

@vstinner
Copy link
Contributor Author

Fedora is still using the Cython 0.29 branch.

@scoder
Copy link
Contributor

scoder commented Nov 26, 2020

6ba0345 should fix it.

@hroncok
Copy link
Contributor

hroncok commented Nov 26, 2020

Testing it.

@vstinner
Copy link
Contributor Author

6ba0345 should fix it.

Oh wow, that was quick, thanks @scoder!

@hroncok
Copy link
Contributor

hroncok commented Nov 27, 2020

Testing it.

It works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants