[RFE] Support Python 3.11 in the C extension #68

gotmax23 · 2022-11-05T02:29:18Z

Welcome! You should write in your Bug Report:

Please add support for Python 3.11 to the C extension.

OS version: Fedora Rawhide

Python3 version (python3 -V -V):
3.11.0

Steps to reproduce:

Attempt to build a wheel

Actual result (with the python stack trace if present):
the result you see

  cc1: fatal error: frozendict/src/3_11/frozendictobject.c: No such file or directory
  compilation terminated.
  error: command '/usr/bin/gcc' failed with exit code 1
  error: subprocess-exited-with-error

The text was updated successfully, but these errors were encountered:

lurch · 2022-11-07T10:12:44Z

#67 (comment)

This improves default behavior of setup.py in sense that py version of frozendict is used for python versions which do not have C extension available yet. This would probably reduce amount of issues similar to [1-4] in future. [1] Marco-Sulla#68 [2] Marco-Sulla#67 [3] Marco-Sulla#65 [4] Marco-Sulla#58

mcepl · 2023-02-21T12:54:49Z

Even copying 3_10 to 3_11 directory doesn’t help:

[   13s] running build_ext
[   13s] building 'frozendict._frozendict' extension
[   13s] creating build/temp.linux-x86_64-cpython-311
[   13s] creating build/temp.linux-x86_64-cpython-311/frozendict
[   13s] creating build/temp.linux-x86_64-cpython-311/frozendict/src
[   13s] creating build/temp.linux-x86_64-cpython-311/frozendict/src/3_11
[   13s] gcc -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -fPIC -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/Include -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/stringlib -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/clinic -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src -I/usr/include/python3.11 -c frozendict/src/3_11/frozendictobject.c -o build/temp.linux-x86_64-cpython-311/frozendict/src/3_11/frozendictobject.o -DPY_SSIZE_T_CLEAN
[   13s] In file included from frozendict/src/3_11/frozendictobject.c:6:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘_PyDict_CheckConsistency’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:491:36: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]   491 |                 CHECK(mp->ma_values[i] != NULL);
[   13s]       |                                    ^
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:434:16: note: in definition of macro ‘CHECK’
[   13s]   434 |     do { if (!(expr)) { _PyObject_ASSERT_FAILED_MSG(op, Py_STRINGIFY(expr)); } } while (0)
[   13s]       |                ^~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:491:23: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]   491 |                 CHECK(mp->ma_values[i] != NULL);
[   13s]       |                       ^~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:434:16: note: in definition of macro ‘CHECK’
[   13s]   434 |     do { if (!(expr)) { _PyObject_ASSERT_FAILED_MSG(op, Py_STRINGIFY(expr)); } } while (0)
[   13s]       |                ^~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘_d_PyDict_Next’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:799:30: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]   799 |         value = mp->ma_values[i];
[   13s]       |                              ^
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:799:17: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]   799 |         value = mp->ma_values[i];
[   13s]       |                 ^~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘dict_dealloc’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:830:25: warning: initialization of ‘PyObject **’ {aka ‘struct _object **’} from incompatible pointer type ‘PyDictValues *’ {aka ‘struct _dictvalues *’} [-Wincompatible-pointer-types]
[   13s]   830 |     PyObject **values = mp->ma_values;
[   13s]       |                         ^~
[   13s] In file included from /usr/include/python3.11/Python.h:45,
[   13s]                  from frozendict/src/3_11/frozendictobject.c:1:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘dict_traverse’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:39: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                                       ^
[   13s] /usr/include/python3.11/objimpl.h:199:13: note: in definition of macro ‘Py_VISIT’
[   13s]   199 |         if (op) {                                                       \
[   13s]       |             ^~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:26: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                          ^~
[   13s] /usr/include/python3.11/objimpl.h:199:13: note: in definition of macro ‘Py_VISIT’
[   13s]   199 |         if (op) {                                                       \
[   13s]       |             ^~
[   13s] In file included from /usr/include/python3.11/Python.h:38:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:39: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                                       ^
[   13s] /usr/include/python3.11/pyport.h:24:38: note: in definition of macro ‘_Py_CAST’
[   13s]    24 | #define _Py_CAST(type, expr) ((type)(expr))
[   13s]       |                                      ^~~~
[   13s] /usr/include/python3.11/objimpl.h:200:30: note: in expansion of macro ‘_PyObject_CAST’
[   13s]   200 |             int vret = visit(_PyObject_CAST(op), arg);                  \
[   13s]       |                              ^~~~~~~~~~~~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:17: note: in expansion of macro ‘Py_VISIT’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                 ^~~~~~~~
[   13s] /usr/include/python3.11/pyport.h:24:37: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]    24 | #define _Py_CAST(type, expr) ((type)(expr))
[   13s]       |                                     ^
[   13s] /usr/include/python3.11/object.h:107:28: note: in expansion of macro ‘_Py_CAST’
[   13s]   107 | #define _PyObject_CAST(op) _Py_CAST(PyObject*, (op))
[   13s]       |                            ^~~~~~~~
[   13s] /usr/include/python3.11/objimpl.h:200:30: note: in expansion of macro ‘_PyObject_CAST’
[   13s]   200 |             int vret = visit(_PyObject_CAST(op), arg);                  \
[   13s]       |                              ^~~~~~~~~~~~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:17: note: in expansion of macro ‘Py_VISIT’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                 ^~~~~~~~
[   13s] frozendict/src/3_11/frozendictobject.c: In function ‘frozendict_merge’:
[   13s] frozendict/src/3_11/frozendictobject.c:453:41: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]   453 |                 value = other->ma_values[i];
[   13s]       |                                         ^
[   13s] frozendict/src/3_11/frozendictobject.c:453:25: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]   453 |                 value = other->ma_values[i];
[   13s]       |                         ^~~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: At top level:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:431:1: warning: ‘_PyDict_CheckConsistency’ defined but not used [-Wunused-function]
[   13s]   431 | _PyDict_CheckConsistency(PyObject *op, int check_content)
[   13s]       | ^~~~~~~~~~~~~~~~~~~~~~~~
[   13s] warning: build_ext: building extension "frozendict._frozendict" failed: command '/usr/bin/gcc' failed with exit code 1

BTW with all versions of Python I get this warning:

[   12s] In file included from frozendict/src/3_9/frozendictobject.c:6:
[   12s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_9/cpython_src/Objects/dictobject.c:443:1: warning: ‘_PyDict_CheckConsistency’ defined but not used [-Wunused-function]
[   12s]   443 | _PyDict_CheckConsistency(PyObject *op, int check_content)
[   12s]       | ^~~~~~~~~~~~~~~~~~~~~~~~

Marco-Sulla · 2023-02-25T19:22:31Z

@mcepl Well, dict in CPython 3.11 is changed a lot. I started to incorporate the new code into frozendict in the cpython_3_11 branch.

PhorstenkampFuzzy · 2023-04-12T12:18:10Z

For me a pipline just broke on because the newer version where not usable on 3.11. But the older versions build just fine for me on 3.11 using poetry and the python:3.11-slim image. Can it lead to problems?

Marco-Sulla · 2023-04-12T12:27:16Z

@PhorstenkampFuzzy

This is quite strange.

frozendict setup now builds a pure python wheel for archs and/or Python versions not (yet) supported by the C Extension.

Can you post:

the frozendict version
the pipeline
the error

?

PhorstenkampFuzzy · 2023-04-12T14:36:34Z

It does work fine for me and builds on the fly.
I am using:
python 3.11.0 on windows64 bit and the current python:3.11-slim docker image.
I am installing it as a poetry dependency and as a dependency when installing a wheel.
I am currently using 2.3.5 and 2.3.4. (Where there is a *.tar.gz published).

I have no problem at all so long as i do not increase the version to something where there is no *.tar.gz to build from.

Marco-Sulla · 2023-04-12T18:16:28Z

@PhorstenkampFuzzy Yes, this is fixed in 2.3.7. Sorry for the confusion.

See also Marco-Sulla/python-frozendict#68, rely on a pure Python implementation when 3.11+ is used.

* py-frozendict: patch up for Python 3.11 See also Marco-Sulla/python-frozendict#68, rely on a pure Python implementation when 3.11+ is used. * mention related Github issue

gotmax23 added the Type: Bug Something isn't working label Nov 5, 2022

arkamar mentioned this issue Dec 3, 2022

Autodetect availability of C extension #69

Merged

jan-janssen mentioned this issue Feb 14, 2023

Update to 3.11 conda-forge/frozendict-feedstock#38

Closed

5 tasks

DMRobertson mentioned this issue Feb 20, 2023

CI fails to test Python 3.11 wheel on Ubuntu 20.04 matrix-org/synapse#15109

Closed

Marco-Sulla added Priority: High A blocking bug or an important feature Effort: High tons of caffeine needed Type: Enhancement New feature or request and removed Type: Bug Something isn't working labels Feb 25, 2023

Marco-Sulla mentioned this issue Feb 25, 2023

TODO list #34

Open

Marco-Sulla mentioned this issue Nov 18, 2023

[BUG] Python implementation pickles differently than native implementation #87

Closed

matz-e added a commit to matz-e/spack that referenced this issue Jan 19, 2024

py-frozendict: patch up for Python 3.11

a1d319a

See also Marco-Sulla/python-frozendict#68, rely on a pure Python implementation when 3.11+ is used.

matz-e mentioned this issue Jan 19, 2024

py-frozendict: patch up for Python 3.11 spack/spack#42192

Merged

matz-e added a commit to matz-e/spack that referenced this issue Jan 22, 2024

py-frozendict: patch up for Python 3.11

1151a4f

See also Marco-Sulla/python-frozendict#68, rely on a pure Python implementation when 3.11+ is used.

skupr-anaconda mentioned this issue Apr 15, 2024

Bump to 2.4.2 AnacondaRecipes/frozendict-feedstock#2

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFE] Support Python 3.11 in the C extension #68

[RFE] Support Python 3.11 in the C extension #68

gotmax23 commented Nov 5, 2022

lurch commented Nov 7, 2022

mcepl commented Feb 21, 2023

Marco-Sulla commented Feb 25, 2023

PhorstenkampFuzzy commented Apr 12, 2023

Marco-Sulla commented Apr 12, 2023

PhorstenkampFuzzy commented Apr 12, 2023

Marco-Sulla commented Apr 12, 2023 •

edited

[RFE] Support Python 3.11 in the C extension #68

[RFE] Support Python 3.11 in the C extension #68

Comments

gotmax23 commented Nov 5, 2022

lurch commented Nov 7, 2022

mcepl commented Feb 21, 2023

Marco-Sulla commented Feb 25, 2023

PhorstenkampFuzzy commented Apr 12, 2023

Marco-Sulla commented Apr 12, 2023

PhorstenkampFuzzy commented Apr 12, 2023

Marco-Sulla commented Apr 12, 2023 • edited

Marco-Sulla commented Apr 12, 2023 •

edited