PyUnicode_READ() fails to build on C++: _Py_CAST() used with a constant type (const void*)

[da-woods]

Bug report

The new _Py_CAST macro (https://github.com/python/cpython/pull/91959/files, and other commits) designed to reduce C++ warnings adds a fairly significant restriction that wasn't there before

cpython/Include/pyport.h

Lines 25 to 26 in 9f68dab

	// The type argument must not be constant. For example, in C++,
	// _Py_CAST(const PyObject*, expr) fails with a compiler error.

The PyUnicode_READ macro casts to a const pointer

cpython/Include/cpython/unicodeobject.h

Lines 341 to 343 in 9f68dab

	#define PyUnicode_READ(kind, data, index) \
	PyUnicode_READ(_Py_STATIC_CAST(int, kind), _Py_CAST(const void*, data), \
	(index))

This is currently reported to be breaking Cython (cython/cython#4790).

I'd propose a fix which removes the const requirement:

template <typename T>
struct _Py_add_const {
    typedef const T type;
};
#  define _Py_CAST(tp, expr) \
       const_cast<tp>(reinterpret_cast<_Py_add_const<tp>::type>(expr))

on C++11 you could use the standard library std::add_cast instead, at the cost of an extra include. However it isn't available on c++03.

I realise this adds an extra struct _Py_add_const to the namespace. I'm not sure if that's desired, or how it should be named. If that's acceptable I'm happy to submit a PR for this.

@vstinner

Your environment

Current 3.11 branch

[tacaswell]

This issue is scattered across a couple of places (which may be my fault) here is an index of the relevant links:

• original issue to clean up casts: [C API] Add private "CAST" macros to clean up casts in C code #91320
• first PR changing how casting was done: gh-91320: Add _Py_reinterpret_cast() macro #91959
• issue to greenlets that was broken by the above py311 issues with const correctness python-greenlet/greenlet#301
• A PR to greenlets that fixes two issues from the above CPython change Clarify that const is being cast away python-greenlet/greenlet#302
• An issue that was opened to track the const related issue In C++, Python 3.11 C API introduces new compiler warnings/errors #92135
• a PR that fixed one of the greenlet compile issues gh-92135: Fix _Py_reinterpret_cast() for const #92138
• a comment from @serge-sans-paille on how to fix this gh-92135: Fix _Py_reinterpret_cast() for const #92138 (comment)
• the cython issue mentioned above [BUG] py311 compatibilty cython/cython#4790

[vstinner]

This is currently reported to be breaking Cython (cython/cython#4790).

The C++ project scipy fails to build with:

/home/tcaswell/.pybuild/bleeding/include/python3.11/cpython/unicodeobject.h:409:57: error: duplicate ‘const’
  409 |     PyUnicode_READ(_Py_STATIC_CAST(int, kind), _Py_CAST(const void*, data), \
      |                                                         ^~~~~

Oh right, that's my fault, _Py_CAST() cannot be used with a const type. _Py_CAST() comment:

// The type argument must not be constant. For example, in C++,
// _Py_CAST(const PyObject*, expr) fails with a compiler error.

[vstinner]

Would you be able to test if #92872 fix scipy?

[tacaswell]

I can confirm that scipy (and a large fraction of the pydata stack) build with #92872 .

[vstinner]

Fixed by #92872 : thanks for the bug report!