Don't expose PyASCIIObjectState on Python3.14 and newer

[ngoldbaum]

Fixes #5131.

This changed in between alpha 7 and beta 1: python/cpython#133085

[ngoldbaum]

Also set_interned is definitely wrong on the free-threaded build. But also why are we exposing setters at all? The C API doesn't expose setters.

[davidhewitt]

I wonder, in 3.14 PyUnicode_KIND and PyUnicode_DATA are now exposed as symbols we can call instead of having to decode the bitfield.

Should we just rip out the bitfield decoding on 3.14 (and up) and instead use the API functions? I think we have to drop a couple of the APIs we currently offer, but overall would be a lot less code to maintain, less coupling to cpython details, and a lot sounder.

[ngoldbaum]

Should we just rip out the bitfield decoding on 3.14 (and up) and instead use the API functions? I think we have to drop a couple of the APIs we currently offer, but overall would be a lot less code to maintain, less coupling to cpython details, and a lot sounder.

Ah neat, guess I should have checked that.

This seems like a good idea, let me see how hard it is...

[ngoldbaum]

This seems like a good idea, let me see how hard it is...

I don't see how to implement PyUnicode_IS_ASCII, PyUnicode_IS_COMPACT and PyUnicode_IS_ASCII_COMPACT without having access to the bitfields....

[ngoldbaum]

For whatever reason, ffi-check is broken on 3.14, but only on Windows 🙃

[ngoldbaum]

I can't reproduce the 3.14 windows failures on my Windows setup.

[davidhewitt]

This seems like a good idea, let me see how hard it is...

I don't see how to implement PyUnicode_IS_ASCII, PyUnicode_IS_COMPACT and PyUnicode_IS_ASCII_COMPACT without having access to the bitfields....

I was thinking maybe to make those functions inaccessible on 3.14 +?

I can't reproduce the 3.14 windows failures on my Windows setup.

I will try to find time to look on Monday.

[clin1234]

@ngoldbaum Beat me to the punch; can you grant me commit access to your branch? I was thinking of adding a deprecation message to the entire PyASCIIObject (due to python/cpython#133085). Also this: clin1234@21ce132

[clin1234]

Also, this: https://github.com/python/cpython/blob/3.14/Include/cpython/unicodeobject.h#L102-L162
The state member has to be aligned in 4-byte boundaries for non-GIL builds, and I don't think the numerous brittle bit-twiddling functions to convert between pyo3's u32 state and struct PyASCIIObjectState are suitable in the long-term.

[ngoldbaum]

The last push removes PyASCIIObjectState on Python 3.14 and newer.

I think the mysterious Windows failures went away? The remaining failures are unrelated issues in the jiff tests.

[ngoldbaum]

I think the mysterious Windows failures went away?

Spoke too soon. Somehow the thing we did with the bindgen ParseCallbacks to handle the anonymous struct in the ob_refcnt field seems to be breaking, but only on Windows, and only intermittently.

[davidhewitt]

Thanks, couple of final tweaks and then I think good to merge 👍

The windows ffi check is at least not going to block merge, so I suggest we merge this, ship 0.25, and investigate / clean up separately.

[davidhewitt]

Thanks!