How to deal with C-API extensions?

[windelbouwman]

Create a prototype on how to interact with a python library with a C-API extension.

See also:
https://pythoncapi.readthedocs.io/

There are some questions regarding C extensions. Primary question: how to handle this? Maybe in a similar way to cpyext?

[windelbouwman]

Just wanted to bump this issue with a link to handlepy: https://github.com/pyhandle/hpy this might be the way to go forwards.

[ShadowJonathan]

hpy is definitely the way, it's getting increasing support and attention from the python community

[encukou]

Hi!
I help maintain CPython's Limited API & Stable ABI. If you want to support a subset of the C API, here's one. Not perfect -- it has decade of historical baggage itself -- but it's there. A bunch of real-world extensions use it, including PyO3 (there's even a suggestion to make it the default).

To support it, you'd need to expose a PyObject struct with a refcount (C int) and a type (C pointer). Roughly speaking, those are the only two instances where Stable ABI extensions reach directly into memory; everything else is a function call. (Replacing refcounting by function calls turned out to be too slow, unfortunately.)

I'd love feedback on what is hard to implement in Rust. We can add better API for the bad parts, and eventually even deprecate/remove stuff.
Feel free to contribute to my shared notes, post here, or reach me in any other way :)