My initial thoughts

[encukou]

I don't know how discussions are supposed to work here, so I'm sending these as an issue:

• This sounds like a Python 2→3 transition, but for the C API. I doubt the existing extension ecosystem can handle the timeline.
• The end goal & rules are very similar to my draft. Of course with a “revolution rather than evolution” approach you can be much stricter & faster, but, this is essentially where I'd like Limited API to end up :)
• For interop with other languages I suggest avoiding some C features: enums, macros, bit fields, to really make this a good common basis for all languages.
• I'd be interested in seeing how type creation would work -- a builder pattern?
• Naming it “new C API” is confusing, at least Victor used the same name for something slightly different.

[markshannon]

This sounds like a Python 2→3 transition

The key difference between this transition and the 2 -> 3 transition is that modules are independent.
If module spam depends on module eggs, spam couldn't be ported to Python 3 until eggs was ported. Also, if eggs were ported to 3, spam would no longer work until it too was ported to 3, hence 6 and other means of supporting both.

With the new C-API, spam and eggs can be ported independently and will continue to work, so the window in which any package can be ported is considerably larger than it was for the 2 -> 3 transition, and there is never a need for a package to support both APIs.

The end goal & rules are very similar to my draft.

I hadn't seen your draft (or if I had, I've forgotten). Are there any specific goals you want to add that aren't included in the design pricinples?

For interop with other languages I suggest avoiding some C features: enums, macros, bit fields, to really make this a good common basis for all languages.

Enums are fine, as they are defined to be the same size as ints, so there are just syntactic sugar for some integer constants.
Enums are OK for defining values, but we need to use ints for struct fields, parameters and return types.

Macros and inline function are OK in the API, as long as they are just convenience wrappers around ABI functions. More so, if they are automatically generated. To support Rust (for example), we generate the equivalent Rust code.

No bit fields, though.

I'd be interested in seeing how type creation would work -- a builder pattern?

My current thinking is that the class would be created declaratively from a layout and inheritance description, then methods and properties would be added procedurally.
Feel free to open an issue.

Naming it “new C API”

It's a description not a name.
#3

[markshannon]

My mistake, enums are not defined to be the same size as ints.
We can use enums to define the values, but return types, struct members and parameters will need to be ints.

[encukou]

Are there any specific goals you want to add

The macros/enums/bitfields is the main point. (I appreciate your addition of va_args to that list.)

After consulting my notes on language interop I have another suggestion. Other languages have issues working with sizes of C integral types, so we should limit which ones are used more than just “Use standard C99 types”.
The examples here use n for uintptr_t in the ABI signatures. IMO, other types should get dedicated sigil letters or be banned.
(Signatures are currently spec'd as Prefix_NameSpace_Operation[_REF_CONSUMPTION] -- that should probably be [_ARGS]?)

We can use enums to define the values, but return types, struct members and parameters will need to be ints.

And by No invalid states, casting such an argument to enum will usually need a range check.
The numeric values will be part of the ABI. For the benefit of non-C languages, they should be documented explicitly.
Not many benefits of enums remain.

Macros and inline function are OK in the API, as long as they are just convenience wrappers around ABI functions. More so, if they are automatically generated.

Yup, reading more of the proposal, this makes sense for many cases. I'll revisit when things are more fleshed out.

type creation

#4

---

And one more:

If a function accepts pointers, it must accept NULL.

This is surprising. Won't all the NULL checks hurt performance?
Wouldn't it be better to say NULL pointers are not safe input? There are lots of unsafe pointer values, singling out NULL is such a C thing to do. (Mandating/autogenerating assert(x) for all pointer args would be good, though.)

Generally I'm not on board with all the tradeoffs -- the other big one for me is “passing the interpreter to each API function” vs. performance, which limits us to platforms with fast per-“thread” storage. I guess that's also for a separate discussion.

[markshannon]

Yes, lots of NULL checks will hurt performance. This was just an example. In this case the best approach is not to have pointers as parameters.
Most functions should take PyRefs or primitive types.
In general, rather than declaring some set of values as unsafe, choose parameters where any reasonable value is safe.

#5

The issue of whether to use thread-local storage or to pass an explicit parameter does need more discussion.
The requirement for safety makes passing interpreters as a parameter problematic though, as we need to check that the passed interpreter is the correct one which costs more than just getting the correct one from thread-local storage.
If we are willing to trust the extension code in this case, and just check in debug mode, then using a parameter would be OK.

#6

[ronaldoussoren]

My mistake, enums are not defined to be the same size as ints.
We can use enums to define the values, but return types, struct members and parameters will need to be ints.

In C++ and the upcoming C23 standard there are ways to declare the underlying integer type for an enum which would solve this. We just have to wait a long time before we can actually use this :-(

[markshannon]

@encukou is there anything left here to act on?

The issue of passing the context explicitly has been resolved. #6

Enums will be only be used to define the set of values that can be returned by a function.
We will use int as the actual return types.

[encukou]

is there anything left here to act on?

That's entirely your call. I listed what you might want to consider, but this is your proposal, so I stay away from arguing points we disagree on.