[libclang/python] Register integer returning library functions as integer instead of bool

[DeinAlptraum]

In the libclang/python bindings, we currently have 43 library library functions registered as ctypes functions returning bool. None of the libclang functions on C-side seem to return booleans, instead they return unsigned (and sometimes signed) ints.

The ctypes module seems to automatically cast the actual unsigned return values to a value of the registered bool type, which works as expected: a return of 0 is interpreted as false, all other returns are interpreted as true.
However, this makes the code confusing to read as it obfuscates the real return type of these library functions in the bindings code. The library functions should be registered with return types matching their declaration, e.g. c_uint.

This means

• checking all bool return types in the FUNCTION_LIST and adapting their return types to c_uint or c_int respectively
• adding a bool cast to the return of all functions directly returning the results of these library functions
• removing type: ignore [no-any-return] annotations where such a cast is applied

[Endilll]

Thank you for filing the issue.
In #162882 (comment) I tried to look at ctypes documentation and implementation to better understand how this (seemingly implicit) conversion happens. I wasn't able to find anything relevant in the docs, but in the implementation I found that PyBool_FromLong might be involved: https://github.com/python/cpython/blob/b3a38438d83245e52dff80f348c7fde539333cea/Modules/_ctypes/cfield.c#L709-L714. Which makes me quite uncomfortable, because both long and long long can be (and often is) wider than unsigned.

[DeinAlptraum]

I just looked at the docs again, it seems that ctypes restype can be both ctypes defined C-types (e.g. c_uint) or Python callables (including classes). In the latter case the result of the library function call is passed to said callable and the result of that is returned, allowing for error-checking and conversion.

To quote from https://docs.python.org/3/library/ctypes.html#return-types

You can also use a callable Python object (a function or a class for example) as the restype attribute, if the foreign function returns an integer. The callable will be called with the integer the C function returns, and the result of this call will be used as the result of your function call. This is useful to check for error return values and automatically raise an exception:

So it seems that annotating a lib function foo() as bool will decorate any call to lib.foo() as bool(foo()) in this case.

[Endilll]

Hmm, I think now I understand what's happening today.

Starting with C++ implementation of C functions, we return true or false from a function that returns unsigned. This integral promotion is specified to return exactly zero or one ([conv.prom]/7), and has always been (I checked C++98). That integral value is put into the return slot of unsigned size. Then control flow goes back to Python, where the return slot is interpreted as int. unsigned int is a corresponding unsigned integer type for int, and they are specified by both Standards to have the same width, which means there are no out-of-bound access concerns. Value from the return slot (either 0 or 1) is passed to bool constructor, then the boolean value is returned out of the C function call to the rest of the Python code.

My conclusion is that nothing is on fire, but this clever behavior adds to the cognitive burden of understanding exactly what happens in this sensitive area of ABI, and no tool can ensure that assumptions made on both side of ABI boundary match exactly.

[DeinAlptraum]

Sounds good to me, I don't really like this double-edged "it's just a type" / "it might be a callable" thing either, and it just obfuscates what the code does. Less magic and more explicitness is the way to go.

[TApplencourt]

I can have a stab at it, if nobody is already working on it.

Do you guys prefer a simple sed adding explicit bool cast, or a meta-generation approach? (from the table definition, we can generate all the function, it have the benefice en enforcing consistency but add a layer of complexity I suppose)

[Endilll]

I can have a stab at it, if nobody is already working on it.

Do you guys prefer a simple sed adding explicit bool cast, or a meta-generation approach? (from the table definition, we can generate all the function, it have the benefice en enforcing consistency but add a layer of complexity I suppose)

I'm not working on this, but @DeinAlptraum might be.
Less layers of complexity, please. Thank you.