syntactic sugar: type coercion on pointer assignment

[PeterASilva]

BPO	5149
Nosy	@amauryfa, @abalkin, @meadori

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2009-02-04.01:20:02.204>
labels = ['ctypes', 'type-feature', '3.10']
title = 'syntactic sugar: type coercion on pointer assignment'
updated_at = <Date 2020-10-25.22:49:47.999>
user = 'https://bugs.python.org/PeterASilva'

bugs.python.org fields:

activity = <Date 2020-10-25.22:49:47.999>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['ctypes']
creation = <Date 2009-02-04.01:20:02.204>
creator = 'PeterASilva'
dependencies = []
files = []
hgrepos = []
issue_num = 5149
keywords = []
message_count = 5.0
messages = ['81123', '141747', '143452', '143657', '143659']
nosy_count = 5.0
nosy_names = ['amaury.forgeotdarc', 'belopolsky', 'PeterASilva', 'meador.inge', 'vladris']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue5149'
versions = ['Python 3.10']

[PeterASilva]

tough% cat toy.py

from ctypes import *

class foo(Structure):
  _fields_ = [ ( "bar", c_char_p ) ]

foofoo = foo()

babar = create_string_buffer(32)

foofoo.bar = babar

tough% python toy.py
Traceback (most recent call last):
  File "toy.py", line 11, in <module>
    foofoo.bar = babar
TypeError: incompatible types, c_char_Array_32 instance instead of
c_char_p instance
tough%

would be nice if ctypes automagically turned the assignment into

   foofoo.bar = cast(babar,c_char_p)

There doesn't seem to be any other possible intent of this syntax.
Color me stupid, but being new to ctypes, it took me approximately 30
hours to discover a method of such assignment that worked (babar.raw is
not rejected, but not correct, debugging that was fun. I actually fell
upon addressof(babar), which does also works, but is probably
undesirable.)

It would be very friendly if ctypes would just do the right thing.
It doesn't seem like there is any other possible intent of such
a syntax.

[vladris]

The main reason for this issue is that internally ctypes doesn't consider c_char_p as a pointer type. This is a bit counter-intuitive, but c_char_p is treated as a simple type and has some other sugar built-in, like easier integration with Python strings. Alternative pointer type is POINTER(c_char), which allows assignment from array.

I would make this clear in the documentation. Even now, documentation says following about c_char_p:

Represents the C char * datatype when it points to a zero-terminated string. For a general character pointer that may also point to binary data, POINTER(c_char) must be used. The constructor accepts an integer address, or a string.

So there already is a hint that c_char_p has limited use for convenience. We should maybe make documentation more clear. If you want equivalent of C char* behavior, POINTER(c_char) is the way to go.

[meadori]

This is busted for plain old assignment too:

Python 3.3.0a0 (default:6374b4ffe00c, Sep  2 2011, 23:50:39) 
[GCC 4.6.0 20110603 (Red Hat 4.6.0-10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> buff = create_string_buffer(b'foo')
>>> p = c_char_p()
>>> p.value = addressof(buff)
>>> print(p.value)
b'foo'
>>> p.value = buff.value
>>> print(p.value)
b'foo'
>>> p.value = buff
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: string or integer address expected instead of c_char_Array_4 instance

I think having the conversion is an entirely reasonable request. It is the equivalent of:

    char buff[128] = "foo";
    char *p = buff;

in C. I imagine a lot of C programmers would expect this behavior.

Also, 'ctypes' already does this type of conversion for function parameters. Consider a function exported from a shared library 'libfoo' called 'print_string':

void print_string (char *str)
{
  printf ("%s\n", str);
}

The following all work fine:

>>> libfoo = CDLL("./libfoo.so.1.0")
>>> buff = create_string_buffer("foo")
>>> libfoo.print_string(buff)
foo
>>> libfoo.print_string("foo")
foo
>>> libfoo.print_string(addressof(buff))
foo

I am working on a patch for this. I will post it soon.

[vladris]

I believe there is a deeper issue here in ctypes design. Basically we provide both c_char_p and POINTER(c_char) which should behave exactly the same since both are the equivalent of char* in C but internally they have different implementations.

c_char_p is considered a "simple type" and I believe supports some conversions to and from Python strings while POINTER(c_char) is considered a pointer type which supports assignment from array etc.

I think a better fix would be to deprecate p_char_p or make it an equivalent of POINTER(c_char), otherwise we will have to do work on c_char_p to make it more like POINTER(c_char) when issues like this get opened and probably also make POINTER(c_char) more like c_char_p. Why not just have POINTER(c_char) which works as expected? I don't have all the historical context on why this pseudo-simple type was provided but I saw a couple of issues where people expect it to behave like a C char* but it won't because it is implemented as a convenience type with limited support.

[meadori]

Vlad, I agree that having both 'c_char_p' and 'POINTER(c_char)' will just be more work for two seemingly identical constructs. I don't fully understand why both would be needed either (or the implications of removing one of them or making them synonyms). I guess a little software archeology is in order.