ctypes: pointer is a function, not a class

[AlexWaygood]

Fixes #8351. Fixes #7747. Closes #7753.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

psycopg (https://github.com/psycopg/psycopg)
+ psycopg/psycopg/pq/_pq_ctypes.pyi:13: error: Unused "type: ignore" comment
+ psycopg/psycopg/pq/_pq_ctypes.pyi:13: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:13: note: Error code "valid-type" not covered by "type: ignore" comment
+ psycopg/psycopg/pq/_pq_ctypes.pyi:13: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:65: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:65: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:72: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:72: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:113: error: Unused "type: ignore" comment
+ psycopg/psycopg/pq/_pq_ctypes.pyi:113: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:113: note: Error code "valid-type" not covered by "type: ignore" comment
+ psycopg/psycopg/pq/_pq_ctypes.pyi:113: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:136: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:136: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:161: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:161: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:185: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:185: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:186: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:186: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:187: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:187: note: Perhaps you need "Callable[...]" or a callback protocol?
+ psycopg/psycopg/pq/_pq_ctypes.pyi:189: error: Function "ctypes.pointer" is not valid as a type  [valid-type]
+ psycopg/psycopg/pq/_pq_ctypes.pyi:189: note: Perhaps you need "Callable[...]" or a callback protocol?

[AlexWaygood]

As expected, the "moral" thing to do here is quite disruptive. @srittau, what do you think? Worth it anyway?

[AlexWaygood]

As expected, the "moral" thing to do here is quite disruptive.

Having said that, it looks like the psycopg stubs which mypy is erroring on here are autogenerated: https://github.com/psycopg/psycopg/blob/cace92a9854130da02e55f416b2bcbefdc7242e1/psycopg/psycopg/pq/_pq_ctypes.py#L727.

So the fix for that library would be pretty trivial, I think.

[srittau]

Let's do this. It might hurt now, but will probably prevent more hurt in the future.

[AlexWaygood]

@dvarrazzo, just to give you a heads-up about this — the typeshed stubs for ctypes have been a bit of a hack for a while. This PR brings them closer to the runtime reality, but it means you might have to change your stubs for psycopg (after a future release of mypy) to use ctypes._Pointer instead of ctypes.pointer.

[dvarrazzo]

@AlexWaygood no problem at all! When this change will be released I'll be happy to update our stubs and set a dependency on the newer mypy version 🙂

[nathanpage-credo]

As @AlexWaygood mentioned, this breaks things where a user may want to define pointer annotations:

import ctypes as ct

class Foo(ct.Structure):
    ... # other stuff defined here 
    test: "ct.pointer[ct.c_int32]"  # these now error
    test2: "ct.pointer[ct.c_bool]"  # these now error

Could ctypes._Pointer be a non-internal type (e.g., ctypes.Pointer)?

[JelleZijlstra]

Could ctypes._Pointer be a non-internal type (e.g., ctypes.Pointer)?

Maybe, but that's not a decision we can make in typeshed. If you want it to be, you should open an issue over at CPython.

[nathanrpage97]

Ok, I don’t think it is worthy of a change to cpython implementation. The big thing is linters not liking a reference to a private api. I think for now I will re-export ctypes._Pointer -> PointerType to reduce linter silencing needed.

[Akuli]

Cpython might accept making it a public type, a bit similarly to how _curses.window is now exposed as curses.window.

[AlexWaygood]

Cpython might accept making it a public type, a bit similarly to how _curses.window is now exposed as curses.window.

Especially given that ctypes._Pointer is actually documented -- it seems like it's not really meant to be a "private" class.