Multiple problems in getters & setters in capsulethunk.h

[encukou]

BPO	24937
Nosy	@ncoghlan, @larryhastings, @encukou, @berkerpeksag
PRs	bpo-24937: Replace the extension module porting HOWTO by links to external projects #9317
Files	capsulethunk.patch capsulethunk-py3c.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-09-24.10:44:31.101>
created_at = <Date 2015-08-25.16:41:32.928>
labels = ['3.8', 'type-feature', '3.7', 'docs']
title = 'Multiple problems in getters & setters in capsulethunk.h'
updated_at = <Date 2018-09-24.10:44:31.100>
user = 'https://github.com/encukou'

bugs.python.org fields:

activity = <Date 2018-09-24.10:44:31.100>
actor = 'petr.viktorin'
assignee = 'docs@python'
closed = True
closed_date = <Date 2018-09-24.10:44:31.101>
closer = 'petr.viktorin'
components = ['Documentation']
creation = <Date 2015-08-25.16:41:32.928>
creator = 'petr.viktorin'
dependencies = []
files = ['40256', '40403']
hgrepos = []
issue_num = 24937
keywords = ['patch']
message_count = 10.0
messages = ['249138', '249139', '249140', '249156', '250181', '250191', '250198', '323125', '325385', '326220']
nosy_count = 5.0
nosy_names = ['ncoghlan', 'larry', 'petr.viktorin', 'docs@python', 'berker.peksag']
pr_nums = ['9317']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue24937'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[encukou]

In https://docs.python.org/3/howto/cporting.html#cobject-replaced-with-capsule (added in bpo-13053):

1. __PyCapsule_GetField is defined as::

    #define __PyCapsule_GetField(capsule, field, default_value) ...

but called as::

    __PyCapsule_GetField(capsule, field)

2. __PyCapsule_SetField returns the wrong value (1 for success, 0 for failure).

3. Both the getter and setter don't set the exception on failure, leading to "SystemError: error return without exception set".

Here's a patch.

[encukou]

Note:
I'm including a tested (+fixed, now) copy of capsulethunk.h in my set of Python3 C extension porting helpers, http://py3c.readthedocs.org

Unfortunately the tests can't be added to CPython, because they need to run on Python 2.6 to be useful. That makes me think an external project might make a better home for this piece of code.
If you agree, I can take over maintainership of capsulethunk, and make py3c the canonical home for it.

[larryhastings]

Given that capsulethunk is only needed for 2.6 (and previous), and those versions are no longer maintained by the Python core community, yes I'd be happy for you to take over maintainership and host it externally. I'm sure we could change the documentation in 2.7, 3.4, and 3.5 to reflect this.

[encukou]

Thanks! I'll take good care of it :)
Give me a few days to prepare the docs change; this isn't very high priority.

[encukou]

This patch removes capsulethunk.h from the docs, and directs the kind reader to the py3c project, which carries a fixed and tested version of it, along with the docs.

Larry, would you be OK with releasing capsulethunk.h under the MIT license? The PSF license is not suitable for external projects, and mixing licenses is somewhat awkward.

[larryhastings]

I'm the author of capsulethunk.h, and I am happy to relicense it under MIT.

How would you feel about leaving capsulethunk.h there, for posterity's sakes, but adding "for updates please see py3c"? We could even update the capsulethunk.h there. I'm not saying "we have to do it this way", I'm asking whether the users would be better served by leaving the example around vs removing it and forcing them to go looking on an external web site.

[encukou]

Thank you for the license.

I mentioned my project mainly as a place where this code can be *tested*. I have no problem with keeping capsulethunk.h in the Python docs, and synchronizing the two projects if changes are made on either side. In other words, feel free to use the first patch :)

Regarding users – in my opinion, including a whole header directly in the docs is questionable (to me it smells of untested copy-paste code), and the link goes to hg.python.org – an external site from the users' point of view.
Also, if someone needs to port CObject, their extension is very likely to be complicated enough to need much more material than the HOWTO provides.
See py3c's modernization docs_ for several things the HOWTO doesn't address.
So, in my obviously biased opinion, linking to py3c will help users.
(Perhaps a link to py3c should be at the top of the HOWTO, but that's not for this issue. I'm working to get to a point where I'm comfortable suggesting that.)

There's obviously the question of whether I should be improving the HOWTO rather than starting my own thing. I have three reasons: testability, the fact that py3c docs are already way too long for a HOWTO, and the fact that py3c is quite opinionated. (The more opinionated, the more more in-depth the solutions can be. I've been toning it down lately, though.)

_ modernization docs: http://py3c.readthedocs.org/en/latest/guide-modernization.html

[berkerpeksag]

It would be nice to remove PyCObject_* entries from Doc/data/refcounts.dat.

+.. _documentation: http://py3c.readthedocs.org/en/latest/capsulethunk.html

Nit: We could the HTTPS link.

[encukou]

Well, hasn't this languished for a while.

I've talked with Benjamin, and he's OK with just removing the "Porting Extension Modules to Python 3" HOWTO, and replacing it by links to external guides that do a much better job at documenting this.

Also, Python 2.6 and 3.0 is EOL; it should be fine to scrub capsulethunk.

[encukou]

New changeset 2d3ff2b by Petr Viktorin in branch 'master':
bpo-24937: Replace the extension module porting HOWTO by links to external projects (GH-9317)
2d3ff2b