Derby #10: Convert 50 sites to Argument Clinic across 4 files

[larryhastings]

BPO	20179
Nosy	@loewis, @pitrou, @taleinat, @larryhastings, @giampaolo, @tiran, @alex, @serhiy-storchaka, @dstufft, @ZackerySpytz
PRs	bpo-20179: Convert the _overlapped module to the Argument Clinic #14275
Files	bytes_and_bytearray.patch bytes_and_bytearray_2.patch ssl_clinic.patch ssl_clinic_2.patch irc.transcript.of._ssl.clinic.discussion.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-07-10.18:49:47.074>
created_at = <Date 2014-01-08.00:15:20.609>
labels = ['extension-modules', 'type-feature', 'expert-argument-clinic']
title = 'Derby #10: Convert 50 sites to Argument Clinic across 4 files'
updated_at = <Date 2020-07-10.18:49:47.074>
user = 'https://github.com/larryhastings'

bugs.python.org fields:

activity = <Date 2020-07-10.18:49:47.074>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-10.18:49:47.074>
closer = 'serhiy.storchaka'
components = ['Extension Modules', 'Argument Clinic']
creation = <Date 2014-01-08.00:15:20.609>
creator = 'larry'
dependencies = []
files = ['33599', '33601', '38842', '39072', '39283']
hgrepos = []
issue_num = 20179
keywords = ['patch', 'needs review']
message_count = 27.0
messages = ['207635', '208590', '208698', '208713', '224134', '224135', '224760', '240149', '240150', '240152', '241234', '242473', '242475', '242477', '242479', '242481', '242493', '242494', '242523', '242524', '242525', '242526', '242550', '242551', '327366', '346162', '373474']
nosy_count = 12.0
nosy_names = ['loewis', 'janssen', 'pitrou', 'taleinat', 'larry', 'giampaolo.rodola', 'christian.heimes', 'alex', 'python-dev', 'serhiy.storchaka', 'dstufft', 'ZackerySpytz']
pr_nums = ['14275']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20179'
versions = ['Python 3.5']

[larryhastings]

This issue is part of the Great Argument Clinic Conversion Derby,
where we're trying to convert as much of Python 3.4 to use
Argument Clinic as we can before Release Candidate 1 on January 19.

This issue asks you to change the following bundle of files:
Modules/overlapped.c: 21 sites
Modules/_ssl.c: 20 sites
Objects/bytes_methods.c: 1 sites
Objects/bytesobject.c: 8 sites

Talk to me (larry) if you only want to attack part of a bundle.

For instructions on how to convert a function to work with Argument
Clinic, read the "howto":
http://docs.python.org/dev/howto/clinic.html

[taleinat]

While converting Objects/stringlib/transmogrify.h as part of bpo-20180 (Derby #11), some changes to Objects/bytesobject.c and Objects/bytearrayobject.c were required. Those changes are included in the relevant patch attached to that issue.

[taleinat]

Attached patch for AC conversion of Objects/bytesobject.c and Objects/bytearrayobject.c.

This is one patch because there are changes that must be done in bytes_methods.h and bytes_methods.c that affect both of bytesobject.c and bytearrayobject.c. Those changes are in this patch as well.

All methods were converted except for two groups:

1. the various find methods which use the common argument parsing function 'stringlib_parse_args_finds_byte'
2. 'new' and 'init' methods

Also note that both of these classes use some common function implementations found under 'stringlib'. However, since that code is in separate files, I'll upload the conversion patches to the relevant issue (and write an appropriate comment here).

[taleinat]

Attached updated patch for bytes and bytearray.

In the previous patch I missed the 'clear' and 'copy' bytearray methods. This patch is a replacement for the previous one, with the additional conversion of these two methods.

[python-dev]

New changeset 4d3d0659b55e by Martin v. Löwis in branch 'default':
Issue bpo-20179: Apply Argument Clinic to bytes and bytearray.
http://hg.python.org/cpython/rev/4d3d0659b55e

[loewis]

Tal: Thanks for the patch. I applied it with updates to the current Argument Clinic.

_ssl and _overlapped are still to be done.

[larryhastings]

All the Derby patches should only go into trunk at this point.

[serhiy-storchaka]

Proposed patch converts the _ssl module to Argument Clinic. Total 39 methods converted.

[alex]

I'm concerned the _ssl changes will make security backports significantly more difficult.

[serhiy-storchaka]

_ssl.enum_certificates() and _ssl.enum_crls() is not converted because their parsing code look incorrect (bpo-23875).

[serhiy-storchaka]

The patch updated to the tip.

[python-dev]

New changeset 1eaaf27b3956 by Serhiy Storchaka in branch 'default':
Issue bpo-20179: Converted the _ssl module to Argument Clinic.
https://hg.python.org/cpython/rev/1eaaf27b3956

[tiran]

I'm with Alex. The move to argument clinic is going to make it very hard to port patches to older versions of Python, especially Python 2.7.

[serhiy-storchaka]

Sorry. Should I rollback 1eaaf27b3956?

[pitrou]

I have no opinion on whether this is a good thing or not. Security backports should be few and far between, so I don't think it's a big problem if they are a bit more difficult.

[larryhastings]

How often are patches backported to 2.7? While I understand the sentiment, I'd like to understand the scale of the objection being raised. I suspect it's infrequent, making the objection a minor one.

[tiran]

You are right, patches are seldomly backported to Python 2.7. Features are never backported. Well, except for one exception: the ssl module. :)

[dstufft]

I think it's worthwhile to maintain the ability to easily backport patches from 3.x to 2.7, especially given the security sensitive nature of the ssl module.

[larryhastings]

We discussed it in IRC a bit (and I got a little education). I can propose three remedies:

A) back out the Clinic conversion in _ssl.c
B) support Clinic in 2.7 just for _ssl.c
C) do a one-time backport of the Clinic generated code for _ssl.c

IMO these are in my reverse order of preference; I'd prefer C, then B, then A. But I'm not supporting _ssl.c, you guys are. And I want you maintainers to be happy. So I'll abide by / help you in implementing whatever you prefer.

FWIW, option C would mean doing an otherwise-inert backport of the current _ssl.c from trunk to 2.7, so that we could also backport (by hand) Modules/clinic/_ssl.c. Then, if in the future, when you change _ssl.c you still copy that file over and tweak it. But: if you change the arguments to a function, you'd have to *hand-edit* Modules/clinic/_ssl.c to match.

[pitrou]

Le 03/05/2015 23:06, Larry Hastings a écrit :

Larry Hastings added the comment:

We discussed it in IRC a bit (and I got a little education).

Can we have a transcript somewhere?

[larryhastings]

Attached. Glad you asked right away, it would have been a lot harder to get later!

[pitrou]

Thank you!

[pitrou]

I can propose three remedies:

A) back out the Clinic conversion in _ssl.c
B) support Clinic in 2.7 just for _ssl.c
C) do a one-time backport of the Clinic generated code for _ssl.c

I'd rather have A or C than B.
By the way, this discussion seems to focus on 2.7, but the same issue happens with 3.4 (although Clinic already exists here, so we can just backport Serhiy's work).

[larryhastings]

Clinic's syntax is diverging from what shipped with 3.4. So if you copied _ssl.c over, it wouldn't work with the Clinic that shipped with 3.4.

Maybe the best thing is if Clinic in trunk supports "legacy mode", where the code it generates is compatible with previous Python versions. That's basically B but without doing something crazy like shipping Clinic with 2.7.

[taleinat]

Can someone clarify whether Modules/overlapped.c should be converted to use AC?

[ZackerySpytz]

PR 14275 converts Modules/overlapped.c. Many methods now use METH_FASTCALL.

[serhiy-storchaka]

New changeset 9650fe0 by Zackery Spytz in branch 'master':
bpo-20179: Convert the _overlapped module to the Argument Clinic (GH-14275)
9650fe0