AC: Accept None as a Py_ssize_t default value

[JulienPalard]

BPO	28933
Nosy	@larryhastings, @serhiy-storchaka, @JulienPalard
Files	issue28933.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-12-13.13:30:22.014>
created_at = <Date 2016-12-10.17:12:56.847>
labels = ['3.7', 'expert-argument-clinic']
title = 'AC: Accept None as a Py_ssize_t default value'
updated_at = <Date 2016-12-13.13:37:19.817>
user = 'https://github.com/JulienPalard'

bugs.python.org fields:

activity = <Date 2016-12-13.13:37:19.817>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2016-12-13.13:30:22.014>
closer = 'mdk'
components = ['Argument Clinic']
creation = <Date 2016-12-10.17:12:56.847>
creator = 'mdk'
dependencies = []
files = ['45838']
hgrepos = []
issue_num = 28933
keywords = ['patch']
message_count = 13.0
messages = ['282861', '282862', '282873', '282874', '282903', '282904', '283042', '283043', '283049', '283056', '283060', '283076', '283098']
nosy_count = 3.0
nosy_names = ['larry', 'serhiy.storchaka', 'mdk']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue28933'
versions = ['Python 3.7']

[JulienPalard]

Today, writing an AC declaration like:

something: Py_ssize_t(c_default="-1") = None

Leads to the almost obvious "Py_ssize_t_converter: default value None for field something is not of type int".

But it actually make sense:

• Accept None as a default python value
• Document "something=None" in the docstring
• Write Py_ssize_t something = -1 in the C code
• Don't try to parse the argument if it's the default value, keeping the value from the C initialization

In other words, it's a "Give -1 to the C implementation when argument is not given or None, and it may be usefull, typically I'll use it in bpo-28754.

[JulienPalard]

Proposed a patch, but I'm not a huge fan of modifying getargs.c. If it's accepted, I'll obviously need to write tests before this is merged.

[larryhastings]

You propose an automatic conversion of "None" into "-1"? That's awful. Please don't commit that patch to CPython.

[larryhastings]

It looks like you did it with a converter for 28754. That's okay. But not in the default implementation.

[JulienPalard]

It looks like you did it with a converter for 28754. That's okay. But not in the default implementation.

It's not by default, we have to declare "… = None" in the AC declaration, which was an error before my patch (incompatible types…).

Before:

something: Py_ssize_t(c_default="-1") = None

> Py_ssize_t_converter: default value None for field something is not of type int

After:

something: Py_ssize_t(c_default="-1") = None

> If the None (default) value is provided, the c_default is untouched.

[JulienPalard]

You propose an automatic conversion of "None" into "-1"? That's awful. Please don't commit that patch to CPython.

Not really, I propose a way to do it with AC when needed. And the AC semantics introduced are not "Automatic conversion of None into -1" but "automatic "don't touch the C default" when the python default is given", so it's reusable for other values:

something: Py_ssize_t(c_default="-1") = None
something: Py_ssize_t(c_default="0") = None
something: Py_ssize_t(c_default="42") = None
…

And this semantic can be extended to other types too if needed:

something: a_type(c_default="a_c_side_default_value") = None

To get "a_c_side_default_value" when None is given.

I however did not introduced a way to keep the C default value by using something else than None in the Python side:

• I'm not sure this is usefull to allow it but it can still be implemented if needed
• Limiting to None permits to keep a type check so something: Py_ssize_t(c_default="-1") = "bar" will still fail at clinic.py runtime.

[JulienPalard]

Hi Larry,

Did you take more time to review this patch and my last comments? I don't think it that awful as it does _not_ apply until explicitly asked for, but I'm open to discuss it.

[serhiy-storchaka]

Agreed with Larry. "Special cases aren't special enough to break the rules." General converter shouldn't be changed for just one use case. Write your own special converter. Or just use the "O" converter and manually convert Python object to C value. That can be simpler and more readable.

[JulienPalard]

for just one use case

I don't think that using None in a default argument is "one use case", nor a "special case" it's more like a "widly used pattern" that I'd like to make simple to implement (also see http://bugs.python.org/issue28754#msg282891).

I'm not sure you're against my idea of accepting None like:

something: Py_ssize_t(c_default="-1") = None

Or you're against my implementation, which I can understand, I'm not a fan of modifying getargs.c for this, it may be doable by implementing a new converter in clinic.py like "Py_ssize_t_optional", and it may not require more work than modifying getargs.c.

But AC is already defining 4 different "default values", counting the "= value" from the AC DSL, I'm not sure that adding the "optional" string somewhere make any of this clearer, typically what about the semantic difference between:

something: Py_ssize_t_optional()

and

something: Py_ssize_t() = None

So my original idea was to accept "= None" indistinctly of the used type, I mean allowing:

something: Py_ssize_t(c_default=…) = None
something: float(c_default=…) = None
something: double(c_default=…) = None
…
…

Which is, I think, one of the clearest way to express "It's an optional parameter, on the Python side the default value is None, but c-side the default value is of the right type an is …".

In other words, I'm proposing to optionally extend the "c_default" usage from "not given" to "not given or given None".

I failed to implement if for any types, mainly because AC is delegating to getargs for Py_ssize_t, so I though starting with my specific need (Py_ssize_t for bpo-28754) was a good start and POC.

Another solution may be to change the Py_ssize_t_converter to drop the format_unit, allowing us to implement the converter in clinic instead of getargs.c. It's opening a long term door of completly dropping non-"O" format units. I mean, if we drop every non-"O" format units in clinic we'll be able to use a faster implementation of _PyArg_ParseStack, one not parsing a mini-arg-definition-language at runtime (the "OO|nO&:").

I'm willing to implement something clean for this but please be more precise in your feedback.

[serhiy-storchaka]

First, it looks weird if the default value affects accepted types. Second, how many use cases for your converter besides the bisect module?

[larryhastings]

I don't want this change committed to CPython, you can do what you need with a converter so do that.

[JulienPalard]

Hi Larry,

Do you mean a new converter in clinic.py like Nullable_Py_ssizze_t, or a converter that I copy/paste every time I need it like http://bugs.python.org/review/28754/patch/19417/76440 ?