ast.arguments has confusing args/posonlyargs ordering

[BenjaminSWolf]

BPO	37593
Nosy	@serhiy-storchaka, @ilevkivskyi, @pablogsal, @miss-islington, @jhdulaney
PRs	bpo-37593: Swap the positions of posonlyargs and args in the constructor of ast.parameters nodes #14778 [3.8] bpo-37593: Swap the positions of posonlyargs and args in the constructor of ast.parameters nodes (GH-14778) #14779

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-07-15.00:08:34.720>
created_at = <Date 2019-07-14.18:55:37.123>
labels = ['type-bug', '3.8']
title = 'ast.arguments has confusing args/posonlyargs ordering'
updated_at = <Date 2019-08-16.19:56:37.496>
user = 'https://bugs.python.org/BenjaminSWolf'

bugs.python.org fields:

activity = <Date 2019-08-16.19:56:37.496>
actor = 'mdulaney'
assignee = 'none'
closed = True
closed_date = <Date 2019-07-15.00:08:34.720>
closer = 'pablogsal'
components = []
creation = <Date 2019-07-14.18:55:37.123>
creator = 'Benjamin.S.Wolf'
dependencies = []
files = []
hgrepos = []
issue_num = 37593
keywords = ['patch']
message_count = 5.0
messages = ['347932', '347945', '347946', '347947', '349883']
nosy_count = 6.0
nosy_names = ['serhiy.storchaka', 'Benjamin.S.Wolf', 'levkivskyi', 'pablogsal', 'miss-islington', 'mdulaney']
pr_nums = ['14778', '14779']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue37593'
versions = ['Python 3.8']

[BenjaminSWolf]

Positional-only arguments come before position-or-keyword arguments.

    def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):

However, the posonlyargs are defined to come after args in the AST:

    arguments = (arg* args, arg* posonlyargs, arg? vararg, arg* kwonlyargs,
                 expr* kw_defaults, arg? kwarg, expr* defaults)

which results in confusing ast.dump output because they share defaults:

>>> r = ast.parse('lambda a=1,/,b=2:a+b', mode='eval')
>>> ast.dump(r.body.args)
"arguments(
    args=[arg(arg='b', annotation=None, type_comment=None)],
    posonlyargs=[arg(arg='a', annotation=None, type_comment=None)],
    vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None,
    defaults=[Constant(value=1, kind=None), Constant(value=2, kind=None)])"
[manually prettified]

Note how the ordering is 'args b', then 'posonlyargs a', but the defaults are still 1 then 2. This can be confusing to someone building an ast.arguments using keywords because the elements in 'defaults' have to be supplied in a specific order, but the keyword args 'args' and 'posonlyargs' do not, or to someone building an ast.arguments using positional arguments (because, maybe ironically, they're not keyword-only arguments) because 'posonlyargs' and 'args' must be supplied in a different order than the ordering of elements in 'defaults' would imply.

Potential solutions:

1. Swap posonlyargs and args.
2. Add a separate pos_defaults list.

[miss-islington]

New changeset cd6e83b by Miss Islington (bot) (Pablo Galindo) in branch 'master':
bpo-37593: Swap the positions of posonlyargs and args in the constructor of ast.parameters nodes (GH-14778)
cd6e83b

[miss-islington]

New changeset cf9a63c by Miss Islington (bot) in branch '3.8':
bpo-37593: Swap the positions of posonlyargs and args in the constructor of ast.parameters nodes (GH-14778)
cf9a63c

[pablogsal]

Thanks for the report, Benjamin!

[jhdulaney]

The patch that was merged to fix this may have caused a regresion in xonsh. Am working on a reproducer.