Support more than 255 arguments

[andersk]

BPO	12844
Nosy	@loewis, @brettcannon, @rhettinger, @Trundle, @andersk, @serhiy-storchaka
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	issue12844_arbitrary_arguments.diff no-args-limit.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-11-28.10:25:17.437>
created_at = <Date 2011-08-26.02:42:57.843>
labels = ['interpreter-core', 'type-feature', '3.7']
title = 'Support more than 255 arguments'
updated_at = <Date 2017-03-31.16:36:20.543>
user = 'https://github.com/andersk'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:20.543>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-11-28.10:25:17.437>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2011-08-26.02:42:57.843>
creator = 'andersk'
dependencies = []
files = ['23089', '45638']
hgrepos = []
issue_num = 12844
keywords = ['patch']
message_count = 11.0
messages = ['142995', '142996', '142998', '143002', '143440', '143451', '281655', '281688', '281836', '281848', '281855']
nosy_count = 9.0
nosy_names = ['loewis', 'brett.cannon', 'rhettinger', 'sebastinas', 'Trundle', 'andersk', 'davidben', 'python-dev', 'serhiy.storchaka']
pr_nums = ['552']
priority = 'low'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue12844'
versions = ['Python 3.7']

[andersk]

This feels like an arbitrary restriction (obvious sequences have been replaced with ‘…’ to save space in this report):

>>> zip([0], [1], [2], …, [1999])
  File "<stdin>", line 1
SyntaxError: more than 255 arguments

especially when this works:

>>> zip(*[[0], [1], [2], …, [1999]])
[(0, 1, 2, …, 1999)]

Apparently that limit bites some people:
https://docs.djangoproject.com/en/1.3/topics/http/urls/#module-django.conf.urls.defaults

The bytecode format doesn’t support directly calling a function with more than 255 arguments. But, it should still be pretty easy to compile such function calls by desugaring
f(arg0, …, arg999, k0=v0, …, k999=v999)
into
f(*(arg0, …, arg999), **{'k0': 'v0', …, 'k999': 'v999'})

[andersk]

I guess the desugaring is slightly more complicated in the case where the original function call already used *args or **kwargs:
f(arg0, …, arg999, *args, k0=v0, …, k999=v999, **kwargs)
becomes something like
f(*((arg0, …, arg999) + args),
**dict({'k0': 'v0', …, 'k999': 'v999'}, **kwargs))

[rhettinger]

On python-dev a few month ago, Guido agreed with you that this is an arbitrary limitation that should be removed at some point. In particular, he worried that programmatically generated code would tend to run into this limitation.

[loewis]

The approach looks fine to me. Would you like to work on a patch?

[Trundle]

Attached is a patch that removes the limit and that allows passing an arbitrary number of positional and keyword arguments. Lacks tests for now.

[davidben]

I don't think that patch works. Consider a dict subclass which has overridden update. Or perhaps a list subclass which has overridden addition. It would be quite poor if Python's behavior here w.r.t. which overrides are followed switched as you added more arguments.

[serhiy-storchaka]

Since bpo-27213 the bytecode no longer have a limitation for numbers of positional or keyword arguments.

[serhiy-storchaka]

No longer changes to Python/compile.c are needed. Here is a patch against 3.7 that just removes the limit of the number of passed arguments in Python/ast.c and adds tests.

But still there is a limitation on the number of function parameters. It is caused by using a bytes objects as co_cell2arg.

[brettcannon]

Patch LGTM.

[python-dev]

New changeset 5c1bb72c0f5d by Serhiy Storchaka in branch 'default':
Issue bpo-12844: More than 255 arguments can now be passed to a function.
https://hg.python.org/cpython/rev/5c1bb72c0f5d

[serhiy-storchaka]

Thanks Brett.

See bpo-18896 for supporting functions with more than 255 parameters.