Add documentation for METH_FASTCALL and _PyObject_FastCall*()

[skrah]

BPO	28805
Nosy	@warsaw, @vstinner, @methane, @skrah, @serhiy-storchaka, @jdemeyer, @miss-islington, @isidentical
PRs	bpo-28805: document METH_FASTCALL #14079 [3.7] bpo-28805: document METH_FASTCALL (GH-14079) #14136 [3.8] bpo-28805: document METH_FASTCALL (GH-14079) #14137

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-11-26.11:08:54.537>
labels = ['type-feature', '3.7', 'docs']
title = 'Add documentation for METH_FASTCALL and _PyObject_FastCall*()'
updated_at = <Date 2019-12-01.09:13:41.667>
user = 'https://github.com/skrah'

bugs.python.org fields:

activity = <Date 2019-12-01.09:13:41.667>
actor = 'BTaskaya'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2016-11-26.11:08:54.537>
creator = 'skrah'
dependencies = []
files = []
hgrepos = []
issue_num = 28805
keywords = ['patch']
message_count = 14.0
messages = ['281766', '281771', '281778', '281784', '281785', '303701', '303702', '303703', '303704', '345577', '345762', '345789', '345790', '357673']
nosy_count = 9.0
nosy_names = ['barry', 'vstinner', 'methane', 'skrah', 'docs@python', 'serhiy.storchaka', 'jdemeyer', 'miss-islington', 'BTaskaya']
pr_nums = ['14079', '14136', '14137']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28805'
versions = ['Python 3.6', 'Python 3.7']

[skrah]

It looks like METH_FASTCALL gives nice speedups (bpo-28754).

Is this an internal interface or can it be documented like
METH_KEYWORDS etc.?

[serhiy-storchaka]

I suggest to keep it internal. It's meaning may be changed in future. I have some ideas about making it more convenient and faster (but I'm not sure that they would work).

[vstinner]

Is it possible to use Argument Clinic for third party extensions? If yes, I
suggest to use it.

It would also be nice if Cython could use it automatically. So we can
suggest to use Cython. I don't know well Cython, maybe your idea is very
dumb :-)

Serhiy Storchaka added the comment:

I suggest to keep it internal. It's meaning may be changed in future. I
have some ideas about making it more convenient and faster (but I'm not
sure that they would work).

Ah? Can you please elaborate your secret plan? :-)

[skrah]

There are a couple of problems with using Argument Clinic for third
party projects.

First, it makes no stability promises:

"Currently Argument Clinic is considered internal-only for CPython. ..."

Then, for large projects that already use some generated code (like
NumPy), IMO the result would be unmaintainable.

So it would be nice to have a public stable API -- of course there
is no hurry if Serhiy has plans to improve it.

[serhiy-storchaka]

Ah? Can you please elaborate your secret plan? :-)

I already said about this. Move the part of parsing keyword argument outside of the function. The caller should unpack keyword arguments and pass just a raw array of PyObject*. Missed arguments are set to NULL. This could make _PyArg_ParseStack() simpler and might even allow to inline arguments parsing code.

[warsaw]

This and the _PyObject_FastCall* APIs really do need to be documented, especially if they're going to be used in the interpreter itself. The leading underscore signifies that it's not part of the public API (well, METH_FASTCALL doesn't have a leading underscore).

When these are not documented, then they aren't discoverable, so there are cases where it may be useful but will be overlooked. It also means that folks will be inclined to cargo cult the few existing uses without perhaps understanding the full semantics.

Can we please document these for Python 3.7 at least?

[vstinner]

Can we please document these for Python 3.7 at least?

I chose to not document FASTCALL on purpose in Python 3.6, I wanted to keep everything private, until we get enough feedback and stabilize the API.

Recently, the FASTCALL API changed: METH_FASTCALL doesn't support keywords anymore. You have to pass METH_FASTCALL|METH_KEYWORDS.

It seems like the API became stable.

I don't know if we should make the API public or not. PyPy would complain as usual that we give them more work to do :-D

[vstinner]

I suggest to document the following 4 functions/macros:

PyAPI_FUNC(PyObject *) _PyObject_FastCallDict(
    PyObject *callable,
    PyObject **args,
    Py_ssize_t nargs,
    PyObject *kwargs);

PyAPI_FUNC(PyObject *) _PyObject_FastCallKeywords(
    PyObject *callable,
    PyObject **args,
    Py_ssize_t nargs,
    PyObject *kwnames);

#define _PyObject_FastCall(func, args, nargs) \
    _PyObject_FastCallDict((func), (args), (nargs), NULL)

#define _PyObject_CallNoArg(func) \
    _PyObject_FastCallDict((func), NULL, 0, NULL)

And the METH_FASTCALL and METH_FASTCALL|METH_KEYWORDS calling convention.

[vstinner]

It would also be nice if Cython could use it automatically.

Cython is using FASTCALL since Python 3.6.

[jdemeyer]

Victor, of the four functions you mentioned:

• _PyObject_FastCallDict: documented by PEP-590

• _PyObject_FastCallKeywords: renamed _PyObject_Vectorcall and documented by PEP-590

• _PyObject_FastCall: not sure if it's useful enough (just use _PyObject_Vectorcall with kwnames=NULL)

• _PyObject_CallNoArg: see bpo-37194

So that leaves documenting METH_FASTCALL.

[methane]

New changeset 5600b5e by Inada Naoki (Jeroen Demeyer) in branch 'master':
bpo-28805: document METH_FASTCALL (GH-14079)
5600b5e

[miss-islington]

New changeset b101fa7 by Miss Islington (bot) in branch '3.7':
bpo-28805: document METH_FASTCALL (GH-14079)
b101fa7

[miss-islington]

New changeset e784f9f by Miss Islington (bot) in branch '3.8':
bpo-28805: document METH_FASTCALL (GH-14079)
e784f9f

[isidentical]

PR 14079 has been merged, is there anything left as unresolved about this issue?

[hauntsaninja]

Thanks, I think we can close this docs issue as completed