Speedup types.coroutine()

[1st1]

BPO	24325
Nosy	@gvanrossum, @ncoghlan, @scoder, @vstinner, @larryhastings, @asvetlov, @markshannon, @vadmium, @1st1
Dependencies	bpo-24400: Awaitable ABC incompatible with functools.singledispatch
Files	types_coroutine_speedup.patch types_coroutine_speedup.patch types_coroutine_speedup.patch: more tests; rebase onto 24400; make GeneratorWrapper a GC type types_coroutine_speedup.patch: rebased; more tests types_coroutine_speedup.patch: new unittests have been committed; the patch now contains only the acceleration part

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/1st1'
closed_at = <Date 2018-01-29.20:10:43.091>
created_at = <Date 2015-05-29.16:23:27.316>
labels = ['type-feature', 'library']
title = 'Speedup types.coroutine()'
updated_at = <Date 2018-01-29.20:10:43.089>
user = 'https://github.com/1st1'

bugs.python.org fields:

activity = <Date 2018-01-29.20:10:43.089>
actor = 'yselivanov'
assignee = 'yselivanov'
closed = True
closed_date = <Date 2018-01-29.20:10:43.091>
closer = 'yselivanov'
components = ['Library (Lib)']
creation = <Date 2015-05-29.16:23:27.316>
creator = 'yselivanov'
dependencies = ['24400']
files = ['39547', '39563', '39754', '39774', '39803']
hgrepos = []
issue_num = 24325
keywords = ['patch']
message_count = 20.0
messages = ['244388', '244418', '244447', '244451', '244490', '244504', '245575', '245581', '245655', '245756', '245761', '245764', '246117', '246118', '246123', '246139', '246142', '246148', '246330', '311162']
nosy_count = 11.0
nosy_names = ['gvanrossum', 'ncoghlan', 'scoder', 'vstinner', 'larry', 'Arfrever', 'asvetlov', 'Mark.Shannon', 'python-dev', 'martin.panter', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue24325'
versions = ['Python 3.5', 'Python 3.6']

[1st1]

Attached patch provides an implementation (part of it) of types.coroutine in C.

The problem with the current pure Python implementation is that it copies the code object of the generator function, which is a small overhead during import.

I'm not sure if this should be merged in 3.5 at all. Please take a look at the patch.

[1st1]

Attached is the second iteration of the patch. Now, besides just speeding up types.coroutine() for pure python generator functions, it also provides a better wrapper around generator-like objects.

[larryhastings]

This looks big and complicated. I'd prefer this skipped 3.5 and just went into 3.6.

[scoder]

I would still like to see a patch in Py3.5 that only flips the bit in C when "_types" is available and otherwise falls back to the existing Python code that creates a new code object. This part is much cleaner and faster to do in C than in the current Python implementation.

[1st1]

Larry, can you accept the first version of the patch (only function that patches code object's co_flags) after beta2?

I'm OK if you think it should only be committed in 3.6, but I also agree with Stefan, that using C is better in this particular case.

[gvanrossum]

Since it's a speedup it could also go into 3.5.1.
On May 30, 2015 1:22 PM, "Yury Selivanov" <report@bugs.python.org> wrote:

Yury Selivanov added the comment:

Larry, can you accept the first version of the patch (only function that
patches code object's co_flags) after beta2?

I'm OK if you think it should only be committed in 3.6, but I also agree
with Stefan, that using C is better in this particular case.

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue24325\>

---

[1st1]

Larry, Nick,

Looking at how bpo-24400 progresses, I think it would be really great if we can merge this one in 3.5.0. It also makes it unnecessary to merge bpo-24468 in 3.5.

[ncoghlan]

+1 from me for merging this for 3.5.0 and deferring bpo-24468 (which now proposes making _opcode a builtin module to allow compiler constants to be easily shared between C code and Python code) to 3.6 instead.

The design changes to address bpo-24400 cleaned up various aspect of both the internal architecture and the public data model of PEP-492, and the latest draft of this patch benefits accordingly.

[1st1]

A rebased version of the patch is attached (now a "review" link should appear).

Nick, Stefan, please take a look.

Larry, can we merge this in 3.5.0? I've invested a lot of time to have 100% test coverage; the test suite is very elaborate now. There seems to be no refleaks too. I think this change is a very natural continuation of what we've done in bpo-24400.

[python-dev]

New changeset eb6fb8e2f995 by Yury Selivanov in branch '3.5':
Issue bpo-24325, bpo-24400: Add more unittests for types.coroutine; tweak wrapper implementation.
https://hg.python.org/cpython/rev/eb6fb8e2f995

New changeset 7a2a79362bbe by Yury Selivanov in branch 'default':
Merge 3.5 (Issue bpo-24325, bpo-24400)
https://hg.python.org/cpython/rev/7a2a79362bbe

[python-dev]

New changeset 9aee273bf8b7 by Yury Selivanov in branch '3.5':
Issue bpo-24400, bpo-24325: More tests for types._GeneratorWrapper
https://hg.python.org/cpython/rev/9aee273bf8b7

New changeset fa097a336079 by Yury Selivanov in branch 'default':
Merge 3.5 (issue bpo-24325 & bpo-24400)
https://hg.python.org/cpython/rev/fa097a336079

[1st1]

I've committed new unittests from this patch (as they are applicable to pure Python implementation of the wrapper too)

The patch now contains only the C implementation of the wrapper and should be ready to be committed. Larry?