PEP 342 Generator enhancements

[pjeby]

BPO	1223381
Nosy	@mwhudson, @gvanrossum, @birkenfeld, @pjeby
Files	yield.patch: PEP 342/343 generator enhancements yield2.diff: mwh's fixed version

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/pjeby'
closed_at = <Date 2005-08-02.00:48:55.000>
created_at = <Date 2005-06-18.23:08:41.000>
labels = ['interpreter-core']
title = 'PEP 342 Generator enhancements'
updated_at = <Date 2005-08-02.00:48:55.000>
user = 'https://github.com/pjeby'

bugs.python.org fields:

activity = <Date 2005-08-02.00:48:55.000>
actor = 'pje'
assignee = 'pje'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2005-06-18.23:08:41.000>
creator = 'pje'
dependencies = []
files = ['6697', '6698']
hgrepos = []
issue_num = 1223381
keywords = ['patch']
message_count = 13.0
messages = ['48478', '48479', '48480', '48481', '48482', '48483', '48484', '48485', '48486', '48487', '48488', '48489', '48490']
nosy_count = 4.0
nosy_names = ['mwh', 'gvanrossum', 'georg.brandl', 'pje']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1223381'
versions = ['Python 2.5']

[pjeby]

The attached patch implements code and test changes for:

• yield expressions
• bare yield (short for yield None)
• yield in try/finally
• generator.send(value) (send value into generator;
  substituted for PEP-342's next(arg))
• generator.throw(typ[,val[,tb]]) (raise error in
  generator)
• generator.close() (per PEP-343)
• GeneratorExit built-in exception type
• generator.__del__ (well, the C equivalent)
• All necessary mods to the compiler, parser module,
  and Python 'compiler' package to support these changes.

It was necessary to change a small part of the eval
loop (well, the initialization, not the loop) and the
gc module's has_finalizer() logic in order to support a
C equivalent to __del__. Specialists in these areas
should probably scrutinize this patch!

There is one additional implementation detail that was
not contemplated in either PEP. in order to prevent
used-up generators from retaining unnecessary
references to their frame's contents, I set the
generator's gi_frame member to None whenever the
generator finishes normally or with an error. Thus, an
exhausted generator cannot be part of a cycle, and it
releases its frame object sooner than in previous
Python versions. For generators used only in a direct
"for" loop, this makes no difference, but for
generators used with the iterator protocol (i.e.
"gen.next()") from Python, this avoids stranding the
generator's frame in a traceback cycle.

[birkenfeld]

Logged In: YES
user_id=1188172

Minor nit: on line 642, 'yield_stmt' should be changed to
'yield_expr'.

[mwhudson]

Logged In: YES
user_id=6656

Argh, I'd hoped you'd done with, too :)

Have you done any work on that yet?

[pjeby]

Logged In: YES
user_id=56214

No, I haven't, and I never really planned to be the person
implementing "with". Doesn't mean I might not end up doing
it eventually, but it's not currently in my plans. I'm
going to have my hands full just updating PEP-342 (Guido has
asked me to take it over), and getting docs written to go
with this patch. Between all that and my distutils-related
work, my pro-bono Python project time is booked up solid for
maybe the next 2 months or so.

[mwhudson]

Logged In: YES
user_id=6656

Fair enough. I want to use with statements in a presentation at
EuroPython next week, so I've hacked together a truly horrible
implementation that will do for now and will see about a real one some time
after...

[gvanrossum]

Logged In: YES
user_id=6380

(Removing 343 from the subject since all this is now in PEP-342.)

Playing with the code a bit, I notice that throw() doesn't
like string exceptions. That could cause some problems.

Also, I think the tb argument should be allowed to be None.

Otherwise, great! In a few hours you can check it in. :-)

[mwhudson]

Logged In: YES
user_id=6656

Heh heh heh.

[pjeby]

Logged In: YES
user_id=56214

I had hoped to be able to avoid dealing with string
exceptions, but I guess it makes sense that if somebody has
thrown one, it should be able to propagate. So I guess I'll
take care of that along with the rest.

[gvanrossum]

Logged In: YES
user_id=6380

Since I've accepted PEP-342+343 last night in my Europython
keynote, you can check this in now.

[mwhudson]

Logged In: YES
user_id=6656

You missed an incref in gen_throw (symptom was

Fatal Python error: ../Objects/tupleobject.c:169 object at
0x4013807c has negative ref count -606348326

on exit after running test_generators). New patch attached.

[pjeby]

Logged In: YES
user_id=56214

mwh, is the only difference in your patch the addition of
the incref?

[mwhudson]

Logged In: YES
user_id=6656

I think so, yes :)