PEP 380 reference implementation for 3.3

[ncoghlan]

BPO	11682
Nosy	@rhettinger, @ncoghlan, @mitsuhiko, @merwok, @alex, @asvetlov
Files	f8349cbc1b26.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ncoghlan'
closed_at = <Date 2012-01-13.12:01:19.860>
created_at = <Date 2011-03-26.03:36:45.133>
labels = ['interpreter-core', 'type-feature']
title = 'PEP 380 reference implementation for 3.3'
updated_at = <Date 2012-01-27.11:17:38.832>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2012-01-27.11:17:38.832>
actor = 'ncoghlan'
assignee = 'ncoghlan'
closed = True
closed_date = <Date 2012-01-13.12:01:19.860>
closer = 'ncoghlan'
components = ['Interpreter Core']
creation = <Date 2011-03-26.03:36:45.133>
creator = 'ncoghlan'
dependencies = []
files = ['24214']
hgrepos = ['40', '107']
issue_num = 11682
keywords = ['patch']
message_count = 35.0
messages = ['132213', '134550', '134556', '134925', '139145', '139471', '139529', '140072', '140110', '140115', '143210', '143492', '143493', '143510', '144325', '144350', '144378', '145144', '146695', '147320', '147347', '147348', '147404', '148153', '148163', '148950', '148959', '148963', '148964', '148977', '151125', '151144', '151171', '151177', '152085']
nosy_count = 12.0
nosy_names = ['rhettinger', 'gcewing', 'ncoghlan', 'rndblnch', 'ron_adam', 'aronacher', 'eric.araujo', 'alex', 'rfk', 'asvetlov', 'zbysz', 'Yury.Selivanov']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue11682'
versions = ['Python 3.3']

[ncoghlan]

Creating an issue to track efforts to forward port the PEP-380 reference implementation to Python 3.3.

[merwok]

(“Create patch” does not work for some reason, I’ll report that to the metatracker in a few days if nobody does it first)

[ncoghlan]

It's because it isn't a true Mercurial clone-and-update - it's a patch queue, which the review generation code doesn't know how to interpret.

[rndblnch]

As nick said, the repo only host a patch queue.
the patch itself is visible here:
https://bitbucket.org/rndblnch/cpython-pep380/qseries?apply=t&qs_apply=pep380

Or it can be download here in raw text:
https://bitbucket.org/rndblnch/cpython-pep380/raw/tip/pep380

I have documented how to apply the patch to a cpython clone here:
https://bitbucket.org/rndblnch/cpython-pep380/wiki

[ncoghlan]

OK, this isn't going to be as simple as I thought it was. The big problem is that the test suite from [1] needs to be refactored into a format that is suitable for use in regrtest. That means porting them to either unittest or doctest, as "golden output" tests of the style that Greg used are no longer accepted as part of the regression test suite.

[1] http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/yield_from.html

[ncoghlan]

Renaud has updated the patch on bitbucket to incorporate Greg's tests (still in golden output form, but in a unittest/regrtest compatible way). That's a good enough starting point for me - they can be refactored into proper assert based tests later.

The only tweak needed is to explicitly save and restore stdout and stderr, since restoring the OS level ones often does the wrong thing when running the regression test suite.

[rndblnch]

I've just updated the PEP-380-test patch to make it slightly simpler (still in golden output form though)

https://bitbucket.org/rndblnch/cpython-pep380/src/317eadf5e3e8/pep380-tests

[ncoghlan]

Once again got close to committing this, but then realised it is missing all the necessary documentation updates for a core language change.

I have uploaded a patch that includes all the changes from Bitbucket as well as the subsequent fixes to avoid modifying sys.stdout when the tests are run and to comply with the whitespace rules set up in the source control hooks (the ACKS, NEWS and a placeholder in whatsnew are also included).

What's missing are updates to at least:

http://docs.python.org/dev/tutorial/classes.html#generators
(a simple example showing delegation should suffice there)

http://docs.python.org/dev/reference/simple_stmts.html#the-yield-statement
http://docs.python.org/dev/reference/expressions.html#grammar-token-yield_expression
http://docs.python.org/dev/reference/simple_stmts.html#the-return-statement
(the language reference must be updated for a post PEP-380 world)

There are likely other places that should also be updated, but these are the critical ones needed before the patch can be included.

[rndblnch]

I can not comment on http://bugs.python.org/review/11682/, so a quick comment here: Doc/whatsnew/3.3.rst patch gives me credit together with Greg Ewing for the implementation, but I've only upgraded his patches to 3.3.
So, the following line:
+(Implementation by Greg Ewing and Renauld Blanch)
should really be:
+(Implementation by Greg Ewing)

And, as I'm on it, the proper spelling for my surname is Renaud (not Renauld as it also appears in the commit message :)

[ncoghlan]

I moved my personal sandbox from hg.python.org to bitbucket, so it should be easier for folks to build off the work in progress. (And fixed the typo in Renaud's name - I at least had it right in ACKS. I also reworded the draft attribution text in What's New)

I skimmed Benjamin's comments and basically agree with them. I've marked bpo-11816 as a dependency as some of the new tests will be easier to refactor once dis.get_opinfo() is available.

[ncoghlan]

The PEP-380 branch in my bitbucket repo has been updated with the refactored tests that Ryan Kelly put together at the PyconAU sprints (as well as being brought up to speed with 3.x tip).

The update depends on the new dis.get_opinfo() API added by issue bpo-11816 (which is currently with Raymond for review).

The AST validator still needs to be updated to take into account the new syntax and the patch attribution needs to be updated to account for Ryan's contribution, but this is getting fairly close to being ready.

[ncoghlan]

Uploaded a patch that should be complete. Note that my PEP-380 branch is based on my get_opinfo branch (see bpo-11816), so if you're applying patches manually rather than updating directly from my sandbox with hg, you'll need to apply the latest patch from bpo-11816 before applying this one.

Raymond: assigning to you, since you said you wanted to review the dis.get_opinfo changes before they went in, and those changes are a dependency for the updated PEP-380 test suite.

[ncoghlan]

Actually, not assigning to Raymond for review yet, after all - I just noticed there are some of Benjamin's review comments relating to cosmetic details rather than functionality that I still need to address.

I'll kick it in Raymond's direction once I've dealt with those (probably next weekend, but possibly some time this week)

[zbysz]

I've created some documentation... The patches are the bitbucket repo.

Nothing is added to the tutorial, because I think that this isn't material for a newcomer to python. The tutorial doesn't mention generator.throw() and send() either, just talks a little about writing simple generator functions.