Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pep 572 update #654

Merged
merged 17 commits into from May 22, 2018

Conversation

@gvanrossum
Copy link
Member

commented May 16, 2018

@Rosuav, @tim-one: On the plane home from PyCon I did some major surgery on PEP 572. The tl;dr is that I removed the changes to comprehensions and added Tim's proposal for making := inside a comprehension write the target in the containing scope. There's a bit more, below are the bullets from my local commits.

- Remove changes to comprehension scope
- Make := in comprehensions assign to containing non-comprehension scope
- Clarify binding precedence (tighter than comma, not at same level as =)
- Remove mention of more complex targets in the future
- Explicitly disallow toplevel :=
- Rewrite section on differences with =, enumerating all of them
- Remove "here's how this could be written without :=" from examples

- Tweak first paragraph of "Syntax and semantics"
- Add "Exceptional cases" (various explicit prohibitions)
- Clarify that lambda is a containing scope
- Clarify that := and = just don't mix
- Added "Open questions" section
- Added two new rejected alternatives: "Allowing commas to the right"
  and "Always requiring parentheses"
- Minor edits
their values from one iteration to another. It would be convenient to use
this feature to create rolling or self-effecting data streams::
point = x, y
(point := x, y)

This comment has been minimized.

Copy link
@Rosuav

Rosuav May 16, 2018

Contributor

The outer parens don't really clarify much; perhaps point := (x, y) would be more indicative?

This comment has been minimized.

Copy link
@gvanrossum

gvanrossum May 16, 2018

Author Member

Ah, you're right. These critics want the following to be equivalent:

(point := (x, y))
(point := x, y)

The rest of this section is my refutation of that position. I'll fix it.

@Rosuav

This comment has been minimized.

Copy link
Contributor

commented May 16, 2018

Do you want discussion here or on the ML?

One point that I find curious is that top-level assignment with := is forbidden due to the similarity with =, but that you're explicitly allowed to use NAME := expr as a positional parameter to a function, which is similarly similar to keyword args. The entire justification for this is the style recommendation of whether there are spaces surrounding the = or :=. That seems a little odd - that common style conventions (even those codified into PEP 8) should govern syntactic legality.

Personally, I would prefer to remove the special cases and just say that := assignments are allowed anywhere, including top level. It's perfectly legal to write f() if cond else g() at top-level, and we don't see heaps of code using that in place of if cond: f() else: g().

The grammar of Python is already a scarily complicated beast to look into. Adopting all of these special cases will make it even more so. Are you happy with the future of Python being "anyone can tinker with the stdlib, someone who knows C can tinker with the implementation of CPython, and only a core dev with at least twenty years' experience can look at the grammar"? Okay, so that's a bit of hyperbole, but only a bit. Every change is going to add some complexity, but IMO the goal should be to add as little as possible.

WRT co-authorship: with the edits you've made here, you absolutely merit putting your own name on it. Tim Peters less obviously, but I definitely wouldn't object to him being a co-author if you and/or he feel it's appropriate.

@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented May 16, 2018

Do you want discussion here or on the ML?

Here please. The list has become too dysfunctional. I am even thinking that we may recommend to have all discussions in a dedicated tracker or issue.

One point that I find curious is that top-level assignment with := is forbidden due to the similarity with =, but that you're explicitly allowed to use NAME := expr as a positional parameter to a function, which is similarly similar to keyword args. The entire justification for this is the style recommendation of whether there are spaces surrounding the = or :=. That seems a little odd - that common style conventions (even those codified into PEP 8) should govern syntactic legality.

The justification for disallowing top level := is mostly political -- some people are very concerned about whether := will become the new spelling for =, and I don't want to encourage that worry. The same concern doesn't apply to keyword arguments.

Personally, I would prefer to remove the special cases and just say that := assignments are allowed anywhere, including top level. It's perfectly legal to write f() if cond else g() at top-level, and we don't see heaps of code using that in place of if cond: f() else: g().

Yeah, but as I said that would weaken support for the proposal.

The grammar of Python is already a scarily complicated beast to look into. Adopting all of these special cases will make it even more so. Are you happy with the future of Python being "anyone can tinker with the stdlib, someone who knows C can tinker with the implementation of CPython, and only a core dev with at least twenty years' experience can look at the grammar"? Okay, so that's a bit of hyperbole, but only a bit. Every change is going to add some complexity, but IMO the goal should be to add as little as possible.

Most of these special cases will have to checked in the second stage.

WRT co-authorship: with the edits you've made here, you absolutely merit putting your own name on it. Tim Peters less obviously, but I definitely wouldn't object to him being a co-author if you and/or he feel it's appropriate.

But if I'm co-author it's less clear that I can also approve (judge and jury and so on). Tim contributed the special case for := in comprehensions, a lot of finer points, and the ideas that real code is important.

@Rosuav

This comment has been minimized.

Copy link
Contributor

commented May 16, 2018

The justification for disallowing top level := is mostly political
... Yeah, but as I said that would weaken support for the proposal.

Fair enough.

But if I'm co-author it's less clear that I can also approve (judge and jury and so on).

Good point. I'll leave it up to your discretion; at this point, it's another political decision really. You have the power to decide, regardless of authorship status; and you absolutely have merited it, by any logical measurement. Deciding to forego that authorship is your call.

Tim as co-author is fine in my view.

@tim-one

This comment has been minimized.

Copy link
Member

commented May 16, 2018

I should clarify some things: I don't have git. I don't know how to use git. I barely know enough about github to enter this comment. And I don't have a C compiler (my MSDN subscription "got lost" somehow when they migrated accounts on their side, and I gave up after some hours of trying to get that straightened out). That's why all I do now is type at mailing lists ;-) Presumably all of that could be repaired with reasonable effort, but there are so many other things I like doing that's not likley to rise to the top of my list soon.

That's why, e.g., when I suggested specific changes for the Reference Manual, I posted the new text rather than generate a patch. It's about all I can do for now.

Since I want the PEP to succeed, whether I'm added as co-author is a purely political issue to me: would that help, or hurt, the PEP's chances? I couldn't care less whether my name is on it either way, and I'm currently unable to generate or test patches either way either.

containing scope, honoring a ``nonlocal`` or ``global`` declaration
for the target in that scope, if one exists. For the purpose of this
rule the containing scope of a nested comprehension is the scope that
contains the outermost comprehension.

This comment has been minimized.

Copy link
@tim-one

tim-one May 16, 2018

Member

That wasn't my intent (I intended "in the parent block, period") - but this variation makes more sense to me - thanks! So long as expressions can't contain statements, it makes things "just work" more often, at the cost of making it impossible for inner comprehensions to rebind outer comprehensions' for target names. That's no real loss.

If, however, Python grows to support "expressions" that can contain arbitrary statements, then it also makes it impossible for comprehensions to rebind other kinds of local names too in outer comprehensions.

As is, this example is annoying ;-)

def f():
    global j
    [[(j := j) for i in range(5)] for j in range(5)]

In j := j, is the left j global and the right j local to the outer listcomp? We already have (or should have) a rule that makes it an error for a name to be both a for and an := target at the same level, but the example dodges that rule. In the "always in the parent block" version, both instances of j refer to the outer listcomp's j.

"Left is global, right is local to containing listcomp" is fine by me in practice, as would be just about any other resolution.. But it merits some thought.

This comment has been minimized.

Copy link
@gvanrossum

gvanrossum May 16, 2018

Author Member

Oh, I forgot about the bit where we agreed that you can't rebind the loop control variable of a comprehension using :=. Let's extend that so that you can't rebind the loop control variable of any containing comprehension. That would make this example invalid, which seems better than allowing the example to dodge the rule.

This comment has been minimized.

Copy link
@tim-one

tim-one May 17, 2018

Member

That works for me. There's a formal property I'm keen to preserve: that at any given comprehension level, all instances of a := target resolve to the same scope. So if any other seeming counterexamples come up, I expect I'd rather say "you can't do that" than pick one more-than-less arbitrarily.

@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented May 17, 2018

OK, I've sent some updates:

  • Correct/clarify "commas to the right" section
  • Add Tim and Guido as authors
  • Update abstract to mention :=
  • Rule out targets conflicting with comprehension loop control

@tim-one If you have time and energy, it would be great if you could provide some motivating examples from your own life.

There are some TODOs in the text, and there's a section of open issues (with some overlap with the TODOs). Would be great to see opinions on these.

The author wishes to thank Guido van Rossum and Nick Coghlan for their
considerable contributions to this proposal, and to members of the
core-mentorship mailing list for assistance with implementation.
The author wishes to thank Guido van Rossum, Nick Coghlan, Tim Peters

This comment has been minimized.

Copy link
@ilevkivskyi

ilevkivskyi May 17, 2018

Contributor

This PR adds multiple authors so this paragraph would be outdated.

@ilevkivskyi

This comment has been minimized.

Copy link
Contributor

commented May 17, 2018

I am even thinking that we may recommend to have all discussions in a dedicated tracker or issue.

Good idea.

@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented May 17, 2018

Hey @stevendaprano are you interested in reviewing this update to PEP 572? (I'm not going to ask Nick since he's vowed to write a PEP about giving.)

@tim-one

This comment has been minimized.

Copy link
Member

commented May 17, 2018

[Guido]

@tim-one If you have time and energy, it would be great if you could provide some motivating examples from your own life.

I've been updating a mini-essay on the topic all along; maybe you could just slam in the following as an appendix ;-)

timcode.txt

@stevendaprano

This comment has been minimized.

Copy link
Member

commented May 18, 2018

@gvanrossum gvanrossum force-pushed the pep-572-update branch from 637e522 to 7f56435 May 18, 2018
@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented May 18, 2018

Thanks Tim, I added that as an appendix.

(Note that GitHub's "gvanrossum added some commits 2 days ago " is a lie -- most of those commits are from two days ago, but "Add timcode.txt as Appendix A" is from just now.)

@tim-one

This comment has been minimized.

Copy link
Member

commented May 19, 2018

I have another example that quite impressed me at the time. But I never posted it because nobody (except Mark Dickinson) would find it obvious on first or second reading ;-) Nevertheless, there may be some use in adding an example that isn't mindless ;-) Somebody else can judge that:

Where all variables are positive integers, and a is at least as large as the n'th root of x, this algorithm returns the floor of the n'th root of x (and roughly doubling the number of accurate bits per iteration):

while a > d := x // a**(n-1):
    a = ((n-1)*a + d) // n
return a

It's not obvious why that works, but is no more obvious in the "loop and a half" form. It's hard to prove correctness without building on the right insight (the "arithmetic mean - geometric mean inequality"), and knowing some non-trivial things about how nested floor functions behave. That is, the challenges are in the math, not really in the coding.

If you do know all that, then the assignment-expression form is easily read as "while the current guess is too large, get a smaller guess", where the "too large?" test and the new guess share an expensive sub-expression.

To my eyes, the original form is harder to understand:

while True:
    d = x // a**(n-1)
    if a <= d:
        break
    a = ((n-1)*a + d) // n
return a
gvanrossum added 12 commits May 16, 2018
- Remove changes to comprehension scope
- Make := in comprehensions assign to containing non-comprehension scope
- Clarify binding precedence (tighter than comma, not at same level as =)
- Remove mention of more complex targets in the future
- Explicitly disallow toplevel :=
- Rewrite section on differences with =, enumerating all of them
- Remove "here's how this could be written without :=" from examples
- Tweak first paragraph of "Syntax and semantics"
- Add "Exception cases" (various explicit prohibitions)
- Clarify that lambda is a containing scope
- Clarify that := and = just don't mix
- Added "Open questions" section
- Added two new rejected alternatives: "Allowing commas to the right"
  and "Always requiring parentheses"
- Minor edits
@stevendaprano

This comment has been minimized.

Copy link
Member

commented May 20, 2018

A very nice set of changes. I'm glad to see the changes to class scopes are gone: that ought to be a separate PEP.

I especially liked your examples of using := to capture witnesses for any/all. There have been a few requests for them to return the truthy/falsey value, and this is a reasonable way to get that effect.

A couple of other observations/comments...

You have prohibited using assignment expressions in function defaults:

def foo(answer=p := 42):  # INVALID

I don't know how I feel about this, but I recall at least one question of Python-List asking why they can't do this:

def foo(answer=42, value=answer+1):

It was some years ago and I can't find the post now. But perhaps we might say this is an obscure enough usage that we don't care to support it.

Regarding the Open Questions, I agree with all of the current answers with the possible exception of the last (function defaults) where I'm on the fence.

@stevendaprano

This comment has been minimized.

Copy link
Member

commented May 20, 2018

Chris, Guido, would you like me to write up a summary of the "Modern language design survey" thread for inclusion in the PEP?

@Rosuav

This comment has been minimized.

Copy link
Contributor

commented May 20, 2018

Yes please! That'd make a great section IMO.

@gvanrossum gvanrossum force-pushed the pep-572-update branch from 77f43c4 to a89fb51 May 20, 2018
@@ -691,6 +691,144 @@ considerable contributions to this proposal, and members of the
core-mentorship mailing list for assistance with implementation.


Appendix A: Tim Peters' findings
================================

This comment has been minimized.

Copy link
@tim-one

tim-one May 21, 2018

Member

s/Peters'/Peters's/ Sorry, but that's English ;-)

Copy link
Member

left a comment

I added comments to two lines suggesting grammatical changes. I'm not sure why github is making me type this too ;-)

It's also nice to trade away a small amount of horizontal whitespace
to get another _line_ of surrounding code on screen. I didn't give
much weight to this at first, but it was so very frequent it added up,
and I soon enough become annoyed that I couldn't actually run the

This comment has been minimized.

Copy link
@tim-one

tim-one May 21, 2018

Member

s/become/became/

@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented May 21, 2018

@ncoghlan

This comment has been minimized.

Copy link
Contributor

commented May 22, 2018

I've assigned my competing PEP a number (577) and merged it now. PR where I did that: #660

I found the scoping discussion with Tim very helpful, to the point where I became a convert to allowing inline assignment as a variant of augmented assignment, and structured the PEP primarily around allowing the existing augmented assignment operators to be used as expressions. Adding NAME := EXPR then becomes primarily about avoiding folks going to silly lengths to emulate it with one of the other operators.

While I've borrowed much of Tim's scoping proposal, what I have in PEP 577 isn't exactly the same as what you have here:

  1. No implicit declarations of augmented assignment targets at function scope (you get TargetNameError instead). In combination with a suggested style rule for linters and type checkers, this takes the place of the general prohibition on top level usage of NAME := EXPR.
  2. Lambda expressions get ignored when calculating augmented assignment targets the same way other expression level scopes do. This allows constructs that currently require 3 keywords (def, nonlocal, return) and three repetitions of a variable name to be replaced with a single keyword (lambda) and a single inline assignment expression.

(Note: now that Guido has pointed me to this PR, I won't post PEP 577 to python-ideas until you're happy with the changes you've made to PEP 572 and have merged them).

@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented May 22, 2018

@Rosuav @tim-one Any further updates or can I push this? (I don't intend to post to python-ideas yet, but I'd rather not sit on such a momentous update forever. Let's iterate.)

@Rosuav

This comment has been minimized.

Copy link
Contributor

commented May 22, 2018

I'm cool with it. +1.

@tim-one

This comment has been minimized.

Copy link
Member

commented May 22, 2018

@gvanrossum +1 . Process question: if I'd like to make small changes later (I installed git and will eventually figure out how to use it ;-) ), would I push them to this branch (pep-572-update) or create a new branch?

@gvanrossum gvanrossum merged commit 6d81538 into master May 22, 2018
1 check passed
1 check passed
continuous-integration/travis-ci/push The Travis CI build passed
Details
@gvanrossum gvanrossum deleted the pep-572-update branch May 22, 2018
@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented May 22, 2018

Create a new branch and make a new PR. This branch is over and done with now that it's been merged! Good luck learning git.

@JimJJewett

This comment has been minimized.

Copy link
Contributor

commented Jul 10, 2018

I'm one who thinks this should be available where statements are used today. But since the reason to use it there would be to de-emphasize the variable (limit the scope conceptually), making it (literally) a parenthetical comment is fine.

I'm not happy about the number (or even existence) of exceptions/special cases listed, but I think that in practice that will be a problem only for implementors, not users, so ... basically, this comment is a reminder that not everyone is upset by the PEP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.