Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: spec: various changes to := #377

Open
agl opened this issue Dec 3, 2009 · 91 comments
Open

proposal: spec: various changes to := #377

agl opened this issue Dec 3, 2009 · 91 comments
Labels
LanguageChange NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Proposal v2 A language change or incompatible library change
Milestone

Comments

@agl
Copy link
Contributor

agl commented Dec 3, 2009

This code is subtly wrong:

func f() (err os.Error) {
  v, err := g();
  if err != nil {
    return;
  }
  if v {
    v, err := h();
    if err != nil {
      return;
    }
  }
}

The := in the if statement causes a new err variable that shadows the 
return parameter.

Maybe doing this should be an error. Maybe return parameters should be 
special so that := doesn't ever shadow them. (I like the latter.)
@peterGo
Copy link
Contributor

peterGo commented Dec 3, 2009

Comment 1:

I noticed this situation a while ago. I argued that it conforms to the scope rules,
which are usual and customary.
The first err is declared under rule 4. The second err is declared under rule 5. The
second declaration is the inner declaration, so the inner redeclaration rule applies,
thereby hiding, within its own scope, the first err.
This is the usual and customary behaviour for many languages. Some languages have a
construct which allows a reference in the inner scope to the variable in the outer scope.
The Go Programming Language Specification
Declarations and scope
The scope of a declared identifier is the extent of source text in which the
identifier denotes the specified constant, type, variable, function, or package.
Go is lexically scoped using blocks:
   1. The scope of a predeclared identifier is the universe block.
   2. The scope of an identifier denoting a constant, type, variable, or function
declared at top level (outside any function) is the package block.
   3. The scope of an imported package identifier is the file block of the file
containing the import declaration.
   4. The scope of an identifier denoting a function parameter or result variable is
the function body.
   5. The scope of a constant or variable identifier declared inside a function
begins at the end of the ConstSpec or VarSpec and ends at the end of the innermost
containing block.
   6. The scope of a type identifier declared inside a function begins at the
identifier in the TypeSpec and ends at the end of the innermost containing block.
An identifier declared in a block may be redeclared in an inner block. While the
identifier of the inner declaration is in scope, it denotes the entity declared by
the inner declaration.

@gopherbot
Copy link

gopherbot commented Dec 9, 2009

Comment 3 by pshah.foss:

Is there anyway to access the variable in outer scope ?

@rsc
Copy link
Contributor

rsc commented Jan 11, 2010

Comment 4:

Issue #514 has been merged into this issue.

@rsc
Copy link
Contributor

rsc commented Jan 19, 2010

Comment 5:

This issue now tracks various proposals that have been made, among them:
  * disallow shadowing outer variables
  * allow arbitary expressions on the lhs
  * don't require something new on the lhs

@rsc
Copy link
Contributor

rsc commented Jan 19, 2010

Comment 6:

Issue #505 has been merged into this issue.

@rsc
Copy link
Contributor

rsc commented Jan 19, 2010

Comment 7:

Issue #469 has been merged into this issue.

@gopherbot
Copy link

gopherbot commented Feb 5, 2010

Comment 8 by jesse.dailey:

The go spec for short variable declaration specifically addresses redeclaration, and  
explicitly states that this should not happen.
From the go spec:
"a short variable declaration may redeclare variables provided they were originally 
declared in the same block with the same type"
Right now, you can shadow global variables, and redeclare their type.
"Redeclaration does not introduce a new variable; it just assigns a new value to the 
original."
var someGlobal = "foo";
func someFunc() (int, os.Error) {
  return 1, nil
}
func TestThree(t *testing.T) {
  if someGlobal, err := someFunc(); err == nil {
    // rather than throwing an error, someGlobal will now silently be an int == 1
  }
  // now it will be a string == "foo" again
}

@rsc
Copy link
Contributor

rsc commented Feb 5, 2010

Comment 9:

@jesse.dailey: The implementation is in line with the spec.  The proposal is a change 
to the spec.
x := 1
{ 
    x := 2
}
The two x are in different blocks so the sentence you quoted does not apply.

@rogpeppe
Copy link
Contributor

rogpeppe commented Mar 26, 2010

Comment 10:

another possibility that i think would be useful:
allow non-variables on the l.h.s. of a := as long
as there's one new variable there.
e.g.
   x := new(SomeStruct)
   x.Field, err := os.Open(...)
i actually think this is less controversial than the original
rule allowing non-new variables - at least it's obvious
at a glance which variables have been declared.

@gopherbot
Copy link

gopherbot commented Apr 21, 2010

Comment 11 by ravenstone13@cox.net:

I think the original poster was making a case for special treatment of return
parameters.  In principle I agree with his argument that it would reduce the
potential for a certain class of subtle errors.  The question is whether this
potential benefit is worth introducing a 'special case' into the spec and eventually
into all go compiler implementations.  Since much is being made by go promoters about
it being a 'safe' language I'm leaning towards agreement with OP, ie. no shadows of
return parameters.  I realize this isn't a democracy, it's just my opinion FWIW  :-)

@rsc
Copy link
Contributor

rsc commented Apr 26, 2010

Comment 12:

Issue #739 has been merged into this issue.

@gopherbot
Copy link

gopherbot commented Nov 20, 2010

Comment 13 by snake.scaly:

I think the OP highlights a more general problem: redeclaring variables from outer
scopes can create subtle, hard to track down errors.
Possible solution: make it an error to redeclare a variable declared in the same
function.
Rationale for this language change:
* A whole class of hard to fix errors is eliminated
* Probably it won't hurt most of existing, correct Go code
* Probably it will highlight bugs or hard-to-maintain spots in the existing code
* Redeclaration of global names is still allowed so that a new version of `import .
"Foo"` package won't hijack your code
* Does not complicate specification
* Does not seem to complicate implementation, at least not much

@mark-summerfield
Copy link

mark-summerfield commented Nov 20, 2010

Comment 14:

One thing that could presumably done right now without changing the language is to
provide a warning when shadowing occurs. (Of course then it would be nice to have a
warning level option to give to the compiler so that the warning could be switched off
by people who don't like it.)

@niemeyer
Copy link
Contributor

niemeyer commented Nov 23, 2010

Comment 15:

I'd like to introduce one additional proposal for consideration,
which I believe addresses the original problem brought up by the OP,
and which hasn't been covered yet.
What if "=" was allowed to be used to declare variables as well,
but only if at least one of the variables has *been* previously
declared?
In other words, this would be valid:
   a, err := f()
   if err == nil {
       b, err = g()
       if err == nil { ... }
   }
   return err
This would be the exact counterpart behavior to :=, which may only be
used when at least one of the variables has *not* been declared
previously.  It feels like I'd appreciate using this in practice, and
would avoid the errors I've personally found with the shadowing.
How do you all feel about this?

@niemeyer
Copy link
Contributor

niemeyer commented Nov 30, 2010

Comment 16:

Another alternative based on the conversation in the mailing list would be to use a
per-variable declaration syntax.
For instance, this:
   a, b := f()
Would be fully equivalent to:
   :a, :b = f()
and a construction in an internal block such as:
   err = f()
might be extended to the following, which is completely clear and unambiguous:
   :a, err = f()
When one doesn't want to redefine err.
One of the things which feels interesting about this proposal is that
it would enable forbidding entirely partial declarations via := if
that's decided to be a good option, without compromising on other
aspects of the language.

@gopherbot
Copy link

gopherbot commented Dec 10, 2010

Comment 17 by czapkofan:

Alternative proposals in spirit similar to comment 16, based on ideas expressed in
http://groups.google.com/group/golang-nuts/browse_thread/thread/5f070b3c5f60dbc1 :
Ideas, Variant 1:
  a, (b) := f1()             // redefines b, reuses a
  (a, b), c, (d, e) := f2()  // redefines c, reuses a, b, d, e
  // Flaw example: redundant with "a = f3()":
  (a) := f3()                // reuses a
Ideas, Variant 2:
  (var a), b = f1()           // redefines a, reuses b
  a, b, (var c), d, e = f2()  // redefines c, reuses a, b, d, e
  // Flaw example: redundant with "var a = f4":
  (var a) = f4()              // redefines a

@gopherbot
Copy link

gopherbot commented Dec 10, 2010

Comment 18 by daveroundy:

I like this approach:
(var a), b = f1()           // redefines a, reuses b
precisely because it is so close to the already-existing equivalence between
a := f1()
and
var a = f1()

@rogpeppe
Copy link
Contributor

rogpeppe commented Dec 10, 2010

Comment 19:

i'm not keen on that, because it's so heavyweight (5 extra characters).
you might as well do
a, nb := f1()
b = nb

@gopherbot
Copy link

gopherbot commented Dec 11, 2010

Comment 20 by james@abneptis.com:

I won't re-raise this on the list, but after thinking a few more days, I think my
biggest disagreement with the current implementation allowing (the above given):
func TestThree(t *testing.T) {
  if someGlobal, err := someFunc(); err == nil {
    // rather than throwing an error, someGlobal will now silently be an int == 1
  }
  // now it will be a string == "foo" again
}
Is that the part that is creating the issue "if someGlobal, err := someFunc(); err ==
nil" doesn't /really/ seem to be part of the inner block scope to the reader;
Yes, it's completely expected that loop setup variables would be available within the
scope of the loop, and perhaps even, by default, not available outside of the loop
scope.  BUT, since the "clause" is outside of the braces, I think it's reasonable for a
coder to assume that it has a "middle" scope, that would by default inherit from the
global scope if available, otherwise creating variable solely available to the inner
loop scope.
I realize that's a complex description of the change, but I think if /clauses/ are
solely targeted with the change, we'd minimize the chance for both confusion and bugs
unintentionally introduced.
(And if unchanged, I'd love a compiler warning, but hey, I know that's not in the plans
;) )

@peterGo
Copy link
Contributor

peterGo commented Dec 11, 2010

Comment 21:

James,
"A block is a sequence of declarations and statements within matching brace brackets.
Block = "{" { Statement ";" } "}" . In addition to explicit blocks in the source code,
there are implicit blocks:
   1. The universe block encompasses all Go source text.
   2. Each package has a package block containing all Go source text for that package.
   3. Each file has a file block containing all Go source text in that file.
   4. Each if, for, and switch statement is considered to be in its own implicit block.
   5. Each clause in a switch or select statement acts as an implicit block."
Blocks, The Go Programming Language Specification.
http://golang.org/doc/go_spec.html#Blocks
"In some contexts such as the initializers for if, for, or switch statements, [short
variable declarations] can be used to declare local temporary variables."
Short variable declarations, The Go Programming Language Specification.
http://golang.org/doc/go_spec.html#Short_variable_declarations
Therefore, until you can do it automatically in your head, you can simply explicitly
insert the implicit blocks. For example,
var x = "unum"
func implicit() {
    fmt.Println(x) // x = "unum"
    x := "one"
    fmt.Println(x) // x = "one"
    if x, err := 1, (*int)(nil); err == nil {
        fmt.Println(x) // x = 1
    }
    fmt.Println(x) // x = "one"
}
func explicit() {
    fmt.Println(x) // x = "unum"
    {
        x := "one"
        fmt.Println(x) // x = "one"
        {
            if x, err := 1, (*int)(nil); err == nil {
                fmt.Println(x) // x = 1
            }
        }
        fmt.Println(x) // x = "one"
    }
}

@gopherbot
Copy link

gopherbot commented Dec 11, 2010

Comment 22 by james@abneptis.com:

Thanks;  It's not so much that I don't understand with it, or even disagree with it; 
It's that it's a frequent source of errors that are hard to physically see (differing
only in colon can have a dramatically different result).
(snip much longer ramble)
I have no problem with
var v; 
func(){ v := 3 }
It's 
foo()(err os.Error){
  for err := bar(); err != nil; err = bar() {
  }
}
being substantially different than
foo()(err os.Error){
  for err = bar(); err != nil; err = bar() {
  }
}
and both being semantically correct.
Essentially, my argument is w/r/t ONLY: "In some contexts such as the initializers for
if, for, or switch statements, [short variable declarations] can be used to declare
local temporary variables";  I would argue that since these are special cases to begin
with, that in multi-variable := usage, resolving those local temporary variables should
be handled via the same scope as the containing block, but stored in the inner scope if
creation is necessary;
I've got no problem with how it works, just been bitten by this more times than I'd care
to admit, and surprised when I'd realized how many others had been as well.

@gopherbot
Copy link

gopherbot commented Jan 21, 2011

Comment 23 by ziutek@Lnet.pl:

I think that Go should be explicit language.
I prefer Go:
    ui = uint(si)
than C:
    ui = si
if ui is unsigned and si is signed. Why do we need an implicit behavior of :=?
So if := is the declaration operator it should work exactly like var for all its lhs.
If some of lhs are previously declared in this scope, it should fail - I believe we
should have a separate explicit construction for this case. Proposal from comment 16 is
nice for me:
    :a, b = f()
In above case it doesn't introduce any additional character. In:
    :a, b, :c = f()
it adds only one.
This notation looks good. I can easily determine what's going on.
    a, b, c := f()
should be an abbreviation for:
    :a, :b, :c = f()
With current := behavior I fill like this:
    :=?
I vote for change this emoticon to:
    :=
in the future.
;)

@nsf
Copy link

nsf commented Apr 6, 2011

Comment 24:

In fact I think there is a perfect solution in one of the proposals. So, I'll sum up
what I think:
1. Allow arbitrary addressable expressions on the left side of ':='.
2. Allow no new variables on the left side of ':=' (a matter of consistency in the code,
see examples).
3. Use the following rule to distinguish between a need of "declare and initialize" and
"reuse":
If the LHS looks like an identifier, then the meaning is: "declare and initialize".
Trying to redeclare a variable in the current block that way will issue an error.
Otherwise LHS must be an addressable expression and the meaning is: "reuse". Rule allows
one to use paren expression to trick the compiler into thinking that an identifier is an
addressable expression.
Examples:
a, err := A()   // 'a' and 'err' are identifiers - declare and initialize
b, (err) := B() // 'b' - declare and initialize, '(err)' looks like an addressable
expression - reuse
type MyStruct struct {
    a, b int
}
var x MyStruct
x.a, err := A()   // 'x.a' is an addressable expression - reuse, 'err' is an identifier
- declare and initialize
x.b, (err) := B() // 'x.b' and '(err)' are addressable expressions - reuse (special case
without any new variables)
Of course it could be:
x.b, err = B()    // and that's just a matter of preferrence and consistency
Note: My idea is a bit different from proposal above, the following syntax is invalid: 
(a, b), c := Foo()
The right way to do this is:
(a), (b), c := Foo()
Yes, it's a bit longer. But keep in mind that the alternative is typing 'var a Type',
'var b Type'. Using parens is perfectly fine to me for such a complex case.
Also this approach has one very cool property - it almost doesn't alter syntax (allowing
arbitrary addressable expressions on the left side of ':=' is the only change), only
special semantic meaning.

@niemeyer
Copy link
Contributor

niemeyer commented Apr 6, 2011

Comment 25:

I'd still prefer
  :a, :b, c = Foo()
But at this point it's really just syntax.  I'd be happy with either approach.

@gopherbot
Copy link

gopherbot commented Apr 6, 2011

Comment 26 by ckrueger:

I am in favor of doing away with := entirely because of the desire to control what is
done per-value on multiple returns.  
The :val syntax described above seems nice and short and would seem like valid syntactic
sugar for a keyword driven solution:
:x = f(), declare(shadow) and initialize x, infer type
x  = f(), assign x, infer type
would be the same as
auto var x = f(), declare(shadow) and initialize x, infer type
auto x = f(), assign x, infer type
to revisit the implicit/explicit example shown above in comment 21:
var x = "unum"
func implicit() {
    fmt.Println(x) // x = "unum"
    :x = "one" //<- potentially make this an error, redeclaration after use in same scope.
    //:x = "two" <- would not compile, can only declare once in scope
    fmt.Println(x) // x = "one", global x still = "unum"
    if :x, :err = 1, (*int)(nil); err == nil {
        fmt.Println(x) // x = 1
    }
    fmt.Println(x) // x = "one"
}
func explicit() {
    fmt.Println(x) // x = "unum"
    {
        :x = "one"
        fmt.Println(x) // x = "one"
        {
            if :x, :err = 1, (*int)(nil); err == nil {
                fmt.Println(x) // x = 1
            }
        }
        fmt.Println(x) // x = "one"
    }
    fmt.Println(x) // x = "unum"
}
to revisit the example in the original post:
func f() (err os.Error) {
  :v, err = g(); <-- reusing err for return
  if err != nil {
    return;
  }
  if v {
    :v, err = h(); <-- shadowing v, but reusing err for return
    if err != nil {
      return;
    }
  }
}
in addition, if one wants to enforce typing per-value, specifying type removes the need
for :val as you cannot re-specify a type on an existing value and thus initialisation is
inferred.
int :x, os.Error err = f(); initialize and assign x/error, don't compile if return value
2 is not os.Error

@rsc
Copy link
Contributor

rsc commented Jul 25, 2011

Comment 27:

I think it's safe to say we're not going to change :=.

Status changed to WorkingAsIntended.

@gopherbot
Copy link

gopherbot commented Jul 26, 2011

Comment 28 by czapkofan:

Could you possibly elaborate a bit why? Especially with regards to the alternative
"explicit" syntax proposals?
I don't plan to argue, the right to any final decision is obviously yours as always; but
I'd be highly interested to know if there are some problems expected to be introduced by
those proposals, or otherwise what is the common reasoning behind this decision.
Thanks.

@niemeyer
Copy link
Contributor

niemeyer commented Jul 26, 2011

Comment 29:

Agreed.  Besides _changing_ :=, there are other proposals, and this problem was brought
up repeatedly in the mailing list by completely different people, with this thread being
referenced as the future answer (how many issues are starred by 46+ people?).
It'd be nice to have some more careful consideration and feedback before dismissal.

@rsc
Copy link
Contributor

rsc commented Jul 26, 2011

Comment 30:

The decision about := is not mine, at least not mine alone.
I am just trying to clean up the bug tracker, so that it reflects
things we need to work on.
1. The bug entry is 1.5 years old at this point.  If it were
going to have an effect, it would have by now.
2. This comes up occasionally on its own.  A bug entry is
not necessary to remind us about it.
I'll change the status back to long term but I remain
skeptical that anything will change.

Status changed to LongTerm.

@niemeyer
Copy link
Contributor

niemeyer commented Jul 26, 2011

Comment 31:

Thanks for the work on cleaning up, it's appreciated. It's also certainly fine for this
to be closed if it reflects a decision made.
The point was mostly that it'd be nice to have some feedback on the features proposed
for solving the problem, given that there's so much interest on the problem and a bit of
love towards a few of the proposed solutions.
E.g. allowing this:
    :v, err = f()
as equivalent to
    var v T
    v, err = f()
If you have internally decided this is off the table, it'd be nice to know it, and if
possible what was the reasoning.

@TapirLiu
Copy link
Contributor

TapirLiu commented Feb 29, 2020

I just got an idea which acts like an inverse of @niemeyer's idea. (edit: It is more like @nsf's idea)

The idea removes redeclaration := syntax (which is the different point to the ideas from @niemeyer and @nsf) and tries to solve both the problems mentioned in the current proposal and this one.

The following code shows the logic of the idea:

package bar

func foo() {
   var x, err = f()
   ...
   // Here ":err" means the "err" declared above.
   var y, z, :err = g()
   ...
   {
   	// In fact, ":id" means the innermost already declared "id"
   	var w, :err = h()
   	...
   	// "::id" means the package-level declared "id".
   	var u, v, ::err = j()
   	...
   	// This "err" is a new declared one.
   	var m, n, err = k()
   	...
   }
}

(edit: the ::id syntax is not essential in this idea.)

Selectors, dereferences, element indexing, and the above mentioned :id/::id forms are allowed to show up in a variable declaration as long as there is at least one pure identifier in the declaration. The pure identifiers represent new declared variables for sure.

var err error

type T struct {
	i *int
	s []int
}
var t T
var p = new(int)

func bar() {
   var *p, t.i, t.s[1], err = 1, p, 2, ::err
   // Yes, the "::id" can be used as sources.
   // ":id" can also but it is a non-sense.
   ...
}

The var keyword can be omitted in the short statements in all kinds of control flows.
For example,

if a, b = c, d; a {
}
//<=>
if var a, b = c, d; a { // some ugly
}

The idea should be Go 1 compatible.

@TapirLiu

This comment has been minimized.

@TapirLiu
Copy link
Contributor

TapirLiu commented Mar 1, 2020

Sorry, the description for control flow short statements is not correct. In fact, the := redeclaration syntax should be kept, but ONLY in the short statements in control flows, just for aesthetics reason. In other words,

if a, b := c, d; a {
}
//<=>
if var a, b = c, d; a { // some ugly
}

The whole idea still keeps Go 1 compatible with this change.

@go101
Copy link

go101 commented Apr 12, 2020

Another simple fix is to prefix *& to re-declared identifiers to indicate the prefixed identifiers are redeclared.

var a, err = A()
var b, *&err = B()

The benefit here is that no new expression notations are invented (same as (err)).

@carnott-snap
Copy link

carnott-snap commented May 13, 2020

Is *& a nop at runtime, or are you suggesting it should be? It feels like the compiler should be smart enough to figure it out today.

@go101
Copy link

go101 commented May 14, 2020

Yes, it has already been a nop now.

@srinathh
Copy link
Contributor

srinathh commented May 30, 2020

Could we consider a clearer (but more verbose) solution to the problem in the same spirit as global keyword in Python which is widely used?

Proposal

  1. A new keyword outerscp is introduced to the language.
  • Usage: outerScp var1, var2, var3
  • It may be used only in the beginning of a scope
  • var1, var2 etc. are variables declared in the outer scope
  1. Whenever a variable name is encountered in the inner scope that has been explicitly specified in the outerscp statement, it is not shadowed
  2. In the absence of this, the standard Go 1 variable shadowing rules apply

Current Go 1 variable shadowing

package main

import (
	"fmt"
)

var a int

func printGlobal() {
	fmt.Printf("Print Global: %d\n", a)
}

func getNum() (int, error) {
	return 4, nil
}

func main() {

	a, err := getNum()
	if err != nil{
		fmt.Printf("error")
		return
	}
	
	printGlobal()
	fmt.Printf("Printing in main: %d\n", a)
}

Output

Print Global: 0
Printing in main: 4

With outerscp keyword

package main

import (
	"fmt"
)

var a int

func printGlobal() {
	fmt.Printf("Print Global: %d\n", a)
}

func getNum() (int, error) {
	return 4, nil
}

func main() {
        outerscp a

	a, err := getNum()
	if err != nil{
		fmt.Printf("error")
		return
	}
	
	printGlobal()
	fmt.Printf("Printing in main: %d\n", a)
}

Output

Print Global: 4
Printing in main: 4

Advantages:

  1. Maintains compatibility with Go 1
  2. Explicitly identifies which variables to not be shadowed improving clarity

DisAdvantages:

  1. The outerscp keyword looks ugly. However
  • no public Go code in github uses this term currently as an identifier
  • outerScope keyword is used in several public repositories
  • outer, global etc. are of course used in many repos as identifiers

@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented Jun 1, 2020

@srinathh That problem with a keyword like outerscp is that it only helps if you know that you have a problem. If you know that you have a problem, there are other things you can do, like not use :=.

@srinathh
Copy link
Contributor

srinathh commented Jun 1, 2020

@ianlancetaylor Yes you're right... but that criticism is equally applicable to any other proposal short of completely removing support for the := operator.

I think the key to reducing possibility of subtle bugs is reduce cognitive load so programmers can better maintain contextual awareness. A keyword approach I think makes things a lot more obvious vs. introducing cryptic notations into the := assignment statement and should be familiar at least to Python programmers who form the biggest chunk of Go users who use multiple languages as per the survey.

@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented Jun 1, 2020

@srinathh Fair point. I guess what I'm mean is that while cryptic notations are definitely cryptic, and perhaps not a good idea, at least they are short. If I have to write out outerscp v, then I might as well just write var v as needed and stop using :=.

@a-canya
Copy link

a-canya commented Mar 29, 2021

Hi, I came up with a slightly different approach for this issue. Will be glad to hear the community's thoughts on this.

Problem

In my point of view, one of the main problems with := is that it hides which variables are actually being declared.

a, b := 1, 1 // depending on the context, this might mean 3 different things
// a and b are declared and initialized
// a is declared and initialized, b is only assigned a new value
// a is only assigned a new value, b is declared and initialized

This might seem harmless but can be very misleading specially when it is combined with shadowed variables:

a, err := foo() // a and err are declared and initialized
b, err := bar() // b is declared and initialized, err is assigned a new value
if c, err := foobar(); err != nil { // c and err are declared and initialized inside
                                    // the scope of the if, err is shadowed
}
// now err has the value returned by bar, not foobar

In this example, the programmer might have expected the same behaviour in the assignments of err from bar and foobar. Assuming that variable masking is here to stay (although that might an interesting debate), I think that the best solution is to change the ambiguity of := declarations. More specifically, the problem is this part of the specification:

Unlike regular variable declarations, a short variable declaration may redeclare variables provided they were originally declared earlier in the same block (or the parameter lists if the block is the function body) with the same type, and at least one of the non-blank variables is new. As a consequence, redeclaration can only appear in a multi-variable short declaration. Redeclaration does not introduce a new variable; it just assigns a new value to the original.

Source

This "redeclaration" in the same block which "does not introduce a new variable" does not only cause the aforementioned confusions, but I also find it difficult to understand conceptually. Why is it even called redeclaration, when this is just simply an assignment? Reading this made me doubt if the address of a "redeclared" variable was preserved (it is).

Proposal

  1. All identifiers in the left hand of a := are always declared as new variables, and redeclarations inside the same block are not allowed without exceptions.
a, err := foo()
b, err := foo() // compilation error because var err is already declared in this block
  1. A mixed used of = and := is allowed in a single line, as shown in the following examples.
a:=, err:= foo() // a and err are declared and initialized (equivalent to: a, err := foo())
b:=, err= foo() // b is declared and initialized, whereas err is only assigned a new value
if true {
    c:=, err= foo() // c is declared and initialized in the if block, and err is assigned a new value
}
if true {
    d:=, err:= foo() // d and err are declared in the if block, err is shadowed
}

Similar proposals

This is similar to other proposals such as using :a, a', ^a or (a) to distinguish which variables are declared and which are just assigned. In my opinion my proposal is clearer and more in line with how the language looks currently. The meanings of := as short declaration+initialization, and = as just assignment is preserved and strengthened.

Backwards compatibility

Part 1 of the proposal is not backwards compatible. Part 2 could be introduced as a Go 1 minor release.

With a partial adoption, we could use automatic tools to break problematic expressions such as a, b := foo() to less ambiguous a:=, b= foo(), a=, b:= foo() or a:=, b:= foo() (the last one could remain as a, b := foo(), at this point this depends on the preferred format).

Pros and cons

  • Pros
    • Avoids confusions in current ambiguous mixed assignment+declarations using :=
    • Does not add unnecessary extra verbosity
    • (In my opinion) it's clearer than other proposals such as a'
    • Eliminates a confusing exception in the current language spec.
  • Cons
    • Can be visually and aesthetically problematic. The order of priorities of = and , is altered, which can be something difficult to get used to.
    • Does not solve the problem of unwanted redeclarations in an inner scope.

Some remarks

With this change, single-variable short declarations (a := 1) and assignments (a = 1) can be viewed as a specific case of the new rule. Moreover, current multivariable expressions such as a, b := foo() and a, b = foo() could be viewed as a simplification of a:=, b:= foo() and a=, b= foo(), respectively.

I suggest as a default gofmt to eliminate the space between identifier and short declaration / assignment operator when there is more than one operator, to emphasize that the operator only applies to the identifier that precedes the operator. Ie: a:=, b= foo() instead of, eg, a :=, b = foo().

@pam4
Copy link

pam4 commented Mar 30, 2021

@a-canya, thanks for the proposal,
the lack of such a mechanism has always been the single most annoying property of the language for me (I've been using a code rewriter for years), therefore at this point I would be happy with (almost) any syntax.
But at first glance I'm not convinced that this one is "clearer and more in line with how the language looks currently".
Your syntax has only an optional space separating the LHS from the RHS; I think having a single operator in the middle is clearer. And we are already familiar with a tuple of the form expr1, expr2,... but not with one of the form expr1 oper1, expr2 oper2,..., and they would have to coexist.
Compare:

a:=, b= c, d

vs.

:a, b = c, d

The := itself is just an assignment plus something more. It feels natural to me to untie : and =, since the = is still meaningful for the whole tuple while the "something more" (:) should be per-variable.
The latter syntax is also totally backward compatible.
That one is still my favorite, though unlikely to go anywhere after 10+ years, that's why I'm pushing for the more popular #30318 (which is just another equivalent syntax, once we get rid of redeclarations).

Why is it even called redeclaration, when this is just simply an assignment?

Good question.
It's a confusing choice of word and a very roundabout way of describing what is essentially an assignment to an existing variable. In my experience most users think of it as a reuse/assignment, and many don't know what a "redeclaration" is, or they tend to confuse it with shadowing.

@go101
Copy link

go101 commented Apr 1, 2021

I haven't seen strong opposition to @nsf's suggestion. IMHO, this is the best way to solve this problem by far. It is just a little (but acceptably) verbose.

@a-canya
Copy link

a-canya commented Apr 1, 2021

@pam4 thanks for the answer! Very valid point indeed, and you show a good example of something that would confusing ugly if my proposal were accepted.

I'd like to remark that using the := with a mix of variables that are declared and "redeclared" only makes sense to me when using a function in the RHS:

// Case 1
a, b:= foo(1, "hello", x)  // current ambiguous approach
:a, b = foo(1, "hello", x) // alternative 1
a:=, b= foo(1, "hello", x) // alternative 2

// Case 2. IMO that's simply bad programming. Just break it into 2 lines for God's sake
a, b := 1, 2 // current ambiguous approach
:a, b = 1, 2 // alternative 1
a:=, b= 1, 2 // alternative 2

I don't like mixing assignments and declarations, they are essentially different operations. I understand that's convenient for function returns, but other than that I would strongly discourage anyone from doing it. And looking at case 1 I don't think I'd have a problem visually distiguishing the RHS and LHS.

In any case, which implementation looks better si very subjective. I hope we can find one alternative that's good enough for everyone to make a change.

@carlmjohnson
Copy link
Contributor

carlmjohnson commented Apr 1, 2021

Here is my proposal. Assume a go.mod gated roll out.

There are various situations in which it is annoying or impossible to specify the type for a variable. For example, a recursive closure must be written as

var f func(a B, b B) c C
f = func(a B, b B) c C {
   // ...
}

This is because f won't be in its own scope if it is assigned with :=. OTOH, if you have an exported function that returns an unexported type, you have to use assignment for initialization:

var x pkg.xtype // illegal
x := pkg.Xer() // ok
var x = pkg.Xer() // ok
var y pkg.Ytype; x, y = pkg.XAndYer() // illegal and lots of weird edge cases about when you can assign to y

So, what should we do? Let's take the auto keyword from the C family. auto means "create a new variable whose type is based on its subsequent first assignment". While we're at it, let's make shadowing with := illegal.

So, the classic nested err thing becomes

a, err := firstOperation()
{  // new scope
    auto b
    b, err = secondOperation() // b has the type of secondOperation's first return value
    c, err := thirdOperation() // illegal! no shadowing err
}

Tricky bits:

  • If an auto variable is not assigned to, or is read before assignment, that's illegal.
  • This needs to be illegal
auto a
if cond {
   a = 1
} else {
   a = "string"
}

OTOH, I think this should probably be legal, so having the assignment to an auto in an inner scope should be okay:

auto a
if a, err = operation(); err != nil {
   // ...
}
// a is in scope here

Advantages of auto:

  • No weird symbols
  • Fixes the := shadowing problem
  • Also useful for recursive closures and unexported variables
  • Keyword is familiar to users of C-family languages

@pam4
Copy link

pam4 commented Apr 1, 2021

@go101

I haven't seen strong opposition to @nsf's suggestion.

As I noted above, nsf's suggestion is basically #30318 plus 2 points:

  1. Allow no new variables on the left side of :=
    this is just cosmetic, if there are no new variables you can use =.
  2. Eliminate := redeclaration
    this is important but it must wait for Go2 (not backward compatible).

There is already some consensus about getting rid of redeclaration, so I think that if we can push #30318 through, we're almost there.

IMHO, this is the best way to solve this problem by far.

I can't think of any (technical) advantage over the backward compatible colon-prefix syntax, but #30318 is far more popular, with many upvotes, and already implemented!

@a-canya

I'd like to remark that using the := with a mix of variables that are declared and "redeclared" only makes sense to me when using a function in the RHS:

I agree, but I think it's important to also consider edge cases.

I don't like mixing assignments and declarations, they are essentially different operations. I understand that's convenient for function returns, but other than that I would strongly discourage anyone from doing it.

I think of the initialization part of a declaration as being an assignment, that's why extending the = operator for this purpose feels natural to me.
But yes, I agree with your point. Without a function returning multiple values there is no advantage in mixing declarations and pure assignments in the same statement.

In any case, which implementation looks better si very subjective.

Indeed.

I hope we can find one alternative that's good enough for everyone to make a change.

Hopefully the steady stream of similar proposals over many years will convince the core team that there is a need for such a mechanism.

@go101
Copy link

go101 commented Apr 2, 2021

@pam4

As I noted above, nsf's suggestion is basically #30318 plus 2 points:

  1. Allow no new variables on the left side of :=
    this is just cosmetic, if there are no new variables you can use =.

Almost, but not for the declaration in if (oldV), newV := expr; condition {...}

  1. Eliminate := redeclaration
    this is important but it must wait for Go2 (not backward compatible).

What's the backward compatibility problem here?

Comparing to the "colon-prefix syntax" one, nsf's solution don't need to invent a syntax.
If we do need to invent a new syntax, I prefer .oldV to :oldV.

@pam4
Copy link

pam4 commented Apr 2, 2021

@go101

  1. Allow no new variables on the left side of :=
    this is just cosmetic, if there are no new variables you can use =.

Almost, but not for the declaration in if (oldV), newV := expr; condition {...}

I don't understand.
Your example is perfectly valid under #30318 alone, because there is one new variable (newV).
According to nsf's proposal, x.b, (err) := B() should be valid too, but he admits that it's just a matter of preference: if there are no new variables there's no need for a := in the first place.
I don't see the connection between my point (#2 in nsf's post) and your example, please explain.

  1. Eliminate := redeclaration
    this is important but it must wait for Go2 (not backward compatible).

What's the backward compatibility problem here?

Without redeclaration, a bare identifier on the LHS of a := would always cause a compile error if it was already declared in the same scope.

// this code currently compiles, but it should fail per nsf's proposal
a, ok := f()
if !ok {
    //...
}
b, ok := g() // ok is redeclared

// it should be written like this instead:
a, ok := f()
if !ok {
    //...
}
b, (ok) := g()

// under #30318 alone, both are supposed to work

Comparing to the "colon-prefix syntax" one, nsf's solution don't need to invent a syntax.
If we do need to invent a new syntax, I prefer .oldV to :oldV.

Just to be sure there's no misunderstanding, the "colon-prefix syntax" works with the = operator (not :=), and it's the new variables that are marked with :, not the old.
The idea is that the : in := is the part that means "declare", so we move it in front of the variables to declare.
I.e. :a, b = f() would correspond to a, (b) := f() (declare a and reuse b).

That said, the actual choice of symbol is not that important to me.

@go101
Copy link

go101 commented Apr 2, 2021

@pam4
I admit I did misunderstood your first point.

The intention of the other issue (#30318) is to allow non-pure-identifier items on the left of :=. I think, for code readability and easy-explanation, it is good to make all pure identifiers on the left of := new declared+initialized and non-pure-identifier items modified. The "colon-prefix syntax" :newV conflicts with this design.

There are some possible ways to go through:

  1. Remove the ... := ... syntax and allow non-pure-identifier items on the left of a variable declaration var ... = ..., and define var ... = ... as simple statement (otherwise there will be no ways to declare new variables at the beginning of if, switch and for code blocks).
  2. Keep the ... := ... syntax but call it as assignment and declaration (instead of the current redeclaration), and allow non-pure-identifier items (including (oldV)) on the left of :=.
    2.a. Still allow oldV on the left of :=
    2.b. Disallow oldV on the left of :=

The way #2.a will keep compatibility but it also keeps the confusion cases the current issue tries to avoid.
I admit the ways #1 and #2.b will break compatibility, but they both can be viewed as feature removal. In other words, the compatibility breaking is controllable with the help of language feature version.

@pam4
Copy link

pam4 commented Apr 2, 2021

@go101

The intention of the other issue (#30318) is to allow non-pure-identifier items on the left of :=. I think, for code readability and easy-explanation, it is good to make all pure identifiers on the left of := new declared+initialized and non-pure-identifier items modified. The "colon-prefix syntax" :newV conflicts with this design.

Sorry, you lost me again. The colon-prefix syntax doesn't use the := operator, so I don't see the conflict.
Of course it doesn't make sense to implement both, but either one or the other. If we go for the colon-prefix syntax there is no need for #30318 anymore (in fact there is no need for := anymore).

  1. Keep the ... := ... syntax but call it as assignment and declaration (instead of the current redeclaration), and allow non-pure-identifier items on the left of := and allow (oldV) in the left list.
    2.a. Still allow oldV on the left of :=
    2.b. Disallow oldV on the left of :=

The way #2.a will keep compatibility but it also keeps the confusion cases the current issue tries to avoid.

What you describe as #2.a is exactly #30318 (as nicely explained here).

I admit the ways #1 and #2.b will break compatibility, but they both can be viewed as feature removal. In other words, the compatibility breaking is controllable with the help of language feature version.

If we go for #30318, dropping redeclarations is the obvious next step (#2.b). I was just saying that we cannot have it before Go2 because it's a huge break and we have a compatibility promise in place.
OTOH the colon-prefix syntax has no compatibility problems (:= could be deprecated but maintained as long as it's convenient), and it has exactly the same advantages as #1 or #2.b.

@go101
Copy link

go101 commented Apr 3, 2021

@pam4

If we go for the colon-prefix syntax there is no need for #30318 anymore (in fact there is no need for := anymore).

Do you mean, if we go for the colon-prefix syntax there, the #30318 cases can be solved as

oldv1.field, oldV2,  :newV = f()

? If this is true, I think this is not an elegant design, for code readability and easy-explanation reasons, and it needs to invent a new expression syntax.

OTOH the colon-prefix syntax has no compatibility problems (:= could be deprecated but maintained as long as it's convenient),

So do you mean the new syntax will coexist with the old redeclartion syntax, to keep back compatibility? If this is true, it doesn't remove the confusion the current issue tries to remove. IMHO, #2.a is more elegant than this design for the same keep-compatibility goal, for no new expression syntax needs to be invented.

@pam4
Copy link

pam4 commented Apr 3, 2021

@go101, ok, it seems that we understood each other now.

oldV1.field, oldV2, :newV = f()   // colon-prefix syntax
oldV1.field, (oldV2), newV := f() // #30318

If this is true, I think this is not an elegant design, for code readability and easy-explanation reasons, and it needs to invent a new expression syntax.

  • Readability: I think it's more readable to mark variables that need to be declared than vice versa, but readability is very subjective and I respect other people views.
  • Easy-explanation: I think the colon-prefix syntax is easier to explain: "the declare operator (:) can be used in front of any bare identifier on the LHS of an assignment; it causes such identifier to be declared", that's all.
    OTOH, the "parenthesized identifier" trick is clever but not as straightforward: var1 and (var1) currently have the same effect on the LHS of an =, so you have to explain why on the LHS of a := they don't (not because parentheses have a special meaning but because a parenthesized identifier is not bare).
  • New syntax: yes, one simple operator (:) is introduced while a complex and problematic one (:=) is removed. I think it's a good trade-off.

So do you mean the new syntax will coexist with the old redeclartion syntax, to keep back compatibility? If this is true, it doesn't remove the confusion the current issue tries to remove.

No, the intent is for the := to be abandoned. I'm just saying that it could be done in 2 steps if necessary:

  1. the colon-prefix syntax is introduced; := is deprecated and not used in new code but old code still works; confusion is limited because := keeps the old familiar behavior; new code without := has all the advantages of #2.b;
  2. := is removed when possible.

The other solution can also be done in 2 steps (#30318 and then dropping redeclarations), but before the second step there is always the possibility of forgetting a () and using redeclarations accidentally.

Anyway, regardless of my preference I'm just arguing that the colon-prefix syntax is a viable alternative. I think your preference is still good and more likely to happen (because of the popularity of #30318). So it's not me that you have to convince, I would gladly introduce #30318 today and drop redeclarations in Go2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LanguageChange NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Proposal v2 A language change or incompatible library change
Projects
None yet
Development

No branches or pull requests