cmd/compile: global variable initialization done in unexpected order

[jcp19]

What version of Go are you using (go version)?

$ go version
go version go1.16.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/joao/Library/Caches/go-build"
GOENV="/Users/joao/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/joao/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/joao/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.16.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.16.4/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/2d/wcw2b3c57jz69cl5tg_s2fx00000gn/T/go-build3928827666=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version go1.16.4 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.16.4
uname -v: Darwin Kernel Version 21.3.0: Wed Jan  5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_X86_64
ProductName:	macOS
ProductVersion:	12.2.1
BuildVersion:	21D62
lldb --version: lldb-1103.0.22.10
Apple Swift version 5.2.4 (swiftlang-1103.0.32.9 clang-1103.0.32.53)

What did you do?

I have a package consisting of the following two files:

f1.go

package main    
   
var A int = 3    
var B int = A + 1    
var C int = A

f2.go

package main    
   
import "fmt"    
                     
var D = f()      
   
func f() int {    
  A = 1    
  return 1    
}    
   
func main() {    
  fmt.Println(A, B, C)    
}  

What did you expect to see?

According to the Go language specification, "package-level variable initialization proceeds stepwise, with each step selecting the variable earliest in declaration order which has no dependencies on uninitialized variables".

As such, I would expect two possible orders in which the global variables can be initialized:

1. A < B < C < D - happens when you compile the project by passing f1.go first to the compiler, followed by f2.go . In this case, the output is "1 4 3"
2. A < D < B < C - happens when f2.go is passed first to the compiler. In this case, the expected output would be "1 2 1".

What did you see instead?

For the second case (when f2.go is passed first), the actual output is "1 2 3". If instead I rewrite file f1.go to the following, I get the expected output for case 2.

Rewritten f2.go

package main    
   
import "fmt"    
   
var A int = initA()    
var B int = initB()    
var C int = initC()    
     
func initA() int {    
  fmt.Println("Init A")    
  return 3    
}    
     
func initB() int {    
  fmt.Println("Init B")    
  return A + 1    
}    
 
func initC() int {    
  fmt.Println("Init C")    
  return A    
} 

Output

Init A
Init B
Init C
1 2 1

Additional Information

This issue was first discussed in the golang-nuts Google Group (link).

[zigo101]

A little clarification: OP means the outputs are different between go run f1.go f2.go and go run f2.go f1.go. And after rewriting f1.go, things changes a bit.

By the specification, the outputs should be always 1 4 3.

[candlerb]

By the specification, the outputs should be always 1 4 3.

The two scenarios are:

• f1.go f2.go: declaration order is A B C D
• f2.go f1.go: declaration order is D A B C

In the first case it's simple:

• A is set to 3
• B is set to 4
• C is set to 3
• D is set to 1, and A is set to 1 as a side-effect

Expected result: 1 4 3

In the second case, I come to a different conclusion from reading the spec than you:

• D is not ready because it depends on A
• A is set to 3
• D is now "the next package-level variable that is earliest in declaration order and ready for initialization", so it gets set to 1, and A is set to 1 as a side-effect
• B is set to 2
• C is set to 1

Expected result: 1 2 1

That is, I read "next" to mean "the next variable to be initialized", not "the next variable following sequentially after the one which was last initialized". Is this an incorrect reading?

[mknyszek]

CC @golang/runtime

For additional context, see #31292. Unclear if this is a bug yet, but since it also involves the spec, CC @griesemer who was central to #31292.

[griesemer]

Edited. There appears to be a bug in the compiler. See #51913 (comment).

This is working as intended. Note that the spec also says:

The declaration order of variables declared in multiple files is determined by the order in which the files are presented to the compiler: Variables declared in the first file are declared before any of the variables declared in the second file, and so on.

We don't need multiple files, we can just arrange the variable declarations accordingly. In the first case:

package main

var A int = 3
var B int = A + 1
var C int = A
var D = f()

func f() int {
	A = 1
	return 1
}

func main() {
	println(A, B, C, D)
}

the output is

1 4 3 1

Here's the corresponding trace from the type checker's initialization order computation (this is the trace produced by types2 which is used by the compiler, but note that at the moment the compiler still uses its own init order computation and not the types2 computation - still they match):

Computing initialization order for package main ("main")

Object dependency graph:
        A has no dependencies
        B depends on
                A
        C depends on
                A
        D depends on
                f
        f depends on
                A
        main depends on
                A
                B
                C
                D

Transposed object dependency graph (functions eliminated):
        A depends on 0 nodes
                B is dependent
                C is dependent
                D is dependent
        C depends on 1 nodes
        B depends on 1 nodes
        D depends on 1 nodes

Processing nodes:
        A (src pos 1) depends on 0 nodes now
        B (src pos 2) depends on 0 nodes now
        C (src pos 3) depends on 0 nodes now
        D (src pos 4) depends on 0 nodes now

Initialization order:
        A = 3
        B = A + 1
        C = A
        D = f()

For the 2nd case:

package main

var D = f()

func f() int {
	A = 1
	return 1
}

func main() {
	println(A, B, C, D)
}

var A int = 3
var B int = A + 1
var C int = A

the output is

1 2 3 1

and the corresponding init computation trace is:

Computing initialization order for package main ("main")

Object dependency graph:
        D depends on
                f
        f depends on
                A
        main depends on
                A
                B
                C
                D
        A has no dependencies
        B depends on
                A
        C depends on
                A

Transposed object dependency graph (functions eliminated):
        A depends on 0 nodes
                D is dependent
                B is dependent
                C is dependent
        D depends on 1 nodes
        C depends on 1 nodes
        B depends on 1 nodes

Processing nodes:
        A (src pos 4) depends on 0 nodes now
        D (src pos 1) depends on 0 nodes now
        B (src pos 5) depends on 0 nodes now
        C (src pos 6) depends on 0 nodes now

Initialization order:
        A = 3
        D = f()
        B = A + 1
        C = A

Thus, in this case D gets initialized before B because it's before B in the source. This explains the difference.

Closing.

[aarzilli]

In the second case, if f runs before the assignemnt to C shouldn't that see the side effect of the call to f?

[griesemer]

@aarzilli Good catch, I totally glanced over this. Indeed go/types and types2 compute the correct initialization order (currently not used by the compiler), while the compiler still has a bug here, I think. Re-opening.

For the 1st case:

1. A is set to 3. Variable state is: A = 3, B = 0, C = 0, D = 0.
2. B is set to A + 1. Variable state is: A = 3, B = 4, C = 0, D = 1.
3. C is set to A. Variable state is: A = 3, B = 4, C = 3, D = 0.
4. D is set to 1 (result of f()), and A is set to 1. Variable state is: A = 1, B = 4, C = 3, D = 1.

For the 2nd case:

1. A is set to 3. Variable state is: A = 3, B = 0, C = 0, D = 0.
2. D is set to 1 (result of f()), and A is set to 1. Variable state is: A = 1, B = 0, C = 0, D = 1.
3. B is set to A + 1. Variable state is: A = 1, B = 2, C = 0, D = 1.
4. C is set to A. Variable state is: A = 1, B = 2, C = 1, D = 1.