cmd/compile: internal compiler error: found illegal assignment on range with closure variables

[stamblerre]

I am unable to run my program, which I think is correct and can be found in https://golang.org/cl/327276.

Repro steps:

$ go get golang.org/dl/gotip
$ gotip download dev.typeparams
$ git clone https://go.googlesource.com/tools
$ cd tools
$ git fetch https://go.googlesource.com/tools refs/changes/76/327276/19 && git cherry-pick FETCH_HEAD
$ cd gopls
$ gotip run -gcflags=-G=3 api-diff/api_diff.go -prev=v0.7.0

Output:

# command-line-arguments
api-diff/api_diff.go:291:81: internal compiler error: found illegal assignment .shape.*uint8 -> main.T₁; 

goroutine 1 [running]:
runtime/debug.Stack()
	/Users/rstambler/sdk/gotip/src/runtime/debug/stack.go:24 +0x65
cmd/compile/internal/base.FatalfAt({0x0, 0x0}, {0x1903696, 0x27}, {0xc000892270, 0x3, 0x3})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/base/print.go:227 +0x154
cmd/compile/internal/base.Fatalf(...)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/base/print.go:196
cmd/compile/internal/noder.assignconvfn({0x1a5cc18, 0xc000a7c680}, 0xc00087ecb0)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/transform.go:413 +0x167
cmd/compile/internal/noder.typecheckaste(0x60, {0xc000a78c60, 0xc000a78c60}, 0x0, 0xc000a7c680, {0xc000a3ab30, 0x1, 0x11af9b2}, 0x0)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/transform.go:465 +0x186
cmd/compile/internal/noder.transformCall(0xc000a78c60)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/transform.go:160 +0x199
cmd/compile/internal/noder.(*subster).node.func1({0x1a5b638, 0xc000130480})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:1043 +0xed5
cmd/compile/internal/ir.editNodes({0xc000a6cf00, 0x2, 0x18cfa00}, 0xc000a727c8)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:1521 +0x74
cmd/compile/internal/ir.(*CallExpr).editChildren(0xc000a78bd0, 0xc000a727c8)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:276 +0xb4
cmd/compile/internal/ir.EditChildren(...)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:185
cmd/compile/internal/noder.(*subster).node.func1({0x1a5b638, 0xc000130510})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:901 +0x423
cmd/compile/internal/ir.editNodes({0xc000a3ab20, 0x1, 0x18c71e0}, 0xc000a727c8)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:1521 +0x74
cmd/compile/internal/ir.(*RangeStmt).editChildren(0xc000a78b40, 0xc000a727c8)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:1000 +0x16c
cmd/compile/internal/ir.EditChildren(...)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:185
cmd/compile/internal/noder.(*subster).node.func1({0x1a5cf38, 0xc0001305a0})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:901 +0x423
cmd/compile/internal/noder.(*subster).node(0xc000a70700, {0x1a5cf38, 0xc0001305a0})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:1184 +0xa5
cmd/compile/internal/noder.(*subster).namelist(0xc000a7e320, {0xc00051e900, 0x14, 0xc0007f3bf0})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:1255 +0xb7
cmd/compile/internal/noder.(*subster).node.func1({0x1a5b890, 0xc000137220})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:1073 +0x1132
cmd/compile/internal/ir.editNodes({0xc000a3aa40, 0x1, 0x18ca040}, 0xc000a72528)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:1521 +0x74
cmd/compile/internal/ir.(*ReturnStmt).editChildren(0xc000a7e280, 0x1a5d190)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:1056 +0x53
cmd/compile/internal/ir.EditChildren(...)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:185
cmd/compile/internal/noder.(*subster).node.func1({0x1a5d190, 0xc000177310})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:901 +0x423
cmd/compile/internal/noder.(*subster).node(0xc000a70700, {0x1a5d190, 0xc000177310})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:1184 +0xa5
cmd/compile/internal/noder.(*subster).list(0xc000692b48, {0xc000061660, 0x1, 0xc0001a25f0})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:1267 +0x8e
cmd/compile/internal/noder.(*irgen).genericSubst(0xc0001c4000, 0xc000a77e00, 0xc000109c70, {0xc000692aa8, 0x1, 0x1}, 0x0, 0xc000a4b440)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:714 +0xcc6
cmd/compile/internal/noder.(*irgen).getInstantiation(0xc0001c4000, 0xc000109c70, {0xc000692aa0, 0x1, 0x1}, 0x40)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:600 +0x33d
cmd/compile/internal/noder.(*irgen).stencil.func1({0x1a5b638, 0xc0006a5950})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:109 +0x2df
cmd/compile/internal/ir.Visit.func1({0x1a5b638, 0xc0006a5950})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:105 +0x30
cmd/compile/internal/ir.doNodes({0xc000765bf0, 0x3, 0xc0008859e0}, 0xc000a72318)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:1512 +0x67
cmd/compile/internal/ir.(*CompLitExpr).doChildren(0xc0001db400, 0xc000a72318)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:392 +0x99
cmd/compile/internal/ir.DoChildren(...)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:94
cmd/compile/internal/ir.Visit.func1({0x1a5ba20, 0xc0001db400})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:106 +0x57
cmd/compile/internal/ir.(*RangeStmt).doChildren(0xc0006a5b90, 0xc000a72318)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:972 +0x64
cmd/compile/internal/ir.DoChildren(...)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:94
cmd/compile/internal/ir.Visit.func1({0x1a5cf38, 0xc0006a5b90})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:106 +0x57
cmd/compile/internal/ir.doNodes({0xc00051e800, 0x9, 0x0}, 0xc000a72318)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/node_gen.go:1512 +0x67
cmd/compile/internal/ir.(*Func).doChildren(0x1a5bf98, 0xc00050a840)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/func.go:152 +0x2e
cmd/compile/internal/ir.DoChildren(...)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:94
cmd/compile/internal/ir.Visit.func1({0x1a5bf98, 0xc00050a840})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:106 +0x57
cmd/compile/internal/ir.Visit({0x1a5bf98, 0xc00050a840}, 0xc000a4b380)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/ir/visit.go:108 +0xb8
cmd/compile/internal/noder.(*irgen).stencil(0xc0001c4000)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/stencil.go:91 +0x26a
cmd/compile/internal/noder.(*irgen).generate(0xc0001c4000, {0xc0001909b0, 0x2, 0x203000})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/irgen.go:265 +0x272
cmd/compile/internal/noder.check2({0xc0001909b0, 0x2, 0x2})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/irgen.go:92 +0x16d
cmd/compile/internal/noder.LoadPackage({0xc0001a0120, 0x2, 0x0})
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/noder/noder.go:90 +0x365
cmd/compile/internal/gc.Main(0x1916670)
	/Users/rstambler/sdk/gotip/src/cmd/compile/internal/gc/main.go:190 +0xaf3
main.main()
	/Users/rstambler/sdk/gotip/src/cmd/compile/main.go:55 +0xdd

I'm sorry that I don't have a more minimal repro case, but I do believe that this used to work (a few weeks ago).

[findleyr]

CC @danscales @mdempsky @griesemer

[randall77]

Thanks for the test case.

@danscales, looks like something didn't get stenciled? It's trying to assign to a (unstenciled) parameter type.

p.s. We just merged to main, so using dev.typeparams is no longer necessary.

[randall77]

Standalone test case:

package main


func main() {
	d := diff([]int{}, func(int) string {
			return "foo"
	})
	d()
}

func diff[T any](previous []T, uniqueKey func(T) string) func() {
	return func() {
		newJSON := map[string]T{}
		for _, prev := range previous {
			delete(newJSON, uniqueKey(prev))
		}
	}
}

[danscales]

@stamblerre Thanks for the bug report, and @randall77 thanks for boiling it down to a simple test case.

The stencilling/substitution code needs some extra logic to deal with the n.Defn pointers of local/closure variables, in particular range variables. The n.Defn pointer of a range variable points to its entire RANGE statement, so we are trying to substitute that before we've finished substituting all the local vars. I probably needed to do the substitution of the Defn nodes in a second pass.

@stamblerre You can work around this by writing out the range statement as a standard i loop such as:

        for i := 0; i < len(previous); i++ {
                prev := previous[i]    
                ...
        }

That works in the simple example that Keith gives. Let me know if that doesn't work for you, and I'll accelerate the fix for this issue.

[fsouza]

Potential duplicate: #47724.

Test case:

package main

import "strconv"

func main() {
	ch := make(chan int, 10)
	for i := 0; i < 10; i++ {
		ch <- i
	}

	Map(ch, strconv.Itoa)
}

func Map[InputType, OutputType any](in <-chan InputType, f func(InputType) OutputType) <-chan OutputType {
	out := make(chan OutputType, cap(in))
	go func() {
		defer close(out)
		for v := range in {
			out <- f(v)
		}
	}()
	return out
}

[danscales]

Yes, thanks for the further report. This is another example of the current bug (relating to stenciling of range loops).

[gopherbot]

Change https://golang.org/cl/346290 mentions this issue: cmd/compile: fix handling of Defn field during stenciling