Repository navigation
cmd/compile: rewrite escape analysis #23109
Description
Activity
I have no idea what a "maximum-copy-started-suffix-level" is, and the x.left.left, &Node{x}, etc. examples don't really help. I'm particularly confused because a Node can represent so many different operations depending on the Op, and I'm lost on what semantic meaning we're divining from syntactic tree depth.
If I understand correctly, the example assumes there is a type
Nodewith a fieldleftwhich is of type*Node, in the source code, instead of the Node representation of the AST in the compiler. This is particularly confusing because I think this refers to source code but uses a data type that is the same as the compiler's AST data type.escAnalyze has a reflooding loop that keeps looping until fixed point. It's not obvious to me why this is necessary. (At least experimentally, I do see that "Reflooding foo" shows up when building the standard library with -m=3.)
The reflooding is added in CL https://go-review.googlesource.com/c/go/+/30693. Its CL description explains it. Probably we should add this as a comment.
Change https://golang.org/cl/121001 mentions this issue:
cmd/compile: improve escape analysis explanation- added a commit that references this issue
on Jun 26, 2018 Over the weekend, I reimplemented a basic variant of esc.go's escape analysis logic, and now I better understand how it works in practice and what the documentation is explaining.
One thing I still don't understand though is the handling of recursive functions. In particular, this bit:
// in a mutually recursive group we lose track of the return values if e.recursive { for _, ln := range Curfn.Func.Dcl { if ln.Op == ONAME && ln.Class() == PPARAMOUT { e.escflows(&e.theSink, ln, e.stepAssign(nil, ln, ln, "returned from recursive function")) } } }It looks like in esccall that we handle recursive calls, including wiring their return values into the flow graph correctly (look for "function in same mutually recursive group. Incorporate into flow graph").
I tried disabling this code by adding
recursive = falseat the top of escAnalyze, and it somewhat changes the-moutput (which does cause test/escape_because.go's f8 and f9 to fail), but it doesn't seem to change which variables actually need to be heap allocated.This code comes from CL 6741044 (507fcf3). It's true that before that CL we lost track of the return values in mutually recursive function calls, but the CL appears to have added both. So maybe the "if e.recursive" logic was written first, then lvd@ decided to fix the return value tracking, but forgot to remove the original (now unnecessary) fix?
When I was working on this and thought I understood it, the recursive punt seemed necessary.
A test case demonstrating this would be tricky, however.I also don't know that this is that worthwhile; I took a stab at handling recursion and even had a CL for it that I somewhat trusted, and it avoided practically no heap allocations, so I didn't pursue it. The problem is that if anything escapes, the this-field-versus-that-field precision is lacking, and so everything escapes.
I am however very interested in any writeup you can put together, because the algorithm is a real pain to understand.
So here's the basic idea I understand:
Escape analysis treats the control flow graph as a bunch of locations, and assignments between them. Individual assignments can involve addressing or an arbitrary number of dereferences. For example:
p = &q // -1 p = q // 0 p = *q // 1 p = **q // 2 ...Note that
p = &&qis nonsensical as values can't be addressed, so the length of an edge can never be less than -1.Assignments can also be to the "sink" (basically, the Go heap).
All Go language constructs can be lowered into this assignment graph. For example,
var x struct { f, g *int } var u []*int x.f = u[0]can be represented as
x = *uAssignments to global variables, through pointers, passing arguments to unknown functions, etc. are recorded as assignments to the sink. Also, if q has a greater loop depth than p, then
p = &qneeds to be treated assink = &q.This is an imprecise but conservative representation of data flow.
As an optimization, the Go source might lower into multiple edges between the same two locations; it's only necessary to track the shortest edge.
Once we have a graph representing one or more functions, we can walk the graph to compute each node's "distance" from the sink and from any return parameters. This is basically just a bunch of shortest-path computations, except that distance is bounded as non-negative. For example, if p has distance 0 from some location, and there's an edge
p = &q, then q has distance 0 from that location too, not distance -1.Finally, we can observe two things from the resulting distance computations:
-
If there's an edge
p = &qwherephas distance 0 from either the sink or any return parameter, thenqescapes (i.e., must be heap allocated). -
For each function
func f(p1, p2, ..., pN) (r1, r2, ..., rM)we can record the (M+1)*N distances from (sink, r1, r2, ...) to (p1, p2, ...) for subsequent escape analysis passes that involve calls to f. (The parameter tags that esc.go generates are basically a quantized encoding of this information.)
--
Notably, esc.go doesn't exactly follow this approach. Instead of simply constructing a graph and walking it as I describe above, escassign/escflow construct a partial graph (synthesizing OADDR/ODEREF nodes in lieu of tracking edge lengths directly) and escflood/escwalk walk the remaining edges. The subsequent computations based on the distance calculations are also intermingled into the walking logic, and tracking/recording distances is much more complicated.
Reacted by vsdmars (Shuo-Huan Chang), HyeJong, tandr and Danilo Cianfrone-
That's a nicer description than mine, I think. I am still trying to figure out an intuitive explanation of "suffixValue".
As best I can tell, the suffixValue logic is captured in my simplified description by having distances bottom out at 0.
Basically, it's to ensure that given
sink = &p p = qthen p needs to be heap allocated, but q doesn't.
As an update, I have a WIP branch where I've written a new escape analysis pass. You can see the new code here: https://github.com/mdempsky/go/blob/esc5/src/cmd/compile/internal/gc/esc2.go
It's still a little rough and I need to add documentation and better debugging facilities. (The latter being mostly ad hocly developed while I bring it up to feature parity with esc.go.)
However, I believe it's about on par with esc.go in terms of correctness, and it's less than half the number of lines of code (1117 vs 2425).
It roughly follows the description I made above: the stmts/stmt/value/assign/call methods all trudge through the AST tracking pointer flows and constructing a graph of EscLocations, and then flood+walk do a simple walk over the graph identifying locations that escape and value flows from pointers to result parameters.
The graphs it builds also tend to be much smaller than esc.go's. For example:
# net/http total high watermark locations: 5172646 10126 // esc2.go's nodes edges: 4784956 9577 // esc2.go's edges state: 12479460 25101 // esc.go's nodes flow: 5826852 11793 // esc.go's edgesesc2.go only needed about 40% as many graph nodes as esc.go, and about 80% as many edges. There's room for further improvement here too.
My next immediate goal is continuing to dig through the differences in behavior between esc.go and esc2.go, and then polishing it up for review for the next dev cycle.
8 remaining items
Change https://golang.org/cl/170321 mentions this issue:
cmd/compile: trim more unnecessary escape analysis messagesChange https://golang.org/cl/170322 mentions this issue:
cmd/compile: rewrite escape analysis- added a commit that references this issue
on Apr 2, 2019 I've uploaded CL 170322, which is in good enough shape for people to try out, if they're interested.
Here are some compilebench numbers:
name old time/op new time/op delta Template 250ms ± 2% 252ms ± 7% ~ (p=0.971 n=10+10) Unicode 115ms ± 3% 115ms ± 8% ~ (p=0.661 n=9+10) GoTypes 880ms ± 3% 848ms ± 3% -3.70% (p=0.000 n=10+10) Compiler 3.79s ± 3% 3.52s ± 3% -7.06% (p=0.000 n=10+10) SSA 10.0s ± 1% 10.0s ± 1% ~ (p=0.573 n=8+10) Flate 148ms ± 8% 149ms ± 5% ~ (p=0.905 n=10+9) GoParser 188ms ± 9% 182ms ± 6% ~ (p=0.315 n=10+10) Reflect 518ms ± 4% 522ms ± 3% ~ (p=0.536 n=9+7) Tar 221ms ± 4% 224ms ± 7% ~ (p=0.601 n=7+10) XML 298ms ± 6% 303ms ± 5% ~ (p=0.237 n=8+10) name old user-time/op new user-time/op delta Template 336ms ± 6% 330ms ±10% ~ (p=0.280 n=10+10) Unicode 169ms ± 9% 169ms ± 5% ~ (p=0.968 n=9+10) GoTypes 1.18s ± 3% 1.14s ± 6% -3.38% (p=0.001 n=10+10) Compiler 4.82s ± 3% 4.47s ± 2% -7.12% (p=0.000 n=10+8) SSA 12.9s ± 3% 12.8s ± 2% ~ (p=0.247 n=10+10) Flate 185ms ± 9% 183ms ±12% ~ (p=0.549 n=9+10) GoParser 237ms ± 7% 230ms ± 7% ~ (p=0.190 n=10+10) Reflect 679ms ± 7% 666ms ± 7% ~ (p=0.447 n=9+10) Tar 283ms ± 8% 286ms ± 6% ~ (p=0.579 n=10+10) XML 391ms ± 2% 392ms ± 5% ~ (p=0.661 n=9+10) name old alloc/op new alloc/op delta Template 39.0MB ± 0% 38.6MB ± 0% -1.04% (p=0.000 n=10+10) Unicode 28.3MB ± 0% 28.3MB ± 0% -0.08% (p=0.000 n=10+10) GoTypes 132MB ± 0% 131MB ± 0% -0.77% (p=0.000 n=10+9) Compiler 625MB ± 0% 619MB ± 0% -0.97% (p=0.000 n=10+10) SSA 2.04GB ± 0% 2.00GB ± 0% -2.11% (p=0.000 n=10+10) Flate 24.2MB ± 0% 24.0MB ± 0% -1.05% (p=0.000 n=10+10) GoParser 29.1MB ± 0% 28.8MB ± 0% -1.19% (p=0.000 n=10+10) Reflect 84.6MB ± 0% 83.5MB ± 0% -1.24% (p=0.000 n=10+10) Tar 36.9MB ± 0% 36.5MB ± 0% -1.05% (p=0.000 n=9+10) XML 48.4MB ± 0% 47.7MB ± 0% -1.43% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Template 382k ± 0% 380k ± 0% -0.60% (p=0.000 n=9+10) Unicode 341k ± 0% 341k ± 0% -0.05% (p=0.000 n=10+10) GoTypes 1.36M ± 0% 1.36M ± 0% -0.20% (p=0.000 n=10+9) Compiler 5.73M ± 0% 5.69M ± 0% -0.60% (p=0.000 n=10+10) SSA 16.9M ± 0% 16.6M ± 0% -1.49% (p=0.000 n=10+9) Flate 237k ± 0% 235k ± 0% -0.95% (p=0.000 n=10+10) GoParser 302k ± 0% 302k ± 0% -0.18% (p=0.000 n=10+10) Reflect 986k ± 0% 976k ± 0% -0.95% (p=0.000 n=10+10) Tar 356k ± 0% 353k ± 0% -0.80% (p=0.000 n=8+10) XML 441k ± 0% 437k ± 0% -1.09% (p=0.000 n=10+10)And here's a list of expressions that esc.go heap allocated, but that escape.go stack allocates:
Escape analysis improvements (350 lines)
archive/tar/reader.go:519:3: moved to heap: cntNewline archive/tar/reader.go:520:3: moved to heap: buf archive/tar/reader.go:526:16: func literal escapes to heap archive/tar/reader.go:543:15: func literal escapes to heap cmd/cgo/gcc.go:175:6: moved to heap: conv cmd/cgo/out.go:420:20: []*ast.Field literal escapes to heap cmd/compile/internal/gc/alg.go:334:32: []*Node literal escapes to heap cmd/compile/internal/gc/alg.go:337:12: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:173:33: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:173:62: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:176:33: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:179:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:179:63: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:181:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:183:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:185:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:187:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:189:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:191:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:193:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:195:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:196:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:199:106: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:199:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:200:127: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:200:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:201:148: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:201:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:202:169: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:202:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:204:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:204:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:205:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:205:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:208:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:208:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:210:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:210:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:211:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:211:64: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:214:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:214:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:215:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:215:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:218:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:218:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:220:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:220:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:221:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:221:83: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:222:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:222:85: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:223:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:223:64: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:224:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:224:83: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:226:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:226:63: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:227:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:227:83: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:228:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:228:83: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:229:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:230:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:232:106: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:232:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:234:39: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:236:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:236:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:237:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:237:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:238:39: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:239:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:239:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:240:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:240:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:241:124: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:241:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:242:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:242:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:243:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:243:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:244:124: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:244:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:245:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:246:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:247:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:248:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:250:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:250:84: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:251:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:251:84: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:253:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:254:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:254:84: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:256:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:258:29: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:259:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:260:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:261:103: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:261:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:262:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:262:84: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:263:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:263:84: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:265:105: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:265:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:266:104: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:266:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:267:105: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:267:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:268:105: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:268:34: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:270:107: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:270:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:271:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:272:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:273:105: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:273:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:274:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:274:84: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:275:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:275:86: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:276:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:276:86: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:277:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:277:65: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:278:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:278:65: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:279:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:279:65: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:280:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:280:65: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:281:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:281:65: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:282:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:282:65: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:283:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:283:86: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:284:35: []*Node literal escapes to heap cmd/compile/internal/gc/builtin.go:285:35: []*Node literal escapes to heap cmd/compile/internal/gc/closure.go:357:19: []*Node literal escapes to heap cmd/compile/internal/gc/closure.go:506:23: []*Node literal escapes to heap cmd/compile/internal/gc/dcl.go:743:17: []*Node literal escapes to heap cmd/compile/internal/gc/inl.go:1076:28: moved to heap: s cmd/compile/internal/gc/lex.go:108:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/lex.go:122:26: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/lex.go:127:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/lex.go:134:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/lex.go:142:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/lex.go:150:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/noder.go:1448:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/noder.go:1460:25: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/noder.go:1471:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/noder.go:1483:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/noder.go:1486:24: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/noder.go:47:25: "cmd/compile/internal/syntax".Error literal escapes to heap cmd/compile/internal/gc/plive.go:1379:13: func literal escapes to heap cmd/compile/internal/gc/reflect.go:1589:33: []*Node literal escapes to heap cmd/compile/internal/gc/reflect.go:1589:70: []*Node literal escapes to heap cmd/compile/internal/gc/select.go:386:27: []*Node literal escapes to heap cmd/compile/internal/gc/sinit.go:717:7: moved to heap: k cmd/compile/internal/gc/sinit.go:718:15: func literal escapes to heap cmd/compile/internal/gc/sinit.go:731:15: func literal escapes to heap cmd/compile/internal/gc/ssa.go:3151:19: func literal escapes to heap cmd/compile/internal/gc/ssa.go:3292:20: func literal escapes to heap cmd/compile/internal/gc/ssa.go:3500:24: func literal escapes to heap cmd/compile/internal/gc/subr.go:1604:32: []*Node literal escapes to heap cmd/compile/internal/gc/subr.go:1608:12: []*Node literal escapes to heap cmd/compile/internal/gc/swt.go:230:7: moved to heap: s cmd/compile/internal/gc/swt.go:492:16: []*Node literal escapes to heap cmd/compile/internal/gc/walk.go:3067:33: []*Node literal escapes to heap cmd/compile/internal/gc/walk.go:3070:13: []*Node literal escapes to heap cmd/compile/internal/gc/walk.go:817:15: func literal escapes to heap cmd/compile/internal/ssa/dom.go:110:11: func literal escapes to heap cmd/compile/internal/ssa/rewrite.go:564:19: func literal escapes to heap cmd/compile/internal/syntax/branches.go:132:66: moved to heap: lstmt cmd/compile/internal/syntax/printer.go:665:12: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:691:12: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:711:12: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:723:11: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:737:11: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:746:11: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:756:11: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:822:12: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:826:13: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:853:12: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:863:12: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:873:10: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:875:11: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:881:12: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:893:11: ... argument escapes to heap cmd/compile/internal/syntax/printer.go:905:11: ... argument escapes to heap cmd/fix/cftype.go:43:25: &TypeConfig literal escapes to heap cmd/fix/typecheck.go:149:10: &TypeConfig literal escapes to heap cmd/fix/typecheck.go:356:12: func literal escapes to heap cmd/gofmt/rewrite.go:59:11: make(map[string]reflect.Value) escapes to heap cmd/gofmt/rewrite.go:60:24: moved to heap: reflect.i cmd/gofmt/rewrite.go:61:25: moved to heap: reflect.i cmd/gofmt/rewrite.go:63:6: moved to heap: rewriteVal cmd/gofmt/rewrite.go:64:15: func literal escapes to heap cmd/go/internal/dirhash/hash.go:60:6: moved to heap: files cmd/go/internal/dirhash/hash.go:62:28: func literal escapes to heap cmd/go/internal/get/get.go:161:6: moved to heap: stk cmd/go/internal/get/get.go:235:11: func literal escapes to heap cmd/go/internal/get/vcs.go:862:14: func literal escapes to heap cmd/go/internal/modfetch/codehost/git.go:566:2: moved to heap: data cmd/go/internal/modfetch/codehost/git.go:571:10: func literal escapes to heap cmd/go/internal/modfetch/coderepo.go:242:19: func literal escapes to heap cmd/go/internal/modfetch/coderepo.go:498:21: &zip.Writer literal escapes to heap cmd/go/internal/modfetch/unzip.go:143:6: moved to heap: dirs cmd/go/internal/modfetch/unzip.go:144:21: func literal escapes to heap cmd/go/internal/modfetch/unzip.go:163:21: func literal escapes to heap cmd/go/internal/modfetch/unzip.go:52:18: make(map[string]string) escapes to heap cmd/go/internal/modfetch/unzip.go:53:6: moved to heap: checkFold cmd/go/internal/modfetch/unzip.go:54:14: func literal escapes to heap cmd/go/internal/modfile/read.go:708:6: moved to heap: sym cmd/go/internal/modload/load.go:72:19: func literal escapes to heap cmd/go/internal/modload/load.go:756:12: func literal escapes to heap cmd/go/internal/modload/search.go:23:11: func literal escapes to heap cmd/go/internal/modload/search.go:24:18: func literal escapes to heap cmd/go/internal/modload/search.go:30:25: map[string]bool literal escapes to heap cmd/go/internal/modload/search.go:36:6: moved to heap: pkgs cmd/go/internal/modload/search.go:40:23: func literal escapes to heap cmd/go/internal/mvs/mvs.go:164:6: moved to heap: postorder cmd/go/internal/mvs/mvs.go:165:49: map[module.Version][]module.Version literal escapes to heap cmd/go/internal/mvs/mvs.go:167:6: moved to heap: walk cmd/go/internal/mvs/mvs.go:168:9: func literal escapes to heap cmd/go/internal/mvs/mvs.go:193:27: map[string]string literal escapes to heap cmd/go/internal/mvs/mvs.go:194:9: func literal escapes to heap cmd/go/internal/search/search.go:166:21: func literal escapes to heap cmd/go/internal/search/search.go:36:11: func literal escapes to heap cmd/go/internal/search/search.go:36:2: moved to heap: match cmd/go/internal/search/search.go:37:18: func literal escapes to heap cmd/go/internal/search/search.go:37:2: moved to heap: treeCanMatch cmd/go/internal/search/search.go:43:25: map[string]bool literal escapes to heap cmd/go/internal/search/search.go:50:9: moved to heap: src cmd/go/internal/search/search.go:59:23: func literal escapes to heap cmd/go/internal/test/test.go:874:26: []string literal escapes to heap cmd/go/internal/test/test.go:878:27: []string literal escapes to heap cmd/go/internal/web/http.go:77:11: func literal escapes to heap cmd/go/internal/webtest/test.go:182:22: new(bufio.Reader) escapes to heap cmd/go/internal/webtest/test.go:183:6: moved to heap: ungetLine cmd/go/internal/webtest/test.go:184:14: func literal escapes to heap cmd/go/internal/webtest/test.go:97:37: &respEntry literal escapes to heap cmd/go/internal/work/action.go:658:75: moved to heap: a1 cmd/go/internal/work/action.go:661:48: func literal escapes to heap cmd/go/internal/work/action.go:740:50: func literal escapes to heap cmd/go/internal/work/action.go:759:27: []string literal escapes to heap cmd/go/internal/work/action.go:761:27: []string literal escapes to heap cmd/go/internal/work/exec.go:532:14: func literal escapes to heap cmd/go/internal/work/gccgo.go:222:24: []string literal escapes to heap cmd/go/internal/work/gccgo.go:222:2: moved to heap: cgoldflags cmd/go/internal/work/gccgo.go:233:18: func literal escapes to heap cmd/go/internal/work/gccgo.go:262:2: moved to heap: newID cmd/go/internal/work/gccgo.go:263:27: func literal escapes to heap cmd/go/internal/work/gc.go:266:15: func literal escapes to heap cmd/internal/obj/wasm/wasmobj.go:138:13: func literal escapes to heap cmd/link/internal/ld/data.go:1602:21: func literal escapes to heap cmd/link/internal/ld/data.go:1607:21: func literal escapes to heap cmd/link/internal/ld/dwarf.go:1558:22: []*sym.Symbol literal escapes to heap cmd/link/internal/ld/lib.go:1316:18: func literal escapes to heap cmd/link/internal/ld/symtab.go:397:14: func literal escapes to heap cmd/link/internal/loadelf/ldelf.go:463:12: func literal escapes to heap cmd/link/internal/loadmacho/ldmacho.go:437:12: func literal escapes to heap cmd/link/internal/loadxcoff/ldxcoff.go:44:12: func literal escapes to heap cmd/trace/pprof.go:176:14: make(map[uint64]Record) escapes to heap cmd/trace/pprof.go:195:14: make(map[uint64]Record) escapes to heap cmd/trace/pprof.go:223:14: make(map[uint64]Record) escapes to heap cmd/trace/pprof.go:243:14: make(map[uint64]Record) escapes to heap cmd/trace/trace.go:541:16: make(map[uint64]*gInfo) escapes to heap cmd/vendor/github.com/google/pprof/internal/driver/commands.go:256:13: func literal escapes to heap cmd/vendor/github.com/google/pprof/internal/elfexec/elfexec.go:127:17: func literal escapes to heap cmd/vendor/github.com/google/pprof/internal/report/report.go:271:15: func literal escapes to heap cmd/vendor/github.com/google/pprof/profile/legacy_profile.go:585:84: moved to heap: value cmd/vendor/github.com/google/pprof/profile/legacy_profile.go:585:99: moved to heap: blocksize cmd/vendor/github.com/google/pprof/profile/legacy_profile.go:593:15: func literal escapes to heap cmd/vendor/github.com/ianlancetaylor/demangle/demangle.go:498:13: func literal escapes to heap cmd/vendor/golang.org/x/arch/arm64/arm64asm/plan9x.go:26:13: func literal escapes to heap cmd/vendor/golang.org/x/arch/arm/armasm/plan9x.go:27:13: func literal escapes to heap cmd/vendor/golang.org/x/arch/ppc64/ppc64asm/plan9.go:20:13: func literal escapes to heap cmd/vendor/golang.org/x/arch/x86/x86asm/gnu.go:21:13: func literal escapes to heap cmd/vendor/golang.org/x/arch/x86/x86asm/intel.go:15:13: func literal escapes to heap cmd/vendor/golang.org/x/arch/x86/x86asm/plan9x.go:23:13: func literal escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/passes/httpresponse/httpresponse.go:73:28: asg.Lhs[0] escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/passes/httpresponse/httpresponse.go:82:29: def.Call.Fun escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel/lostcancel.go:210:10: func literal escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel/lostcancel.go:233:14: make(map[*cfg.Block]bool) escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel/lostcancel.go:234:15: func literal escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel/lostcancel.go:273:14: make(map[*cfg.Block]bool) escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel/lostcancel.go:274:6: moved to heap: search cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel/lostcancel.go:275:11: func literal escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/unitchecker/unitchecker.go:286:10: func literal escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/validate.go:17:15: make(map[string]bool) escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/validate.go:20:19: make(map[reflect.Type]*Analyzer) escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/validate.go:29:15: make(map[*Analyzer]uint8) escapes to heap cmd/vendor/golang.org/x/tools/go/analysis/validate.go:30:6: moved to heap: visit cmd/vendor/golang.org/x/tools/go/analysis/validate.go:31:10: func literal escapes to heap cmd/vendor/golang.org/x/tools/go/types/objectpath/objectpath.go:228:15: make([]byte, 0, 48) escapes to heap crypto/x509/x509.go:1178:73: moved to heap: unhandled crypto/x509/x509.go:1213:15: func literal escapes to heap crypto/x509/x509.go:1834:16: func literal escapes to heap crypto/x509/x509.go:1842:27: func literal escapes to heap debug/dwarf/type.go:304:5: moved to heap: err debug/dwarf/type.go:319:7: moved to heap: typedefList debug/dwarf/type.go:333:2: moved to heap: nextDepth debug/dwarf/type.go:336:10: func literal escapes to heap debug/dwarf/type.go:374:12: func literal escapes to heap debug/dwarf/typeunit.go:99:48: make(map[Offset]Type) escapes to heap debug/elf/file.go:1131:17: func literal escapes to heap debug/macho/file.go:591:17: func literal escapes to heap debug/pe/file.go:236:17: func literal escapes to heap encoding/gob/decode.go:1121:47: make(map[typeId]*decOp) escapes to heap encoding/xml/typeinfo.go:239:3: moved to heap: f go/printer/nodes.go:1503:20: "expected n = 1; got" escapes to heap go/printer/nodes.go:1503:20: n escapes to heap go/printer/nodes.go:752:20: "depth < 1:" escapes to heap go/printer/nodes.go:752:20: depth escapes to heap go/printer/nodes.go:86:19: "setComment found pending comments" escapes to heap go/printer/printer.go:789:18: "intersperseComments called without pending comments" escapes to heap go/types/assignments.go:313:16: ... argument escapes to heap go/types/builtins.go:551:13: ... argument escapes to heap go/types/decl.go:422:32: []ast.Expr literal escapes to heap go/types/expr.go:1255:14: ... argument escapes to heap go/types/expr.go:1278:13: ... argument escapes to heap go/types/expr.go:1348:13: ... argument escapes to heap go/types/expr.go:1408:34: []ast.Expr literal escapes to heap go/types/stmt.go:239:3: moved to heap: res internal/trace/parser.go:591:18: func literal escapes to heap internal/xcoff/ar.go:104:23: func literal escapes to heap math/big/sqrt.go:129:15: new(Float) escapes to heap math/big/sqrt.go:130:15: new(Float) escapes to heap math/big/sqrt.go:131:8: func literal escapes to heap math/big/sqrt.go:92:10: new(Float) escapes to heap math/big/sqrt.go:93:8: func literal escapes to heap net/http/client.go:514:21: moved to heap: req net/http/client.go:528:3: moved to heap: reqs net/http/client.go:529:3: moved to heap: resp net/http/client.go:531:3: moved to heap: reqBodyClosed net/http/client.go:537:10: func literal escapes to heap net/http/fs.go:154:14: func literal escapes to heap net/http/fs.go:621:14: func literal escapes to heap net/http/h2_bundle.go:4271:14: make(http2gate) escapes to heap net/http/h2_bundle.go:7479:28: func literal escapes to heap net/http/transport.go:1212:51: moved to heap: cm net/interface_linux.go:254:11: make([]byte, IPv6len) escapes to heap net/ipsock.go:263:14: func literal escapes to heap reflect/type.go:2017:16: func literal escapes to heap reflect/type.go:2619:16: func literal escapes to heap runtime/pprof/pprof.go:410:9: func literal escapes to heapThere are no instances in the main repo where a variable used to be stack allocated, but now heap allocates. (In particular, the regressions that were discussed on golang-dev@ have all been fixed.)
I need to figure out why the js/wasm bot is failing, and polish the documentation further. Then I'll prepare for review.
Reacted by Justin Clift, Hajime Hoshi, Gergely Bräutigam, Sławomir Kur and Tamás GulácsiReacted by Marcelo Pires, reusee, Andrew Pogrebnoi, Justin Nuß, Artyom Pervukhin, Ainar Garipov, Michael Munday, Aliaksandr Valialkin, George Moschovitis, theGeekPirate and 8 moreChange https://golang.org/cl/170323 mentions this issue:
runtime/internal/atomic: fix wasm's StorepNoWB implementation- added a commit that references this issue
on Apr 2, 2019 Change https://golang.org/cl/170447 mentions this issue:
cmd/compile: update escape analysis tests for newescapeChange https://golang.org/cl/170448 mentions this issue:
cmd/compile: enable -newescape by defaultChange https://golang.org/cl/170450 mentions this issue:
test: add regress tests for unsafe.Pointer rules- added a commit that references this issue
on Apr 3, 2019 - added 2 commits that reference this issue
on Apr 15, 2019 - locked and limited conversation to collaborators
on Apr 15, 2020
Despite repeated efforts, I'm still confused by our escape analysis code. Also, based on discussion with other compiler devs, apparently this difficulty understanding is not unique to me. It would be good to better document this code so we can hopefully simplify and improve it.
Some specific questions I've had trying to understand the code:
The "Escape analysis" documentation has a very high-level explanation, but leaves me with more questions than answers:
It mixes discussion of "nodes" and "variables," which makes it unclear to me whether expressions other than ONAME Nodes are present in the flow graph. (I think they are, otherwise I can't make sense of how we know if
new(T)can be stack allocated or not.) It also talks about "values" having addresses, when only variables do. Using terminology consistent with the Go spec would help, IMO.It mentions storing edges between "pointer-containing nodes". It would be nice to understand how this is expected to work in the presence of
unsafe.Pointer<->uintptrconversions.Pseudo-code like
flow(closure, &var)is confusing too: what's the&here supposed to signify?I can't make sense of the phrase "tags all variables of it can reach an & node as escaping".
The Level godocs are super opaque to me:
I kind of understand the meaning of
value, but I don't completely understand why adding the +1 and -1 values is sound.I have no idea what a "maximum-copy-started-suffix-level" is, and the
x.left.left,&Node{x}, etc. examples don't really help. I'm particularly confused because aNodecan represent so many different operations depending on theOp, and I'm lost on what semantic meaning we're divining from syntactic tree depth.escAnalyze has a reflooding loop that keeps looping until fixed point. It's not obvious to me why this is necessary. (At least experimentally, I do see that "Reflooding foo" shows up when building the standard library with -m=3.)
/cc @dr2chase @cherrymui @paranoiacblack