Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: additional internal debugging support for escape.go #31489

Open
mdempsky opened this issue Apr 16, 2019 · 9 comments

Comments

@mdempsky
Copy link
Member

commented Apr 16, 2019

esc.go and escape.go both support -m=1 for emitting diagnostics about escape analysis results, but esc.go additionally supported -m=2 and above for emitting diagnostics about internal code execution and decision making. escape.go should provide similar functionality for compiler maintainer use.

@mdempsky

This comment has been minimized.

Copy link
Member Author

commented Jul 8, 2019

Seems like it's worthwhile to start discussing this so this can be fixed during Go 1.14.

The straightforward solution is to just reimplement esc.go's -m=2 (and above) output, but I've always found that rather difficult to read and understand. Rather than just reimplementing that, I'd like to take a lesson from GOSSAFUNC and consider creating a more visual and interactive interface for viewing the escape analysis graph.

For example, I think escape.go could output a JSON (or whatever) description of the escape analysis graphs, including details about all nodes and edges and why they were added to the graph, along with details about escape.go's own analysis results. Then an HTML page could visualize the output, allow focusing on individual escaping locations, identify the paths that caused them to escape, cross-referencing with original source positions, etc.

@mdempsky

This comment has been minimized.

Copy link
Member Author

commented Jul 8, 2019

Here's a first draft of a JSON-compatible schema for describing escape analysis of a compilation unit.

type Package struct {
	Tool   string   `json:"tool"`  // creator; e.g., "cmd/compile 1.14"
	Files  []string `json:"files"` // source file paths
	Graphs []Graph  `json:"graphs"`
}

type Graph struct {
	Funcs []Func `json:"funcs"`
	Nodes []Node `json:"nodes"`
}

type Func struct {
	Pos     Pos    `json:"pos"`
	Name    string `json:"name"`    // linker name of function
	Params  []int  `json:"params"`  // indices into Graph.Nodes
	Results []int  `json:"results"` // indices into Graph.Nodes
}

type Node struct {
	Pos      Pos    `json:"pos"`
	Name     string `json:"name"` // user friendly description; not necessarily unique
	Func     int    `json:"func"` // index into Graph.Funcs, or -1 for heap
	Incoming []Edge `json:"incoming"`

	LoopDepth int  `json:"loopDepth"`
	Escapes   bool `json:"escapes"`
	Transient bool `json:"transient"`

	// ParamEsc summarizes the shortest dereference path from the
	// Node to the heap (ParamEsc[0]) or result parameter i
	// (ParamEsc[i]). A value of -1 indicates no path.
	ParamEsc []int `json:"paramEsc"`
}

type Edge struct {
	Pos    Pos    `json:"pos"`
	Why    string `json:"why"`    // compiler explanation for this edge
	Source int    `json:"source"` // index into Graph.Nodes

	Derefs int `json:"derefs"` // >= -1
}

type Pos struct {
	File   int `json:"file"`   // index into Package.Files
	Line   int `json:"line"`   // starting at 1
	Column int `json:"column"` // starting at 1 (byte count)
	// TODO(mdempsky): Inlining?
}

This is basically just a dump of escape.go's internal details.

@cherrymui

This comment has been minimized.

Copy link
Contributor

commented Jul 8, 2019

cc @dr2chase, who has been thinking about machine-readable compiler diagnostics.

@mdempsky

This comment has been minimized.

Copy link
Member Author

commented Jul 9, 2019

In https://github.com/mdempsky/go/tree/escape-debug, I've dumped some rough proof-of-concept code that supports dumping rudimentary escape analysis details as JSON, and then rendering the graph with js.cytoscape.org.

Basic usage is:

# Checkout and build
git clone -b escape-debug https://github.com/mdempsky/go go-escape-debug
cd go-escape-debug/src
./make.bash
export PATH=$PWD/../bin:$PATH

# Build net/http and extract JSON output to http.json
./dump-esc-json.sh net/http > http.json

# Open visualization (http.json is hard coded)
xdg-open analyze.html

Current functionality:

  • You can use the select box at the top-left to browse through all the different escape analysis graphs.
  • You can drag nodes around to make the graph easier to read.
  • You can click on nodes/edges to get details on the right. (Edges don't have position information currently, because it's not yet tracked by escape.go.)
  • Red nodes indicate an edge that escapes.
  • Edge labels are the number of dereferences associated with the assignment edge; -1 (which indicates an "address-of" operation) are highlighted in blue.

Edit: One-click demo: https://escape-debug-demo.web.app/

@gopherbot

This comment has been minimized.

Copy link

commented Jul 25, 2019

Change https://golang.org/cl/187617 mentions this issue: test: remove -newescape from regress tests

gopherbot pushed a commit that referenced this issue Aug 28, 2019
test: remove -newescape from regress tests
Prep for subsequent CLs to remove old escape analysis pass.

This CL removes -newescape=true from tests that use it, and deletes
tests that use -newescape=false. (For history, see CL 170447.)

Notably, this removes escape_because.go without any replacement, but
this is being tracked by #31489.

Change-Id: I6f6058d58fff2c5d210cb1d2713200cc9f501ca7
Reviewed-on: https://go-review.googlesource.com/c/go/+/187617
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
@ardan-bkennedy

This comment has been minimized.

Copy link

commented Aug 31, 2019

I am to believe that 1.13 will not contain the same level of information found in 1.12. Specicially this critical information that explains why an escape happened.

./stream.go:83:26: &bytes.Buffer literal escapes to heap
./stream.go:83:26: 	from ~R0 (assign-pair) at ./stream.go:83:26
./stream.go:83:26: 	from input (assigned) at ./stream.go:83:8
./stream.go:83:26: 	from input (interface-converted) at ./stream.go:93:26
./stream.go:83:26: 	from io.r (assign-pair) at ./stream.go:93:26
./stream.go:83:26: 	from io.r (passed to call[argument escapes]) at ./stream.go:93:26
@mdempsky

This comment has been minimized.

Copy link
Member Author

commented Aug 31, 2019

@ardan-bkennedy Yes, that's what this issue is about. :) esc.go reported those diagnostics, but escape.go does not (yet).

@ardan-bkennedy

This comment has been minimized.

Copy link

commented Aug 31, 2019

Thanks to @FiloSottile I learned I can turn off the new escape analysis in 1.13.

-gcflags "-newescape=false -m=2"

It's enough to get me through the time between now and version 1.14 when this gets fixed.

tomocy added a commit to tomocy/go that referenced this issue Sep 1, 2019
test: remove -newescape from regress tests
Prep for subsequent CLs to remove old escape analysis pass.

This CL removes -newescape=true from tests that use it, and deletes
tests that use -newescape=false. (For history, see CL 170447.)

Notably, this removes escape_because.go without any replacement, but
this is being tracked by golang#31489.

Change-Id: I6f6058d58fff2c5d210cb1d2713200cc9f501ca7
Reviewed-on: https://go-review.googlesource.com/c/go/+/187617
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
t4n6a1ka added a commit to t4n6a1ka/go that referenced this issue Sep 5, 2019
test: remove -newescape from regress tests
Prep for subsequent CLs to remove old escape analysis pass.

This CL removes -newescape=true from tests that use it, and deletes
tests that use -newescape=false. (For history, see CL 170447.)

Notably, this removes escape_because.go without any replacement, but
this is being tracked by golang#31489.

Change-Id: I6f6058d58fff2c5d210cb1d2713200cc9f501ca7
Reviewed-on: https://go-review.googlesource.com/c/go/+/187617
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
@gopherbot

This comment has been minimized.

Copy link

commented Sep 20, 2019

Change https://golang.org/cl/196619 mentions this issue: cmd/compile: cleanup escape graph construction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.