cmd/compile/internal/devirtualize: PGO devirtualization selects an edge with a weight of 0 as the hottest one #72092

yukatan1701 · 2025-03-04T14:16:49Z

Go version

go version devel go1.25-4f45b2b7e0 Tue Mar 4 03:10:17 2025 -0800 linux/arm64

Output of `go env` in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/jlapenko/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/jlapenko/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2889077743=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='arm64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/home/jlapenko/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/jlapenko/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/jlapenko/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/jlapenko/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/jlapenko/go/pkg/tool/linux_arm64'
GOVCS=''
GOVERSION='devel go1.25-4f45b2b7e0 Tue Mar 4 03:10:17 2025 -0800'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Consider the following example (it is based on the function (*Resolver).resolveAddrList from net/dial.go):

// file zero_test.go

package zero_weight

import "testing"

type Addr interface {
	Network() string
}

type UnixAddr struct {
	Net string
}

type NotUnixAddr struct {
	Net string
}

func (a *UnixAddr) Network() string {
	return a.Net
}

func (a *NotUnixAddr) Network() string {
	return a.Net
}

//go:noinline
func Unreachable(addr *UnixAddr, hint Addr) int {
	if addr.Network() != hint.Network() { // redundant devirtualization?
		return 1
	}
	return 0
}

//go:noinline
func Hot() int {
	return 100
}

//go:noinline
func Action(param bool) {
	if param {
		addr := &UnixAddr{
			Net: "net",
		}
		hint := &NotUnixAddr{
			Net: "hint",
		}
		Unreachable(addr, hint)
	} else {
		Hot()
	}
}

func BenchmarkZeroWeight(b *testing.B) {
	for i := 0; i < b.N; i++ {
		Action(false)
	}
}

Build and collect a profile:

$ go test -c -a -o t.out ./zero_test.go
$ ./t.out -test.cpuprofile=default.pprof -test.count=3 -test.bench=.

What did you see happen?

As can be seen, Unreachable is never called. Hence, it hasn't appeared in the profile:

$ go tool pprof -raw default.pprof
PeriodType: cpu nanoseconds
Period: 10000000
Time: 2025-03-04 12:53:46.38332888 +0000 UTC
Duration: 4.72
Samples:
samples/count cpu/nanoseconds
         58  580000000: 1 2 3 4 5
         92  920000000: 6 3 4 5
        147 1470000000: 7 3 4 5
         79  790000000: 8 4 5
         73  730000000: 9 4 5
          5   50000000: 10 3 4 5
Locations
     1: 0x11c1e0 M=1 command-line-arguments.Hot /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:37:0 s=36
     2: 0x11c253 M=1 command-line-arguments.Action /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:51:0 s=41
     3: 0x11c2af M=1 command-line-arguments.BenchmarkZeroWeight /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:57:0 s=55
     4: 0xd74ef M=1 testing.(*B).runN /home/jlapenko/go/src/testing/benchmark.go:202:0 s=182
     5: 0xd7fd3 M=1 testing.(*B).launch /home/jlapenko/go/src/testing/benchmark.go:333:0 s=307
     6: 0x11c1f0 M=1 command-line-arguments.Action /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:41:0 s=41
     7: 0x11c258 M=1 command-line-arguments.Action /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:53:0 s=41
     8: 0x11c2a4 M=1 command-line-arguments.BenchmarkZeroWeight /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:56:0 s=55
     9: 0x11c2b8 M=1 command-line-arguments.BenchmarkZeroWeight /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:56:0 s=55
    10: 0x11c250 M=1 command-line-arguments.Action /home/jlapenko/PGO_devirt/zero_weight/zero_test.go:51:0 s=41
Mappings
1: 0x10000/0x127000/0x0 /home/jlapenko/PGO_devirt/zero_weight/t.out 9e696feb90352f1483c3df478ba7a8a1ee3e6aa0 [FN]
2: 0xffff94660000/0xffff94661000/0x0 [vdso]

However, the call hint.Network() in the Unreachable function has been devirtualized:

$ go test -c -a -pgo=default.pprof -gcflags='-m' -o t.pgo.out ./zero_test.go
# command-line-arguments [command-line-arguments.test]
./zero_test.go:29:35: PGO devirtualizing interface call hint.Network to (*UnixAddr).Network
./zero_test.go:19:6: can inline (*UnixAddr).Network
./zero_test.go:23:6: can inline (*NotUnixAddr).Network
./zero_test.go:55:6: can inline BenchmarkZeroWeight
./zero_test.go:29:17: inlining call to (*UnixAddr).Network
./zero_test.go:29:35: inlining call to (*UnixAddr).Network
...

The reason is the following. WeightedCG contains all functions from a package being compiled. In our case (taken using -d=pgodebug=3):

hot-cg before inline in dot format:
digraph G {
forcelabels=true;
"command-line-arguments.Unreachable" [color=black, style=solid, label="command-line-arguments.Unreachable"];
"command-line-arguments.Hot" [color=black, style=solid, label="command-line-arguments.Hot"];
"command-line-arguments.BenchmarkZeroWeight" [color=black, style=solid, label="command-line-arguments.BenchmarkZeroWeight"];
"command-line-arguments.Action" [color=black, style=solid, label="command-line-arguments.Action"];
"command-line-arguments.(*UnixAddr).Network" [color=black, style=solid, label="command-line-arguments.(*UnixAddr).Network"];
"command-line-arguments.(*NotUnixAddr).Network" [color=black, style=solid, label="command-line-arguments.(*NotUnixAddr).Network"];
edge [color=black, style=solid];
"command-line-arguments.Unreachable" -> "command-line-arguments.(*UnixAddr).Network" [label="0.00"];
edge [color=black, style=solid];
"command-line-arguments.Action" -> "command-line-arguments.Unreachable" [label="0.00"];
edge [color=black, style=solid];
"command-line-arguments.Action" -> "command-line-arguments.Hot" [label="12.78"];
edge [color=red, style=solid];
"command-line-arguments.BenchmarkZeroWeight" -> "command-line-arguments.Action" [label="55.51"];
}

There are nodes for both (*UnixAddr).Network and (*NotUnixAddr).Network functions. The edge Unreachable -> (*UnixAddr).Network has a zero weight because there are no suitable samples in the profile.

PGO devirtualization considers the call hint.Network() and tries to find the most appropriate candidate. For the Unreachable node, it goes through OutEdges with the same callsite offset. Note that we can't distinguish callsites addr.Network() and hint.Network() because they are placed on the same line. The edge to (*UnixAddr).Network is the only one because there are no samples related to the Unreachable function. This callee's AST != nil, and the method receiver type implements the interface Addr. So, hint.Network() is devirtualized to (*UnixAddr).Network, although this caller relates to addr.Network() and the weight of the edge is 0.

I suppose we shouldn't devirtualize a call when a weight of its hottest edge is 0 even if its AST != nil. Moreover, the devirtualized call will be inlined later (as can be seen in the debug output shown above), and this leads to unjustified increase of code size.

What did you expect to see?

The call hint.Network() shouldn't be devirtualized.

The text was updated successfully, but these errors were encountered:

gabyhelp · 2025-03-04T14:33:38Z

Related Issues

cmd/compile/internal/devirtualize: PGO devirtualization lost when inlining function containing devirtualized call #62245
cmd/compile: changing a hot concrete method to interface method triggers a PGO ICE #67016 (closed)
runtime/pprof: TestCPUProfileMultithreadMagnitude fail in linux6.2 ubuntu in VirtualBox #63650
cmd/compile: PGO fails to do multiple levels of inlining into a single function #69046

Related Code Changes

cmd/compile: enable PGO-driven call devirtualization

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

JunyangShao · 2025-03-04T17:24:33Z

CC @golang/compiler @prattmic

prattmic · 2025-03-04T20:47:45Z

Nice find. I agree we shouldn't devirtualize if the weight is zero.

randall77 · 2025-03-04T22:04:05Z

Keep in mind that weight 0 can mean two different things:

The function was never called in the instrumented binary (up to sampling accuracy)
The function didn't exist in the instrumented binary

The second case means it is a brand new function. Brand new functions are actually reasonably likely to be hot, as someone just wrote them.

It would be nice if we had a way to distinguish those two cases. If we could, brand new functions should probably get some default weight to start.

gopherbot · 2025-03-06T16:42:30Z

Change https://go.dev/cl/655475 mentions this issue: cmd/compile/internal/devirtualize: do not select a zero-weight edge as the hottest one

yukatan1701 · 2025-03-06T16:44:39Z

@randall77, if I understand correctly, we may encounter the second case when using an outdated profile. Since this usage can negatively affect some other optimizations, I think preventing devirtualization for zero-weight edges won't be the biggest problem. Anyway, it seems to me that the problem of checking the relevance of the profile is separate. I suggest moving it to a new issue.

gabyhelp added the BugReport label Mar 4, 2025

JunyangShao added the NeedsInvestigation label Mar 4, 2025

prattmic added the compiler/runtime label Mar 4, 2025

mknyszek added this to Go Compiler / Runtime Mar 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/compile/internal/devirtualize: PGO devirtualization selects an edge with a weight of 0 as the hottest one #72092

cmd/compile/internal/devirtualize: PGO devirtualization selects an edge with a weight of 0 as the hottest one #72092

yukatan1701 commented Mar 4, 2025

gabyhelp commented Mar 4, 2025

JunyangShao commented Mar 4, 2025

prattmic commented Mar 4, 2025

randall77 commented Mar 4, 2025

gopherbot commented Mar 6, 2025

yukatan1701 commented Mar 6, 2025

cmd/compile/internal/devirtualize: PGO devirtualization selects an edge with a weight of 0 as the hottest one #72092

cmd/compile/internal/devirtualize: PGO devirtualization selects an edge with a weight of 0 as the hottest one #72092

Comments

yukatan1701 commented Mar 4, 2025

Go version

Output of go env in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

gabyhelp commented Mar 4, 2025

JunyangShao commented Mar 4, 2025

prattmic commented Mar 4, 2025

randall77 commented Mar 4, 2025

gopherbot commented Mar 6, 2025

yukatan1701 commented Mar 6, 2025

Output of `go env` in your module/workspace: