x/build: perf.go.dev/dashboard fails to report x/tools benchmarks

[adonovan]

The https://perf.golang.org/dashboard/?repository=tools page usually shows the benchmarks for x/tools including those of gopls, but it is currently failing to find any benchmark data:

[mknyszek]

To be clear, this is because the builder has been failing for months and has not been generating new data.

We need to do better on alerting when the builder is broken, but I'd appreciate the tools' team's help with fixing the builder.

Failure: https://ci.chromium.org/ui/p/golang/builders/ci/x_tools-go1.25-linux-arm64_c4as16-perf_vs_gopls_0_11/b8695214781159870321/overview

Output: https://logs.chromium.org/logs/golang/buildbucket/cr-buildbucket/8695214781159870321/+/u/step/27/log/3

--- FAIL: BenchmarkLocalModcache/budget=0s/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=0s/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=0s
--- FAIL: BenchmarkLocalModcache/budget=100ms/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=100ms/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=100ms
--- FAIL: BenchmarkLocalModcache/budget=200ms/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=200ms/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=200ms
--- FAIL: BenchmarkLocalModcache/budget=500ms/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=500ms/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=500ms
--- FAIL: BenchmarkLocalModcache/budget=1s/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=1s/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=1s
--- FAIL: BenchmarkLocalModcache/budget=5s/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=5s/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=5s
--- FAIL: BenchmarkLocalModcache

--- FAIL: BenchmarkDiagnoseSave/kubernetes.types
    didchange_test.go:134: formatting before save: editing "staging/src/k8s.io/api/core/v1/types.go": column is beyond end of line; edits:
...

[gopherbot]

Change https://go.dev/cl/730120 mentions this issue: gopls/internal/cache/parsego: construct File.Cursor lazily

[adonovan]

To be clear, this is because the builder has been failing for months and has not been generating new data.

We need to do better on alerting when the builder is broken, but I'd appreciate the tools' team's help with fixing the builder.

Of course! I didn't to cast blame, I just have been blithely ignorant of exactly how this particular sausage gets made.

Will investigate tomorrow.

[mknyszek]

I didn't to cast blame, I just have been blithely ignorant of exactly how this particular sausage gets made.

No worries at all, I didn't interpret your writing as casting blame. :) Just wanted to make sure that the root cause and next steps got into the issue.

... Communicating via text is hard.

[gabyhelp]

Related Issues

• x/build: restore performance benchmarking dashboard #20552 (closed)
• dashboard: perf dashboard shows "no commits" #9485 (closed)
• x/build: linux-amd64-perf repeatedly failing on x/tools with "error committing perfdata file: upload failed: 500 Internal Server Error no valid benchmark lines found" #58875 (closed)
• Go dashboard - benchmarks missing #683 (closed)
• x/tools,x/build: performance monitoring for gopls #53538 (closed)
• wiki: PerfDashboard links broken #32911 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[pjweinb]

I'll fix the failing, but it shouldn't be run routinely. It depends on having a usable local module cache, which I don't think happens on the builders. (but it was computing its location in an obsolete way).

It was useful to compare the old way of doing unimported completions with the new way, demonstrating that frequently the old way failed to find the completion at all. I don't think it is useful any longer. It could either be removed or b.Skip("only run manually")

[adonovan]

https://go-review.googlesource.com/c/tools/+/732000 was merged but the benchmark run is still failing.

The latest crop of failures from Jan 8 includes:

--- FAIL: BenchmarkDiagnoseSave/kubernetes.types
    didchange_test.go:134: formatting before save: editing "staging/src/k8s.io/api/core/v1/types.go": column is beyond end of line; edits: ...
        [{3609:2-3609:6 
        //	} {4671:2-4671:2 
        //	} {4671:3-4671:5 } {4672:2-4672:2 	} {4672:3-4707:51 Subsets: [
        //	   {
        //	     Addresses: [{"ip": "10.10.1.1"}, {"ip": "10.10.2.2"}],
        //	     Ports: [{"name": "a", "port": 8675}, {"name": "b", "port": 309}]
        //	   },
        //	   {
        //	     Addresses: [{"ip": "10.10.3.3"}],
        //	     Ports: [{"name": "a", "port": 93}, {"name": "b", "port": 76}]
        //	   },
        //	]
        type Endpoints struct {
        	metav1.TypeMeta `json:",inline"`
        	// Standard object's metadata.
        	// More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
        	// +optional
        	metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
        
        	// The set of all endpoints is the union of all subsets. Addresses are placed into
        	// subsets according to the IPs they share. A single address with multiple ports,
        	// some of which are ready and some of which are not (because they come from
        	// different containers) will result in the address being displayed in different
        	// subsets for the different ports. No address will appear in both Addresses and
        	// NotReadyAddresses in the same subset.
        	// Sets of addresses and ports that comprise a service.
        	// +optional
        	Subsets []EndpointSubset `json:"subsets,omitempty" protobuf:"bytes,2,rep,name=subsets"`
        }
        
        // EndpointSubset is a group of addresses with a common set of ports. The
        // expanded set of endpoints is the Cartesian product of Addresses x Ports.
        // For example, given:
        //
        //	{
        //	  Addresses: [{"ip": "10.10.1.1"}, {"ip": "10.10.2.2"}],
        //	  Ports:     [{"name": "a", "port": 8675}, {"name": "b", "port": 309}]
        //	}
        //
        // The resulting set of endpoints can be viewed as:
        //} {4708:2-4708:7 	} {4709:2-4709:7 	} {5581:111-5581:111 
        //}]

and

--- FAIL: BenchmarkLocalModcache/budget=0s/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=0s/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=0s
--- FAIL: BenchmarkLocalModcache/budget=100ms/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=100ms/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=100ms
--- FAIL: BenchmarkLocalModcache/budget=200ms/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=200ms/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=200ms
--- FAIL: BenchmarkLocalModcache/budget=500ms/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=500ms/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=500ms
--- FAIL: BenchmarkLocalModcache/budget=1s/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=1s/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=1s
--- FAIL: BenchmarkLocalModcache/budget=5s/source=gopls
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=5s/source=goimports
    unimported_test.go:128: open /home/swarming/.swarming/w/ir/x/w/xdgcache/go/imports/index-name-0-2ac3dd3d29ec0325670df82b5bf076117d12268906cadc25004c6cfe41ff2aa9: no such file or directory
--- FAIL: BenchmarkLocalModcache/budget=5s
--- FAIL: BenchmarkLocalModcache