cmd/go: go list cmd/compile takes 40x longer than go list cmd/...

As in the title:

```
> time go list cmd/compile
cmd/compile

real	0m8.523s
user	0m8.610s
sys	0m0.068s
```

vs
```
> time go list cmd/...
...output, including cmd/compile

real	0m0.200s
user	0m0.492s
sys	0m0.185s
```

Here is my go version:
```
> go version
go version devel go1.21-e51b5f8384 Thu May 25 10:06:01 2023 -0700 linux/amd64
```

CC @bcmills @matloob @adonovan 