Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: slow compilation of a package with a very large API #53092

Open
baywet opened this issue May 26, 2022 · 1 comment
Open

cmd/compile: slow compilation of a package with a very large API #53092

baywet opened this issue May 26, 2022 · 1 comment
Labels
NeedsInvestigation
Milestone

Comments

@baywet
Copy link

@baywet baywet commented May 26, 2022

What version of Go are you using (go version)?

go1.18.2 windows/amd64

the OS doesn't seem to be making a big difference, and I was experiencing the issue since at least go 17

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\vibiret\AppData\Local\go-build
set GOENV=C:\Users\vibiret\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\vibiret\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\vibiret\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.18.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\sources\gitlocal\gorepro\go.mod
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\vibiret\AppData\Local\Temp\go-build1532428258=/tmp/go-build -gno-record-gcc-switches

What did you do?

go mod init main
go get github.com/microsoftgraph/msgraph-sdk-go@v0.24.1 #takes about 20 seconds on my machine, expected, this is the large package we're maintaining
go get github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.0.0 # takes about 5 seconds, expected
go get github.com/microsoft/kiota-authentication-azure-go@v0.3.0 # takes about 2 seconds, expected

notepad main.go #or code, vim,... whatever you have on your OS

# paste the content from the appendix

go build # takes about 1m30s, long but expected

Now replace the following lines

-result, err := client.Me().Messages().Get()
+result, err := client.Me().Drive().Get()

Then run go build again, which takes about 30 seconds, not expected, should take a couple of seconds thanks to caching.

What did you expect to see?

Subsequent builds should be in the sub seconds time as the packages should be cached.

What did you see instead?

Takes about the original build time.

Additional context

We're maintaining a large package that's generated from an API description. The API itself is fairly large. The initial ask from our SDK users was about the build time increasing significantly when adding our package and caching seeming like it's not fully working from one go build to another.

With the release of 1.18 and generics, we're already identified opportunities for improvements to generate less code, but haven't acted on it yet. @breml @josharian @dominikh were really helpful with framing the issue and suggested we posted here before making those improvements.

Additionally, the go doc for our package is not working either.

Appendix

main.go content

package main

import (
	"context"
	"fmt"

	azidentity "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	a "github.com/microsoft/kiota-authentication-azure-go"
	r "github.com/microsoft/kiota-http-go"
	msgraphsdk "github.com/microsoftgraph/msgraph-sdk-go"
)

func main() {
	cred, err := azidentity.NewDeviceCodeCredential(&azidentity.DeviceCodeCredentialOptions{
		TenantID: "placeholder",
		ClientID: "placeholder",
		UserPrompt: func(ctx context.Context, message azidentity.DeviceCodeMessage) error {
			fmt.Println(message.Message)
			return nil
		},
	})
	if err != nil {
		fmt.Printf("Error creating credentials: %v\n", err)
	}
	auth, err := a.NewAzureIdentityAuthenticationProviderWithScopes(cred, []string{"Mail.Read", "Mail.Send"})
	if err != nil {
		fmt.Printf("Error authentication provider: %v\n", err)
		return
	}
	adapter, err := r.NewNetHttpRequestAdapter(auth)
	if err != nil {
		fmt.Printf("Error creating adapter: %v\n", err)
		return
	}
	client := msgraphsdk.NewGraphServiceClient(adapter)
	result, err := client.Me().Messages().Get()
	if err != nil {
		fmt.Printf("Error executing request: %v\n", err)
		return
	}
	if result != nil {
		fmt.Printf("Got a result")
		return
	}
}
@mvdan
Copy link
Member

@mvdan mvdan commented May 26, 2022

should take a couple of seconds thanks to caching.

To add some context for people coming across this issue: Go's build cache works at the package level, so modifying a package in any way will generally mean rebuilding the entire package. See https://pkg.go.dev/cmd/go#hdr-Build_and_test_caching.

@toothrot toothrot added the NeedsInvestigation label May 26, 2022
@toothrot toothrot added this to the Backlog milestone May 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

3 participants