Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
cmd/go: load packages in parallel #29758
Currently, when the Go command loads a set of packages and their dependencies, it does so on a single goroutine. Tracing shows that most cores are idle when loading packages for a large build, so there's significant performance opportunity here. On the one core that's busy, most of the time is spent in
One may expect the package loading process to be dominated by I/O, but the files loaded are usually in the kernel file cache, and we do a lot of work in userspace to parse files, extract imports, and apply build constraints. We actually spend very little time in the kernel, at least on Linux (system call overhead is very low).
Based on this information, it seems that we can significantly speed up the package loading process by parallelizing it. This will benefit
Also just to note https://go-review.googlesource.com/c/tools/+/158097. Chatting with @heschik, ideally
So hopefully this change will go a good step towards that goal.