New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: record and collect compiler benchmarks #17167

Open
mdempsky opened this Issue Sep 20, 2016 · 8 comments

Comments

Projects
None yet
5 participants
@mdempsky
Member

mdempsky commented Sep 20, 2016

Recently we added a -bench flag to cmd/compile to benchmark compiler phases. It might be interesting if the builders used this flag and then persisted the output file so we could monitor compiler performance over time.

There's a risk that measuring the standard library compile time is confounded with changes to the standard library, but I think only a couple packages (runtime, net, net/http) still seem under active development. So it's probably still a decent signal.

/cc @bradfitz @griesemer

@mdempsky mdempsky added the Builders label Sep 20, 2016

@mdempsky mdempsky added this to the Unreleased milestone Sep 20, 2016

@mdempsky

This comment has been minimized.

Member

mdempsky commented Sep 20, 2016

/cc @davecheney too, since he's been benchmarking build times against other code bases too.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Sep 20, 2016

I've been logging more and more to Datastore & BigQuery, and it's on my short-term list to add more timing info there, as part of optimizing build+test speeds for #17104

So, this isn't much more work.

/cc @quentinmit

@quentinmit

This comment has been minimized.

Contributor

quentinmit commented Sep 20, 2016

Yep, and I'm working on collecting benchmark results. This sounds great. Where does -bench put its output?

Also, which compile command would you instrument? There are a bunch involved in all.bash.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Sep 20, 2016

You can use -bench=foo.txt and the compiler will append to foo.txt, so the entire build can be accumulated into a single file. The individual measurements indicate the package being compiled.

I would probably tweak all.bash so that the "go install std cmd" step adds -gcflags="-bench=/tmp/bench.txt" on the builders, and then save /tmp/bench.txt somewhere we can easily collect and analyze.

I think benchmarking the Go1 benchmarks' compile times would be nice too, since they're a fixed target.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Sep 20, 2016

We don't use all.bash on the builders. And make.bash already respects the env var $GO_GCFLAGS

@mdempsky

This comment has been minimized.

Member

mdempsky commented Sep 20, 2016

I meant "all.bash" generically to refer to whatever build process was used by the builders. GO_GCFLAGS sounds good to me if that means we can do it non-intrusively.

@davecheney

This comment has been minimized.

Contributor

davecheney commented Sep 21, 2016

SGTM!

On Wed, Sep 21, 2016 at 4:19 AM, Matthew Dempsky notifications@github.com
wrote:

/cc @davecheney https://github.com/davecheney too, since he's been
benchmarking build times against other code bases too.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#17167 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAcA26_YEbWWITcnZNrkOVI6n0kZvA3ks5qsCO2gaJpZM4KB8V8
.

@gopherbot

This comment has been minimized.

gopherbot commented Jul 26, 2017

CL https://golang.org/cl/51192 mentions this issue.

gopherbot pushed a commit to golang/build that referenced this issue Jul 26, 2017

cmd/buildlet/stage0: more logging about various delays, pass in env t…
…o child

Updates golang/go#21153
Updates golang/go#17167

Change-Id: Ibe575d295468235a16c01f3420a9597373ab3891
Reviewed-on: https://go-review.googlesource.com/51192
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment