Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: mention notable benchmark changes in Gerrit comment #20412

Open
bradfitz opened this issue May 18, 2017 · 6 comments

Comments

@bradfitz
Copy link
Member

commented May 18, 2017

I think it's going to be too easy to miss significant benchmark changes with the current state of things where the URL to perf.golang.org is always included in the final Gerrit comment.

I think the gerrit comment should also mention any notable changes.

/cc @quentinmit @josharian

@gopherbot gopherbot added this to the Unreleased milestone May 18, 2017
@gopherbot gopherbot added the Builders label May 18, 2017
@josharian

This comment has been minimized.

Copy link
Contributor

commented May 18, 2017

Yes please! And perhaps also always include any relevant geomeans?

This should get implemented behind a flag that is easy to turn on and off, because past experience with performance dashboards suggests that we're going to have lots of problems with noise.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented May 18, 2017

I'd rather start with some noise than have no information. We have lots of noise as-is with flaky tests, sadly.

@josharian

This comment has been minimized.

Copy link
Contributor

commented May 18, 2017

I'm not so sure noise is better than nothing, particularly if it desensitizes people and causes them to systematically ignore/discount the results. I'll file an issue with an example and we can discuss some details there.

@gopherbot

This comment has been minimized.

Copy link

commented Jun 2, 2017

CL https://golang.org/cl/44613 mentions this issue.

@mvdan

This comment has been minimized.

Copy link
Member

commented Jan 15, 2019

I don't intend to resurrect old threads, but it seems like the CL above went unnoticed. Is @quentinmit still planning to work on this? If so, we should find someone who can review the code. There are also some conflicts, after all this time.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Jan 15, 2019

I don't think so. @dmitshur might be able to pick this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.