x/build: mention notable benchmark changes in Gerrit comment

[bradfitz]

I think it's going to be too easy to miss significant benchmark changes with the current state of things where the URL to perf.golang.org is always included in the final Gerrit comment.

I think the gerrit comment should also mention any notable changes.

/cc @quentinmit @josharian

[josharian]

Yes please! And perhaps also always include any relevant geomeans?

This should get implemented behind a flag that is easy to turn on and off, because past experience with performance dashboards suggests that we're going to have lots of problems with noise.

[bradfitz]

I'd rather start with some noise than have no information. We have lots of noise as-is with flaky tests, sadly.

[josharian]

I'm not so sure noise is better than nothing, particularly if it desensitizes people and causes them to systematically ignore/discount the results. I'll file an issue with an example and we can discuss some details there.

[gopherbot]

CL https://golang.org/cl/44613 mentions this issue.

[mvdan]

I don't intend to resurrect old threads, but it seems like the CL above went unnoticed. Is @quentinmit still planning to work on this? If so, we should find someone who can review the code. There are also some conflicts, after all this time.

[bradfitz]

I don't think so. @dmitshur might be able to pick this up.