Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: proposal: automatically scale benchmark metrics in results output #33328

Open
jpap opened this issue Jul 28, 2019 · 5 comments

Comments

@jpap
Copy link

commented Jul 28, 2019

When performing a benchmark, we can currently use SetBytes(int64) to set the number of input or output bytes per run, and get a measurement like X MB/s in the output.

It would be nice to be able to set an arbitrary unit instead of "bytes", and to provide multiple scales instead of just "mega" (1e6).

Example 1: pixels per second (px/s): b.SetUnits(1920*1080, "px/s") when benchmarking a full-HD image would print to the output one of the following, depending on the scale of the result:

  • "X px/s"
  • "X kpx/s"
  • "X Mpx/s"
  • "X Gpx/s"

Example 2: frames per second (fps): b.SetUnits(1, "fps") when benchmarking one frame operation.

The existing implementation of SetBytes(b int64) could just call SetUnits(b, "B/s") for backwards compatibility.

@agnivade

This comment has been minimized.

Copy link
Member

commented Jul 28, 2019

I believe this https://tip.golang.org/pkg/testing/#B.ReportMetric is what you want.

@toothrot

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2019

@jpap Thanks for writing this up. Does @agnivade's suggestion work for you?

@jpap

This comment has been minimized.

Copy link
Author

commented Jul 29, 2019

Thanks @agnivade, it's great that arbitrary units of work are supported in Go 1.13+.

@toothrot, the second part to this issue, auto-scaling the results {_, k, M, G} instead of scaling by 1e6 (M) uniformly as is done now, still remains and doesn't appear to be already implemented (having just checked tip).

@toothrot toothrot changed the title testing: proposal: allow benchmark to use arbitrary units of work per run testing: proposal: automatically scale MB/s metric in benchmark results Jul 29, 2019

@toothrot

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2019

Great, glad to hear the arbitrary units work (#26037) will help! @jpap Let me know if I captured your request correctly in the title.

@aclements Thoughts on this?

@jpap

This comment has been minimized.

Copy link
Author

commented Jul 29, 2019

@toothrot, it would be great if the unit-scaling applied to all reported metrics in the benchmark data format, not just bytes/sec (MB/s).

@toothrot toothrot changed the title testing: proposal: automatically scale MB/s metric in benchmark results testing: proposal: automatically scale benchmark metrics in results output Jul 30, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.