Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report standard deviation of the speedup ratio #83

Merged
merged 2 commits into from
Sep 15, 2018

Conversation

Shnatsel
Copy link
Contributor

Report standard deviation of the speedup ratio.

Here are results for running the same Rust binary that exits immediately twice:

Before:

  'target/release/examples/decode' ran
    1.03x faster than 'target/release/examples/decode'

After:

  'target/release/examples/decode' ran
    1.03x ± 0.65 faster than 'target/release/examples/decode'

Results from a real-world benchmarking run:

  'flac -fd testsamples/extra/*.flac' ran
    1.56x ± 0.08 faster than 'target/release/examples/decode testsamples/extra/*.flac'

I'd also really like to see statistical significance estimation and getting rid of the word "faster" and all that green in case it's not, but I don't have the time to work on it right now.

@sharkdp
Copy link
Owner

sharkdp commented Sep 15, 2018

Thank you very much - this looks great!

My only concern is readability. The ran 1.03x ± 0.65 faster than doesn't read like an English sentence anymore if you pronounce x as "times". Maybe:

    1.03 ± 0.65 times faster than

?

@Shnatsel
Copy link
Contributor Author

I've updated the code with the message you suggested.

@sharkdp sharkdp merged commit 1569334 into sharkdp:master Sep 15, 2018
@sharkdp
Copy link
Owner

sharkdp commented Sep 15, 2018

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants