Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken benchmarks #205

Merged
merged 2 commits into from Apr 5, 2020
Merged

Conversation

elichai
Copy link
Member

@elichai elichai commented Mar 24, 2020

Currently the benchmarks in recovery weren't compiling.
I fixed and made travis compile them too.

I experimented with switching to criterion which looks pretty cool (works on stable, produces html plots, remembers last results to show you improvements/regressions etc.)
But dev-dependencies aren't allowed to be optional so this will break the tests on 1.22.0
rust-lang/cargo#1596

(We could add it to the regular dependencies though, like @TheBlueMatt is doing with mutagen https://github.com/rust-bitcoin/rust-lightning/pull/453/files#r368478034 )

real-or-random
real-or-random previously approved these changes Mar 24, 2020
@real-or-random
Copy link
Collaborator

Thanks!

I experimented with switching to criterion which looks pretty cool (works on stable, produces html plots, remembers last results to show you improvements/regressions etc.)

Do you something to share (e.g., a screenshot) to see what this is about? My first feeling is that nicer benchmarks would be more helpful upstream (but yeah, C...)

But dev-dependencies aren't allowed to be optional so this will break the tests on 1.22.0
rust-lang/cargo#1596

(We could add it to the regular dependencies though, like @TheBlueMatt is doing with mutagen https://github.com/rust-bitcoin/rust-lightning/pull/453/files#r368478034 )

Hm, it's indeed ugly. Not sure if we want this.

@elichai
Copy link
Member Author

elichai commented Mar 24, 2020

The "naive" output (from running cargo bench) is: https://i.imgur.com/Zq1nc78.png
(this is the result after I enable endomorphism)
But they also have HTML ouput, example: https://bheisler.github.io/criterion.rs/book/user_guide/html_report/Fibonacci2/Iterative/20/report/index.html
And CSV, and more things I haven't looked into yet ( https://bheisler.github.io/criterion.rs/book )

@elichai
Copy link
Member Author

elichai commented Mar 29, 2020

The no-std test was broken because of this PR: rust-lang/rust#67502
they removed 2 lang items that were required before.

And tests on stable are failing because of: #203

real-or-random
real-or-random previously approved these changes Mar 30, 2020
@apoelstra
Copy link
Member

Can we make the "unstable" feature imply recovery and rand?

@elichai
Copy link
Member Author

elichai commented Apr 5, 2020

Can we make the "unstable" feature imply recovery and rand?

Done. that also solved the conflicts :)

Copy link
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@apoelstra
Copy link
Member

Travis is done. I dunno why it's not updating on the PR.

@apoelstra apoelstra merged commit 3fd0897 into rust-bitcoin:master Apr 5, 2020
@real-or-random
Copy link
Collaborator

I've seen Travis status failing to update a few times in the past week (on upstream)...

Another question: Why are my review comments gone here in the PR? :/

@elichai
Copy link
Member Author

elichai commented Apr 5, 2020

That's really weird. I can see you reviewed it but I can't see the actual review/text

@elichai elichai deleted the 2020-03-bench branch April 13, 2020 08:29
@apoelstra apoelstra mentioned this pull request Aug 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants