Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

META: benchmarks to include #1

Open
anp opened this issue Feb 22, 2018 · 12 comments

Comments

Projects
None yet
5 participants
@anp
Copy link
Owner

commented Feb 22, 2018

This is a tracking issue for new benchmarks we'd like to add. If you're landing here from hacktoberfest (welcome!), feel free to see the project README for more information :D.

The basic process goes:

  1. find where the project in question keeps its benchmarks (usually in its repo under benches)
  2. read the documentation on adding new benchmarks
  3. add the benchmarks
  4. send a PR
  5. at a point in the near future, we'll start seeing results from the benchmarks you ported on https://blog.anp/lol/lolbench-data

unclaimed

please comment on individual issues to claim them!

finished

@anp

This comment was marked as outdated.

Copy link
Owner Author

commented Apr 3, 2018

@anp

This comment was marked as outdated.

Copy link
Owner Author

commented Apr 3, 2018

@anp

This comment was marked as outdated.

Copy link
Owner Author

commented Apr 3, 2018

@gnzlbg

This comment was marked as outdated.

Copy link

commented May 3, 2018

We should probably also include the benchmark-game programs from https://github.com/TeXitoi/benchmarksgame-rs .

Also I always wanted to port the remaining examples of the rust-lang-nursery/simd crate to rust-lang-nursery/stdsimd, maybe we should include those once they are done. They will probably both have a scalar and vectorized implementation, but catching regressions on LLVM vector type optimizations is worth pursuing.

The ISPC compiler has a bunch of examples that we might want to port to Rust: https://github.com/ispc/ispc/tree/master/examples

From the standard suites, porting SPEC2017 (https://www.spec.org/benchmarks.html) to Rust.


IMO when porting thins, we should port benchmarks to "idiomatic" Rust code.

@anp

This comment was marked as outdated.

Copy link
Owner Author

commented May 3, 2018

I agree! :p

Right now the benchmark count bottleneck is having enough machines to run them in a timely fashion, I need to dig in more on the bench timings and get the new server(s) I'm setting up ready.

I think ISPC and SPEC benchmarks would be fantastic to track as well!

@anp

This comment was marked as outdated.

Copy link
Owner Author

commented May 9, 2018

per @eddyb on irc, we should also include an equivalent to the inflate 0.3.4 benchmark with a more recent version of the crate.

@eddyb

This comment was marked as outdated.

Copy link

commented May 11, 2018

The very important bit is whether or not the debloating image-rs/inflate#35 is included.

@gnzlbg

This comment was marked as outdated.

Copy link

commented Jun 5, 2018

I've uploaded the ambient occlusion (ray-tracing) benchmarks: https://github.com/gnzlbg/aobench

The readme shows how to run the scalar, vectorized, and parallel versions. Probably it only makes sense to add the scalar and scalar_par benchmarks initially. The vector and vector_par benchmarks use std::simd and that's still in flux.

@newpavlov

This comment was marked as outdated.

Copy link

commented Oct 2, 2018

How about some crates from RustCrypto? sha2, sha3 and one of AES crates can be a good start.

@anp anp changed the title Bench suites to include META: Tracking benchmarks and suites to include Oct 4, 2018

@anp

This comment has been minimized.

Copy link
Owner Author

commented Oct 5, 2018

Going to lock this so it can just be a tracking issue. If you have ideas for other benchmarks we should include, please open an issue!

Repository owner locked and limited conversation to collaborators Oct 5, 2018

@anp anp changed the title META: Tracking benchmarks and suites to include META: benchmarks to include Oct 14, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.