Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fork/join benchmarks #209

Merged
merged 12 commits into from Dec 29, 2017
Merged

Add fork/join benchmarks #209

merged 12 commits into from Dec 29, 2017

Conversation

smarr
Copy link
Owner

@smarr smarr commented Dec 28, 2017

This PR adds a few benchmarks based on the Work–Stealing Without The Baggage paper.

The original version originates from: https://github.com/vivkumar/TryCatchWS/tree/master/test
But was first ported to Java's fork/join framework here:
https://github.com/smarr/are-we-fast-yet/tree/fork-join/benchmarks/Java/src/forkjoin

The list of ported benchmarks includes:

  • CilkSort
  • Fib(onacci)
  • Integrate
  • Jacobi
  • LUDecomposition
  • NQueens
  • QuickSort

It does not yet include FFT and UTS.
FFT is just too much code for the moment.
And UTS does some bit shifting, which is not yet nicely supported in SOMns.

The benchmarks are generally not very sophisticated, even in their optimized version, they rarely achieve good scalability on Java's fork/join framework. Most do not use a sequential cut off, and thus task creation overhead is rather high.

To assess the characteristics of the benchmarks, they are all available in four versions:

  • Seq: no spawns, everything is done sequentially
  • Opt: uses spawns, but does at least one task locally
  • Nai: uses spawns for all tasks, so, it is only joining locally
  • One: spawns a task, and waits for it to finish. Thus, this is the worst case scenario, introducing f/j overhead without any gain.

Thanks to @VAISHALI-DHANOA, who started this work.

@eleinadani @eregon, this might be interesting to you. If we continue this, it might be interesting to port to Ruby. The benchmarks I got are pretty straight forward to translate, one just needs a lightweight fork/join mechanism. And, of course, pretty heavy on array data structures.

@daumayr possibly interesting for you, another bunch of benchmarks, and a change to the harness to add support for running benchmarks by file name.

VAISHALI-DHANOA and others added 12 commits December 27, 2017 10:54
Signed-off-by: Stefan Marr <git@stefan-marr.de>
This benchmark seems problematic, because we run out of stack, but the results seem fine.

Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
This is better in case one reflects on a module, avoiding possible confusion. Implemented as a rule in the SOMns linter.

Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
Signed-off-by: Stefan Marr <git@stefan-marr.de>
@smarr smarr added the enhancement Improves the implementation with something noteworthy label Dec 28, 2017
@smarr smarr added this to the v0.6.0 - Black Diamonds milestone Dec 28, 2017
@smarr smarr self-assigned this Dec 28, 2017
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.07%) to 79.859% when pulling 4aeb157 on fj-benchmark into b62155b on dev.

@smarr smarr merged commit 40830cb into dev Dec 29, 2017
@smarr smarr deleted the fj-benchmark branch December 29, 2017 01:02
@eregon
Copy link

eregon commented Dec 29, 2017

Nice set of benchmarks!
Would definitely be interested to port some to Ruby and see how they run and what we can optimize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improves the implementation with something noteworthy
Projects
No open projects
MetaConc
Awaiting triage
Development

Successfully merging this pull request may close these issues.

None yet

4 participants