Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding fork_join_executor to foreach_benchmark #5697

Merged
merged 6 commits into from
Dec 16, 2021
Merged

Adding fork_join_executor to foreach_benchmark #5697

merged 6 commits into from
Dec 16, 2021

Conversation

hkaiser
Copy link
Member

@hkaiser hkaiser commented Dec 5, 2021 •
edited by msimberg
Loading

  • minor tweaks to fork_join_executor
  • disable lock verification for RelWithDebInfo builds
  • more constexpr and noexcept
  • fixing issue when HPX_WITH_THREAD_PHASE_INFORMATION=On
  • remove duplicate macro definition
  • remove bogus assert in latch
  • fixing problem with small_vector
  • minor optimization for wait_all and handle_local_exceptions
  • making chunk iterators random access
  • always #define HPX_SMT_PAUSE

This is a collection of minor optimizations and fixes applied during doing some performance analysis of the fork-join executor. Some of the changes are unrelated, I can separate those, if needed.

@hkaiser hkaiser added this to the 1.8.0 milestone Dec 5, 2021
@hkaiser hkaiser force-pushed the fork_join_executor branch 3 times, most recently from 3400340 to fe02e08 Compare December 6, 2021 13:32
msimberg
msimberg requested changes Dec 6, 2021
View reviewed changes
Copy link
Contributor

@msimberg msimberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks @hkaiser!

libs/core/algorithms/include/hpx/parallel/util/detail/chunk_size_iterator.hpp Outdated Show resolved Hide resolved
libs/core/algorithms/tests/performance/foreach_scaling.cpp Outdated Show resolved Hide resolved
libs/core/config/include/hpx/config.hpp Outdated Show resolved Hide resolved
libs/core/datastructures/include/hpx/datastructures/detail/small_vector.hpp Show resolved Hide resolved
libs/core/executors/include/hpx/executors/fork_join_executor.hpp Outdated Show resolved Hide resolved
libs/core/executors/include/hpx/executors/fork_join_executor.hpp Show resolved Hide resolved
@hkaiser hkaiser force-pushed the fork_join_executor branch 4 times, most recently from bd9643d to be376c3 Compare December 8, 2021 03:06
@hkaiser hkaiser force-pushed the fork_join_executor branch 2 times, most recently from e0200d0 to 8dd10dc Compare December 8, 2021 13:45
@StellarBot
Copy link

Performance test report

HPX Performance

Comparison

BENCHMARKNO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch(=)

Info

PropertyBeforeAfter
HPX Commit96a2e4b23c68e4
HPX Datetime2021-11-11T08:14:57+00:002021-12-08T13:45:30+00:00
Envfile
Datetime2021-11-11T09:28:13.071121+01:002021-12-08T15:05:17.228927+01:00
Clusternamedaintdaint
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Hostnamenid00006nid01547

Comparison

BENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add(=)--
Stream Benchmark - Scale(=)(=)(=)
Stream Benchmark - Triad(=)--
Stream Benchmark - Copy(=)(=)(=)

Info

PropertyBeforeAfter
HPX Commit71d8dbe23c68e4
HPX Datetime2021-11-10T19:14:21+00:002021-12-08T13:45:30+00:00
Envfile
Datetime2021-11-10T20:28:18.266961+01:002021-12-08T15:05:32.891181+01:00
Clusternamedaintdaint
Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Hostnamenid00120nid01547

Explanation of Symbols

SymbolMEANING
=No performance change (confidence interval within ±1%)
(=)Probably no performance change (confidence interval within ±2%)
(+)/(-)Very small performance improvement/degradation (≤1%)
+/-Small performance improvement/degradation (≤5%)
++/--Large performance improvement/degradation (≤10%)
+++/---Very large performance improvement/degradation (>10%)
?Probably no change, but quite large uncertainty (confidence interval with ±5%)
??Unclear result, very large uncertainty (±10%)
???Something unexpected…

@hkaiser hkaiser force-pushed the fork_join_executor branch 3 times, most recently from 2b80d1c to af3523d Compare December 9, 2021 02:03
@hkaiser hkaiser force-pushed the fork_join_executor branch 4 times, most recently from d8b670d to a768c4d Compare December 9, 2021 21:08
@hkaiser
Copy link
Member Author

hkaiser commented Dec 9, 2021

retest cscs

@hkaiser
Copy link
Member Author

hkaiser commented Dec 10, 2021 •
edited
Loading

@msimberg this seems to be ready to go now. Also for good measure, the promised performance plot:

image

This graph shows the achieved performance comparing different versions of the taskbench benchmark. The important part for this PR are the yellow and lilac lines (old vs. new fork_join executor).

@hkaiser hkaiser force-pushed the fork_join_executor branch 3 times, most recently from 69c6b15 to 020deb8 Compare December 14, 2021 17:39
@hkaiser hkaiser force-pushed the fork_join_executor branch 2 times, most recently from 9ff3441 to a3f41de Compare December 14, 2021 20:00
@msimberg msimberg added the split: local PR targets local functionality label Dec 15, 2021
@hkaiser
Adding fork_join_executor to foreach_benchmark
b340a20 
- minor tweaks to fork_join_executor
- more constexpr and noexcept
- fixing issue when HPX_WITH_THREAD_PHASE_INFORMATION=On
- remove duplicate macro definition
- remove bogus assert in latch
- fixing problem with small_vector
- minor optimization for wait_all and handle_local_exceptions
- making chunk iterators random access
- always #define HPX_SMT_PAUSE
@hkaiser
More work on fork_join_executor
8d7caaf 
- remove atomics that are not needed
- split static and dynamic scheduling functions
@hkaiser
Moving configuration options from config.hpp to build system
ad990be 
- this disables the silent special handling of certain options if HPX_DEBUG is defined
@hkaiser
Fixing test issues
32a5be1
@hkaiser
Replace high_resolution_timer with hardware::timestamp
76e37e8
@hkaiser hkaiser force-pushed the fork_join_executor branch 2 times, most recently from d0c0462 to 8caedba Compare December 15, 2021 17:09
@msimberg
Copy link
Contributor

bors merge

@bors
Copy link

bors bot commented Dec 16, 2021

Build succeeded:

@bors bors bot merged commit 97c75e8 into master Dec 16, 2021
@bors bors bot deleted the fork_join_executor branch December 16, 2021 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msimberg msimberg msimberg approved these changes

@aurianer aurianer Awaiting requested review from aurianer

@biddisco biddisco Awaiting requested review from biddisco

Assignees
No one assigned
Labels
category: executors split: local PR targets local functionality type: compatibility issue type: enhancement type: optimization
Projects
None yet
Milestone
1.8.0
Development

Successfully merging this pull request may close these issues.
3 participants
@hkaiser @StellarBot @msimberg