Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement parallel::unique. #2867

Merged
merged 27 commits into from Sep 25, 2017
Merged

Implement parallel::unique. #2867

merged 27 commits into from Sep 25, 2017

Conversation

taeguk
Copy link
Member

@taeguk taeguk commented Aug 28, 2017 •
edited

It is related to #1141, #2804.

Check Box

  • Implementation of parallel::unique using temporary storage. (eecac2c1)
  • Implementation of parallel::unique using sequentially performed f3. (The lastest commit)
  • Unit tests.
  • Benchmark codes. (with various data type)
  • Adapt to Ranges TS. (Container version)
  • Some modifications of parallel::util::scan_partitioner.

Issues

  1. What way should we select between the two ways to implement parallel::unique discussed in The in-place parallel algorithms are hard to implement efficiently. #2804?
    -> I implemented both ways and benchmarked them. In conclusion, using the way to perform f3 sequentially is better. So, I choosed that.

Note for future

  1. The performance of parallel::unique is bad. The performance is different depending on the situation, but it is slightly faster than the sequential version, and sometimes it is slower than the sequential version. There is need to improve this algorithm if we can find the better way.

@taeguk
Reduce code duplications in parallel::util::scan_partitioner.
71716f3
@taeguk
Implement parallel::unique.
71b1ee0
@taeguk
Add unit tests for parallel::unique.
ea61017
@taeguk
Add benchmark codes for parallel::unique.
e7c4435
@taeguk
Add container version of parallel::unique. And add unit tests for that.
ff72d98
@taeguk
Improve benchmark codes of parallel::unique for various data type.
bf0a8be
@taeguk
Use move operations instead of copy operations in parallel::unique.
ad2f0fd
@taeguk
Add 'array' data type and rename 'heavy' to 'vector' in benchmark cod…
eecac2c 
…es of parallel::unique.
@taeguk
Add new version of parallel::util::scan_partitioner for sequantially …
8f23ff5 
…performed f3.
@taeguk
Re-implement parallel::unique using sequentially performed f3.
4110113 
Original way: allocating temporary storage and utilizing out-place version.
New way: perform f3 sequentially.

See also: STEllAR-GROUP#2804
@taeguk
Fix a bug in reimplemented parallel::unique.
77249f5
@taeguk
Refactor parallel::unique.
df84991 
1. Remove non-meaning codes.
2. Exchange the order of codes of parallel::unique and parallel::unique_copy.
@taeguk
Fix a bug about multipass guarantee in parallel::util::scan_partitioner.
85b35eb 
See also: STEllAR-GROUP#2866
@taeguk
Fix a bug which is caused by self-assignment in parallel::unique.
52121c9
@taeguk
Fix inspection errors.
5e028b4
@hkaiser hkaiser added this to the 1.1.0 milestone Aug 28, 2017
@taeguk
Copy link
Member Author

taeguk commented Aug 28, 2017

https://github.com/taeguk/hpx/blob/da7fd5d7dd38ca0074c579d248ef7ea916c200bf/hpx/parallel/util/scan_partitioner.hpp#L284-L289
@hkaiser I used explicit waiting like above.
But, you said "Prefer dataflow rather than explicit waiting".
https://github.com/taeguk/hpx/blob/da7fd5d7dd38ca0074c579d248ef7ea916c200bf/hpx/parallel/util/scan_partitioner.hpp#L304
Yes, I agree. But, if we want to use dataflow instead of explicit waiting, we should change the type of finalitems from std::vector<hpx::future<Result2>> to std::vector<hpx::shared_future<Result2>> because final item should be passed to both 'f3 on next partition' and 'f4'. And, the interface of f3 should be also changed. These changes should be also applied even if ScanPartTag is scan_partitioner_normal_tag because maybe it is better for same template to have same interface. So, f3 of existed codes using scan_partitioner should be a little modified.

How do you think about it? Keep using current explicit waiting? Or use dataflow with changing the interface of f3 in scan_partitioner.

@hkaiser
Copy link
Member

hkaiser commented Sep 5, 2017

Please fix the conflicts, otherwise this can't go ahead.

@taeguk
Merge branch 'master' into tg_unique
4290cc9
@taeguk
Use static_scan_partitioner_helper instead of static_scan_partitioner…
2bab536 
…_async for fixing a bug when parallel_task_policy is used.

The bug    : static_scan_partitioner doesn't work as async when parallel_task_policy is used.
The reason : because of ```finalitems.back().wait();``` when
             scan_partitioner_sequential_f3_tag is used as ScanPartTag.
How to fix : Use static_scan_partitioner_helper which returns R instead of hpx::future<R>.
@taeguk
Support optional throwing of errors in handle_local_exceptions.
8387573
@taeguk
Fix a problem that exceptions of finalitems may not be handled in par…
290732f 
…allel::util::scan_partitioner.
@taeguk
Copy link
Member Author

taeguk commented Sep 10, 2017

@hkaiser I'm ready to be reviewed and merged.

@hkaiser
Copy link
Member

hkaiser commented Sep 22, 2017

@taeguk Please fix the conflicts for this to be merged. And sorry for the long delay ...

@taeguk
Copy link
Member Author

taeguk commented Sep 24, 2017

@hkaiser I resolved conflicts. And I did somethings related to #2908, too.
I'm ready to be merged :)

@hkaiser
Copy link
Member

hkaiser commented Sep 24, 2017

@taeguk: thanks a lot for your continued work on this! Just a heads up - there are still problems reported by inspect.

hkaiser
hkaiser approved these changes Sep 25, 2017
View reviewed changes
Copy link
Member

@hkaiser hkaiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot!

@hkaiser hkaiser merged commit 0e82bf9 into STEllAR-GROUP:master Sep 25, 2017
This was referenced Sep 25, 2017
Closed
Closed
@hkaiser hkaiser added this to Merged to master in Standard Algorithms Oct 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hkaiser hkaiser hkaiser approved these changes

Assignees
No one assigned
Labels
category: algorithms type: enhancement
Projects
Standard Algorithms
  
Merged to master
Milestone
1.1.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@taeguk @hkaiser