Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a distributed object abstraction #3807

Closed
wants to merge 5 commits into from

Conversation

@weilewei
Copy link

commented Apr 19, 2019

This PR aims at adding dist_object for HPX.

A distributed object is a single logical object partitioned across a set of localities. With this dist_object functionality, it provides a nice interface for programmers to write a distributed object. The users do not need to write much code in order to build a distributed component in HPX, and the code will be more readable and portable.

This PR contains dist_object itself, comments for Doxygen, many test cases, and one example (matrix transpose). Please review code when you have time, and I will revise accordingly. Thanks a lot!

@weilewei weilewei requested review from msimberg and hkaiser Apr 19, 2019

@biddisco
Copy link
Contributor

left a comment

I didn't look very closely at the examples yet

hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
hpx/include/dist_object.hpp Outdated Show resolved Hide resolved
tests/unit/lcos/barrier.cpp Outdated Show resolved Hide resolved
hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
@biddisco

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

Also - general question - can any of the partitioned vector + friends, be integrated with this code so as to reduce any duplicated functionality?

examples/transpose/transpose_dist_object.cpp Outdated Show resolved Hide resolved
examples/transpose/transpose_dist_object.cpp Outdated Show resolved Hide resolved
examples/transpose/transpose_dist_object.cpp Outdated Show resolved Hide resolved
@msimberg

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

Could you please add (documented) header files to this list, please?

@hkaiser

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

@weilewei thanks for working on this! I believe the dist_object will become a very useful dictionary type for HPX.

@biddisco This is another good candidate (next to channel) to expose RDMA capabilities directly to the user.

@hkaiser hkaiser changed the title Dist object pr Adding a distributed object abstraction Apr 19, 2019

@hkaiser

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

Also - general question - can any of the partitioned vector + friends, be integrated with this code so as to reduce any duplicated functionality?

@biddisco We could potentially re-implement partitioned_vector on top of the distributed object. That might be even desirable once dist_object has been made RDMA aware...

@weilewei

This comment has been minimized.

Copy link
Author

commented Apr 19, 2019

Also - general question - can any of the partitioned vector + friends, be integrated with this code so as to reduce any duplicated functionality?

@biddisco Hi John, thanks for all your detailed comments above and I am fixing on every single issue mentioned here. However, I don't think I understand this quoted question well, could you please explain more and provide some small example that can help me understand better, if possible?

What does a partitioned vector mean? Do you mean a distributed vector that each locality owns part of the data of the vector? Also, what does the friend mean exactly? Also, can you point out where does the duplicated code exist or potentially will be? Any suggestion will help for my next fix. Thanks!

@biddisco

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

What does a partitioned vector mean? Do you mean a distributed vector that each locality owns part of the data of the vector? Also, what does the friend mean exactly? Also, can you point out where does the duplicated code exist or potentially will be? Any suggestion will help for my next fix.

What I mean is that there is a class in HPX called partitioned_vector that is a distributed vector, it probably has a lot of similarities to your class (or parts of it) - here's a quick list of some code that either uses it or depends on it - have a look at these just to see if there's any duplicated functionality.
(By friends, I just mean that there are lot's of related classes/files in HPX that are connected)

git grep -l partitioned_vector | grep -v docs | sort
...
hpx/components/containers/container_distribution_policy.hpp
hpx/components/containers/partitioned_vector/detail/view_element.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_component_decl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_component.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_component_impl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_decl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_fwd.hpp
hpx/components/containers/partitioned_vector/partitioned_vector.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_impl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_local_view.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_local_view_iterator.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_predef.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_segmented_iterator.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_view.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_view_iterator.hpp
hpx/components/containers/unordered/unordered_map.hpp
hpx/components/containers/unordered/unordered_map_segmented_iterator.hpp
hpx/include/partitioned_vector.hpp
hpx/include/partitioned_vector_predef.hpp
hpx/include/partitioned_vector_view.hpp
hpx/include/serialization.hpp
hpx/runtime/serialization/partitioned_vector.hpp
src/components/containers/CMakeLists.txt
src/components/containers/partitioned_vector/CMakeLists.txt
src/components/containers/partitioned_vector/partitioned_vector_component.cpp
src/components/containers/partitioned_vector/partitioned_vector_component_double.cpp
src/components/containers/partitioned_vector/partitioned_vector_component_int.cpp
src/components/containers/partitioned_vector/partitioned_vector_component_std_string.cpp
tests/performance/local/CMakeLists.txt
tests/performance/local/partitioned_vector_foreach.cpp
tests/performance/network/algorithms/minmax_element_performance.cpp
tests/regressions/components/CMakeLists.txt
tests/regressions/components/partitioned_vector_2201.cpp
tests/unit/component/CMakeLists.txt
tests/unit/component/coarray_all_reduce.cpp
tests/unit/component/coarray.cpp
tests/unit/component/partitioned_vector_subview.cpp
tests/unit/component/partitioned_vector_view.cpp
tests/unit/component/partitioned_vector_view_iterator.cpp
tests/unit/parallel/segmented_algorithms/CMakeLists.txt
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_difference1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_difference2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_find1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_find2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_all_of1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_all_of2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_any_of1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_any_of2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_copy.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_exclusive_scan2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_exclusive_scan.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_fill.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_find2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_find.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_for_each.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_handle_values.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_inclusive_scan2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_inclusive_scan.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_iter.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_move.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_none1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_none2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_reduce.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_scan.hpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_target.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_binary1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_binary2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce_binary1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce_binary2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_scan2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_scan.cpp
tests/unit/parallel/segmented_algorithms/test_transform_binary2.hpp
tests/unit/parallel/segmented_algorithms/test_transform_binary.hpp
@sithhell
Copy link
Member

left a comment

I think this implementation in general still needs to be polished at various points.

One thing I am still unsure about is at which situations I would prefer Meta_Object over All_to_All.
I would prefer to have this implementation mature in the repository of the actual usecases (Phylanx) and add it to HPX proper once we know it a bit better.

hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
hpx/lcos/dist_object.hpp Outdated Show resolved Hide resolved
adding distributed_object to hpx
- add header file to docs CMake
- make constructor accepts sub locality list
- fix typos, redundant expression in transpose example
- not involving change to barrier.cpp
- move ensure_ptr to constructor
- fix order error for base member list
- make client's serialization function to delete status
- need to clean up documentation
- need to remove barrier in the constructor for all to all method
- need to change enum naming style

@weilewei weilewei force-pushed the dist_object_pr branch from 66ccdf4 to 1e149f1 Apr 26, 2019

@weilewei

This comment has been minimized.

Copy link
Author

commented Apr 29, 2019

What does a partitioned vector mean? Do you mean a distributed vector that each locality owns part of the data of the vector? Also, what does the friend mean exactly? Also, can you point out where does the duplicated code exist or potentially will be? Any suggestion will help for my next fix.

What I mean is that there is a class in HPX called partitioned_vector that is a distributed vector, it probably has a lot of similarities to your class (or parts of it) - here's a quick list of some code that either uses it or depends on it - have a look at these just to see if there's any duplicated functionality.
(By friends, I just mean that there are lot's of related classes/files in HPX that are connected)

git grep -l partitioned_vector | grep -v docs | sort
...
hpx/components/containers/container_distribution_policy.hpp
hpx/components/containers/partitioned_vector/detail/view_element.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_component_decl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_component.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_component_impl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_decl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_fwd.hpp
hpx/components/containers/partitioned_vector/partitioned_vector.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_impl.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_local_view.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_local_view_iterator.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_predef.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_segmented_iterator.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_view.hpp
hpx/components/containers/partitioned_vector/partitioned_vector_view_iterator.hpp
hpx/components/containers/unordered/unordered_map.hpp
hpx/components/containers/unordered/unordered_map_segmented_iterator.hpp
hpx/include/partitioned_vector.hpp
hpx/include/partitioned_vector_predef.hpp
hpx/include/partitioned_vector_view.hpp
hpx/include/serialization.hpp
hpx/runtime/serialization/partitioned_vector.hpp
src/components/containers/CMakeLists.txt
src/components/containers/partitioned_vector/CMakeLists.txt
src/components/containers/partitioned_vector/partitioned_vector_component.cpp
src/components/containers/partitioned_vector/partitioned_vector_component_double.cpp
src/components/containers/partitioned_vector/partitioned_vector_component_int.cpp
src/components/containers/partitioned_vector/partitioned_vector_component_std_string.cpp
tests/performance/local/CMakeLists.txt
tests/performance/local/partitioned_vector_foreach.cpp
tests/performance/network/algorithms/minmax_element_performance.cpp
tests/regressions/components/CMakeLists.txt
tests/regressions/components/partitioned_vector_2201.cpp
tests/unit/component/CMakeLists.txt
tests/unit/component/coarray_all_reduce.cpp
tests/unit/component/coarray.cpp
tests/unit/component/partitioned_vector_subview.cpp
tests/unit/component/partitioned_vector_view.cpp
tests/unit/component/partitioned_vector_view_iterator.cpp
tests/unit/parallel/segmented_algorithms/CMakeLists.txt
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_difference1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_difference2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_find1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_adjacent_find2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_all_of1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_all_of2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_any_of1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_any_of2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_copy.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_exclusive_scan2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_exclusive_scan.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_fill.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_find2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_find.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_for_each.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_handle_values.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_inclusive_scan2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_inclusive_scan.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_iter.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_move.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_none1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_none2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_reduce.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_scan.hpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_target.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_binary1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_binary2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce_binary1.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_reduce_binary2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_scan2.cpp
tests/unit/parallel/segmented_algorithms/partitioned_vector_transform_scan.cpp
tests/unit/parallel/segmented_algorithms/test_transform_binary2.hpp
tests/unit/parallel/segmented_algorithms/test_transform_binary.hpp

As our conversation with Dr. Kaiser, it seems we will update partitioned_vector in the future. Please suggest, if you have any ideas. Thanks!

@weilewei

This comment has been minimized.

Copy link
Author

commented Apr 29, 2019

Could you please add (documented) header files to this list, please?

added here:

"${PROJECT_SOURCE_DIR}/hpx/lcos/distributed_object.hpp"

@hkaiser

This comment has been minimized.

Copy link
Member

commented Jun 27, 2019

We have added a simplified version of this to Phylanx (see STEllAR-GROUP/phylanx#990) for now. Once we're happy with what we have over there we can reconsider adding it to HPX proper. Closing this for now.

@hkaiser hkaiser closed this Jun 27, 2019

@hkaiser hkaiser deleted the dist_object_pr branch Jun 27, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.