Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilities::MPI::create_mpi_data_type_n_bytes #12964

Merged
merged 3 commits into from Nov 19, 2021

Conversation

tjhei
Copy link
Member

@tjhei tjhei commented Nov 17, 2021

part of #12752 and #12873

Copy link
Member

@bangerth bangerth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The design here invited resource deallocation failures because the type has no destructor associated with it. The function is probably still useful, but in a follow-up it might be useful to move the function into an internal namespace and write a wrapper function that returns something like std::unique_ptr<MPI_Datatype, ...> with a custom deallocation function as second argument that takes care of the call to MPI_Type_free. I can probably write that for you.

include/deal.II/base/mpi.h Outdated Show resolved Hide resolved
source/base/mpi.cc Show resolved Hide resolved
// https://github.com/jeffhammond/BigMPI/blob/5300b18cc8ec1b2431bf269ee494054ee7bd9f72/src/type_contiguous_x.c#L74
// (code is MIT licensed)

// We create an MPI datatype that has the layout A*nB where A is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean this?

Suggested change
// We create an MPI datatype that has the layout A*nB where A is
// We create an MPI datatype that has the layout A*n+B where A is

source/base/mpi.cc Outdated Show resolved Hide resolved
using namespace dealii;

void
test_data_type(long long n_bytes)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
test_data_type(long long n_bytes)
test_data_type(const std::uint64_t n_bytes)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. what is your preference for the argument ofcreate_mpi_data_type_n_bytes()? MPI_Count, size_t, std::uint64_t?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole point of this function is to work around limitations in MPI data type sizes :-) So let's do std::uint64_t.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or std::size_t. Maybe that's the better choice because that's always going to be big enough to address all elements of any array you can think of.

@tjhei
Copy link
Member Author

tjhei commented Nov 18, 2021

and write a wrapper function that returns something like std::unique_ptr<MPI_Datatype, ...> with a custom deallocation function as

I am not a big fan of mixing c-style MPI calls (which will happen with this newly created MPI_Datatype) with c++ style. See duplicate_communicator for an example that is handled similarly. I changed to return the result by value. I am assume this works because this is typically some kind of handle (32 or 64 bit int). Do you like that better?

* </code>
*/
MPI_Datatype
create_mpi_data_type_n_bytes(std::size_t n_bytes);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
create_mpi_data_type_n_bytes(std::size_t n_bytes);
create_mpi_data_type_n_bytes(const std::size_t n_bytes);

* else
* {
* MPI_Datatype bigtype =
* Utilities::MPI::create_mpi_data_type_n_bytes(buffer.size());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indent this line a bit more

@@ -295,6 +297,85 @@ namespace Utilities



MPI_Datatype
create_mpi_data_type_n_bytes(std::size_t n_bytes)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
create_mpi_data_type_n_bytes(std::size_t n_bytes)
create_mpi_data_type_n_bytes(const std::size_t n_bytes)

@tjhei
Copy link
Member Author

tjhei commented Nov 19, 2021

All good points. Thanks.

@bangerth bangerth merged commit 108f08a into dealii:master Nov 19, 2021
@tjhei tjhei deleted the mpi_create_big_datatype branch November 19, 2021 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants