Skip to content

Commit

Permalink
Implement MPI_IN_PLACE for MPI Operations (#232)
Browse files Browse the repository at this point in the history 
* Extract the value of MPI_IN_PLACE during build phase
* Add non allocating version of MPI_Allreduce, support for MPI_IN_PLACE, and tests.
* Add non allocating version of MPI_Scatter, another method supporting MPI_IN_PLACE, and tests.
* Add non allocating version of MPI_Allgather, support for MPI_IN_PLACE, and tests.
* Add non allocating version of MPI_Allgatherv, support for MPI_IN_PLACE, and tests.
* Add non allocating version of MPI_Alltoall, support for MPI_IN_PLACE, and tests.
* Add non allocating version of Alltoallv and tests
* Modify definition of IN_PLACE and revert function signatures to the old type.
* Make the compiler check if a function can accept ConstantPtr.
* Allreduce:
 - Rename `allreduce(sendbuf, op, comm)` to `Allreduce(sendbuf, op, comm)` for consistency + fix test
 - Move `Allreduce!(send, recv, count op::Function, comm)` converting user provided-functions from `mpi-op.jl` to `mpi-base.jl`
* Reduce:
	- Add nonallocating version `Reduce!`
	- Move and rename `Reduce(send, recv, count op::Function, comm)` converting user provided-functions from `mpi-op.jl` to `mpi-base.jl`
* Add Reduce_in_place! function
* Modify allreduce tests to test N-dimensional tensors
* Gather: Add a non-allocating version
* Add Gather_in_place!
* Add non-allocating Gatherv and support for MPI.IN_PLACE + tests
* Add Gatherv_in_place!
* Add non-allocating Scatterv! and Scatterv_in_place
  • Loading branch information
@PhilipVinc @barche
PhilipVinc authored and barche committed Feb 7, 2019
1 parent 568c5bb commit 73936bb
Show file tree
Hide file tree
Showing 16 changed files with 877 additions and 111 deletions.
41 changes: 25 additions & 16 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -248,22 +248,31 @@ Julia Function (assuming `import MPI`) | Fortran Function
`MPI.Waitsome!` | [`MPI_Waitsome`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Waitsome.3.php)


#### Collective communication
Julia Function (assuming `import MPI`) | Fortran Function
---------------------------------------|--------------------------------------------------------
`MPI.Allgather` | [`MPI_Allgather`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Allgather.3.php)
`MPI.Allgatherv` | [`MPI_Allgatherv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Allgatherv.3.php)
`MPI.Alltoall` | [`MPI_Alltoall`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Alltoall.3.php)
`MPI.Alltoallv` | [`MPI_Alltoallv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Alltoallv.3.php)
`MPI.Barrier` | [`MPI_Barrier`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Barrier.3.php)
`MPI.Bcast!` | [`MPI_Bcast`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Bcast.3.php)
`MPI.Exscan` | [`MPI_Exscan`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Exscan.3.php)
`MPI.Gather` | [`MPI_Gather`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Gather.3.php)
`MPI.Gatherv` | [`MPI_Gatherv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Gatherv.3.php)
`MPI.Reduce` | [`MPI_Reduce`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Reduce.3.php)
`MPI.Scan` | [`MPI_Scan`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Scan.3.php)
`MPI.Scatter` | [`MPI_Scatter`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Scatter.3.php)
`MPI.Scatterv` | [`MPI_Scatterv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Scatterv.3.php)
#### Collective communication (assuming `import MPI`)
Non-Allocating Julia Function |Allocating Julia Function | Fortran Function | Supports `MPI_IN_PLACE`
--------------------------------------|---------------------------------------|-----------------------------------------------------------------------------------|-----------
`MPI.Allgather!` | `MPI.Allgather` | [`MPI_Allgather`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Allgather.3.php) | ✅
`MPI.Allgatherv!` | `MPI.Allgatherv` | [`MPI_Allgatherv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Allgatherv.3.php) | ✅
`MPI.Allreduce!` | `MPI.Allreduce` | [`MPI_Allreduce`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Allreduce.3.php) | ✅
`MPI.Alltoall!` | `MPI.Alltoall` | [`MPI_Alltoall`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Alltoall.3.php) | ✅
`MPI.Alltoallv!` | `MPI.Alltoallv` | [`MPI_Alltoallv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Alltoallv.3.php) | ❌
-- | `MPI.Barrier` | [`MPI_Barrier`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Barrier.3.php) | ❌
`MPI.Bcast!` | `MPI.Bcast!` | [`MPI_Bcast`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Bcast.3.php) | ❌
-- | `MPI.Exscan` | [`MPI_Exscan`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Exscan.3.php) | ❌
`MPI.Gather!` | `MPI.Gather` | [`MPI_Gather`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Gather.3.php) | `Gather_in_place!`
`MPI.Gatherv!` | `MPI.Gatherv` | [`MPI_Gatherv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Gatherv.3.php) | `Gatherv_in_place!`
`MPI.Reduce!` | `MPI.Reduce` | [`MPI_Reduce`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Reduce.3.php) | `Reduce_in_place!`
`MPI.Scan` | `MPI.Scan` | [`MPI_Scan`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Scan.3.php) | missing
`MPI.Scatter!` | `MPI.Scatter` | [`MPI_Scatter`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Scatter.3.php) | `Scatter_in_place!`
`MPI.Scatterv!` | `MPI.Scatterv` | [`MPI_Scatterv`](https://www.open-mpi.org/doc/v1.10/man3/MPI_Scatterv.3.php) | `Scatterv_in_place!`

The non-allocating Julia functions map directly to the corresponding MPI operations, after asserting that the size of the output buffer is sufficient to store the result.

The allocating Julia functions allocate an output buffer and then call the non-allocating method.

All-to-all collective communications support in place operations by passing
`MPI.IN_PLACE` with the same syntax documented by MPI.
One-to-All communications support it by calling the function `*_in_place!`, calls the MPI functions with the right syntax on root and non root process.

#### One-sided communication
Julia Function (assuming `import MPI`) | Fortran Function
Expand Down
4 changes: 4 additions & 0 deletions deps/gen_functions.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -112,5 +112,9 @@ int main(int argc, char *argv[]) {
printf("primitive type CInfo %d end\n", (int)(sizeof(MPI_Info) * 8));
printf("primitive type CWin %d end\n", (int)(sizeof(MPI_Win) * 8));

printf("\n");
printf("\n");
printf("const MPI_IN_PLACE_VAL = %d\n", ((int)MPI_IN_PLACE));

return 0;
}
3 changes: 3 additions & 0 deletions src/MPI.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ include("cman.jl")
const mpitype_dict = Dict{DataType, Cint}()
const mpitype_dict_inverse = Dict{Cint, DataType}()

# Initialize the void* IN_PLACE with the value extracted during the build phase.
const IN_PLACE = reinterpret(ConstantPtr, MPI_IN_PLACE_VAL)

"""
Setter function for mpitype_dict and mpitype_dict_inverse
"""
Expand Down
Loading

0 comments on commit 73936bb

Please sign in to comment.