mpi: Mitigate SparseFunction setup costs #1720
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1720 +/- ##
==========================================
- Coverage 89.61% 86.93% -2.69%
==========================================
Files 206 201 -5
Lines 33216 32655 -561
Branches 4314 4277 -37
==========================================
- Hits 29766 28388 -1378
- Misses 2960 3758 +798
- Partials 490 509 +19
Continue to review full report at Codecov.
|
mloubout
force-pushed
the
mpi-sparse-setup
branch
3 times, most recently
from
02e2326
to
bf680e0
Compare
FabioLuporini
changed the title
mloubout
force-pushed
the
mpi-sparse-setup
branch
2 times, most recently
from
16ef69d
to
4388657
Compare
devito/mpi/distributed.py
Outdated
| return as_tuple(ret) | ||
| elif sum(index.shape) == 1: | ||
| return index | ||
| print(index.shape, self.ndim) |
There was a problem hiding this comment.
aside from this print, is this PR basically good to go?
There was a problem hiding this comment.
Sorry yes did the fix locally and then forgot about pushing so gotta redo it, basically is yes. Did you give it a spin?
devito/types/sparse.py
Outdated
| for r in self.grid.distributor.glb_to_rank(s): | ||
| ret.setdefault(r, []).append(i) | ||
| return {k: filter_ordered(v) for k, v in ret.items()} | ||
| dmap = self.grid.distributor.glb_to_rank(self._support) |
There was a problem hiding this comment.
single line return self. ... ?
| """ | ||
| This method is analogous to :meth:`_dist_scatter_mask`, although | ||
| the mask is now suitable to index into self's SubFunctions, rather | ||
| than into ``self.data``. | ||
| """ | ||
| return self._dist_scatter_mask[self._sparse_position] | ||
| return self._dist_scatter_mask(dmap=dmap)[self._sparse_position] |
There was a problem hiding this comment.
this is, essentially, to avoid recomputation of _dist_datamap when processing eg coordinates?
There was a problem hiding this comment.
yes, it's called like 8 times so this gain quite a bit to do it only once.
devito/mpi/distributed.py
Outdated
| index = np.expand_dims(index, axis=2) | ||
|
|
||
| ret = {} | ||
|
|
| retrieved. | ||
| """ | ||
| assert isinstance(index, (tuple, list)) | ||
| if len(index) == 0: | ||
| assert isinstance(index, np.ndarray) |
There was a problem hiding this comment.
Maybe this fixes the python 3.9 issue as well?
There was a problem hiding this comment.
possibly, where is the issue for it again?
There was a problem hiding this comment.
I think no issue was opened, need to check
mloubout
force-pushed
the
mpi-sparse-setup
branch
from
4388657
to
5b3d994
Compare
API: make SubFunction hashable
mloubout
force-pushed
the
mpi-sparse-setup
branch
from
5b3d994
to
65d5097
Compare
Some better use of numpy broadcasting to setup sparse functions
I get a ~X5 speedup on 256*256 receivers on my desktop.