AggregateNumbering #158
AggregateNumbering #158
Conversation
| // the owner rank of unowned entities, so we replace | ||
| // it with the passed-in sharing here before globalization | ||
| // which requires the rank information | ||
| n->setSharing(share); |
There was a problem hiding this comment.
This may not be required anymore since StaticSharing now supplies ownership information as well.. but may also be the cleaner solution in the end
apf/apfAggregateNumbering.h
Outdated
| class AggregateNumberingOf; | ||
| typedef AggregateNumberingOf<int> AggNumbering; | ||
| typedef AggregateNumberingOf<long> GlobalAggNumbering; | ||
| AggNumbering * createAggNumbering(Field * f, |
There was a problem hiding this comment.
This should probably just return Numbering* since AggNumbering inherits from Numbering and the entire point of the class is to be used as a Numbering
apf/apfAggregateNumbering.h
Outdated
| MPI_Comm cm, | ||
| Sharing * share = NULL); | ||
| void destroyAggNumbering(AggNumbering * n); | ||
| int getElementNumbers(AggNumbering * n, MeshEntity * e, NewArray<int> & numbers); |
There was a problem hiding this comment.
Ah but this needs an AggNumbering specifically rather than just a Numbering?
There was a problem hiding this comment.
Oh I might be able to just get rid of this since the underlying FieldData is now a UserData<int/long> which should operate correctly in the FieldDataOf<T>::getElementData() function I think.
apf/apfAggregateNumbering.cc
Outdated
| // init was called | ||
| int sz = PCU_Comm_Peers(); | ||
| frsts.setSize(sz); | ||
| MPI_Allgather(&frst,1,MPI_INTEGER, |
There was a problem hiding this comment.
I don't recall if PCU provides Allgather. If not, we should consider adding it to avoid/minimize direct MPI calls. (repeat for the other Allgather calls)
apf/apfAggregateNumbering.cc
Outdated
| { | ||
| int new_ownr = -1; | ||
| if(new_cm != MPI_COMM_NULL) | ||
| MPI_Group_translate_ranks(pcu_grp,1,&copies[ii].peer, |
There was a problem hiding this comment.
Can we add a wrapper call to PCU for this translation?
apf/apfAggregateNumbering.cc
Outdated
| new_cm = PCU_Get_Comm(); | ||
| MPI_Comm_group(pcu_cm,&pcu_grp); | ||
| if(new_cm != MPI_COMM_NULL) | ||
| MPI_Comm_group(new_cm,&new_grp); |
There was a problem hiding this comment.
Could we easily provide PCU wrappers for the group function and MPI_Group_translate_ranks?
wrtobin
force-pushed
the
wtobin-aggnumbering
branch
3 times, most recently
from
4c5c125
to
e089dae
Compare
|
Sure thing I'll resolve that momentarily. |
wrtobin
force-pushed
the
wtobin-aggnumbering
branch
from
7d3aa73
to
0c041b6
Compare
|
Resolved the conflicts, still need to add a test case for the PCU version of Allgather I added before this can be merged. |
wrtobin
force-pushed
the
wtobin-aggnumbering
branch
from
0c041b6
to
b773ac9
Compare
…g several operations to PCU including PCU_Allgather and rank translation between MPI_Comms as needed in the implementation of AggregateNumberings.
wrtobin
force-pushed
the
wtobin-aggnumbering
branch
from
b773ac9
to
ec2e97e
Compare
…nce they are likely more optimized
|
I hit a few |
|
Once PR #234 is merge I am going to start looking at getting this functionality in as well, at least supporting/exposing implicit fields/numberings so we can supply closed-form representations of the fields/numberings under the hood and still use the 'read-only' version that results for various purposes. |
Implementation of a read-only numbering type with minimal memory footprint to allow for novel block-structures in the matrices resulting from finite element assembly using these numbering. Mostly for use in the fusion projects, but they can be used as regular a Numbering as well if the user simply wants to reduce memory usage. Aggregation scopes are determined by MPI_Comms passed to the creation functions but the creation is still a globally parallel process across the typical PCU comm.
They inherit from the Numbering type and require only a singe integer or long per node, they are treated by the Mesh as normal Numbering or GlobalNumberings, and are written out to VTK as expected.
They cannot be written to so they cannot be synchronized (though they are synchronized during creation so are globally valid following creation).
Minimal changes have been made outside of the new files to introduce the new concept. Possibly the most impactful is changing several member functions of the NumberingOf class to be virtual member functions.