Mesh connectivity

[avergnaud002]

Hello,
I'm new to the AMReX "community"!

With my research team, based in the I2M lab - Bordeaux France, we have the project to develop a CFD code based on CDO schemes (Compatible Discrete Operators ; these numerical schemes are part of the family of so-called mimetic schemes) and relying on an AMR mesh, plus an HPC component (with GPU and CPU). To do so we want to use AMReX to generate the mesh and the data structure.

CDO schemes needs to have access to mesh connectivity information. For example, for a given cell we want to have access to the neighboring cells and their faces, edges, etc... (and other information like faces centroid). The main difficulty we see at this stage concerns this connectivity information at coarse/fine interface (i.e. for cells with "hanging nodes"). In particular, when two boxes and/or patches are on different levels and on different rank, does the "dependency" of "child" and "parents" is available ? In other words, we want to know which cells are immediately neighboring a given cell, and have access to the information of these neighboring cells (fields, faces, edges, etc.)

Is there an easy way to access this type of connectivity information for meshes generated with AMReX ? Unfortunately, I couldn't find this information in the documentation... (such information seems available for EB, but is it the case for cells far from a boundary ?)

I don't know if I was clear?

I must specify that we want to develop our code in fortran, and therefore use the fortran interface proposed by AMReX.

Best Regards
Alban Vergnaud

[drummerdoc]

Welcome Alban,
In AMReX, if you know your level, index and centering, you automatically know where all your neighbors are in index space. If you also know the grid spacing at level 0 and the refinement ratio array, you know where you are physically, and if you have the distribution mapping of the distributed data structure you're looking into, you know which processor the data is on. Reach out if this is too cryptic.
-Marc

[drummerdoc]

finally, if data is defined on a BoxArray (an array of boxes), ba, that has N boxes, then one defines a DistributionMapping, d, that is essentially an integer array o N elements, one per box. then one can define a MultiFab to hold data on that BoxArray, distributed using d, such that d[I] is the rank that will own the data that lives on ba[I]. All processors have a copy of ba and d.

[avergnaud002]

Ok I think it is more clear to me. So if I resume, given a cell at level n, we always know if this cell is overlapped by cells at level n+1 (and if it is the case, which are these cells). And reciprocally, given a cell at level n+1, we always know which cell at level n it covers. And these two "search" do not need a communication between processors.

[drummerdoc]

Correct. All BoxArrays and DistributionMappings that are used to create distributed data are copied across all MPI ranks, so in practice, all ranks know who owns data at a particular index within a BoxArray at a level in the AMR hierarchy. Again, getting the actual data that lives there may require communication, but you know which rank owns it.

[avergnaud002]

Ok, thank you for these answers to my newbie questions about AMReX.

[avergnaud002]

One last precision/interrogation:
To know if a given cell at level n is covered by cells at level n+1 (and what are these cells), do we need to do some sort of loops over boxes and/or cells at level n+1 and "detect" some intersections, or is there a more direct way to have this information ? And reciprocally, to know which cell(s) at level n contains a given cell at level n+1 ?

[zingale]

one thing to keep in mind is that a fine cell can have multiple coarse parents. It is not requires that a fine cell be completely contained in a single coarse cell.

[avergnaud002]

Thank you for this clarification which indeed was not obvious at first sight.