FillPatch on solid boundary (walls)

[pkufourier]

Recently I am using the InterpFromCoarseLevel & FillPatchTwoLevels near the wall (not periodic). And I have write a small testing code as attached, which is a simplifed version from the tutorial codes.
test_interp.zip

In this testing case, I have two levels of mesh. The lev 0 mesh is a 32x32 squre, while lev=1 is a small squre close to the j=0 wall, since the ErrorEst function's kernel is:

amrex::ParallelFor(bx,
[=] AMREX_GPU_DEVICE(int i, int j, int k) noexcept
{
if (i > 14 && i < 18 && j < 4)
tagfab(i, j, k) = tagval;
else
tagfab(i, j, k) = clearval;
});

In the InitData function, I only initialize the lev=0 data (with ghost cells), so that the lev=0 are all 1, while the lev=1 data will print as nan at all cells.

Then the FillCoarsePatch() is called with cell_bilinear_interp interpolator in the Evolve function. If the bndry_func is provided as nullptr or a dummy empty function, as the present status in the code, the lev=1 data will be printed as:

j=-1:nannannannannannannannannannannannannannan
j= 0:nannannannannannannannannannannannannannan
j= 1: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 2: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 3: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 4: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 5: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 6: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 7: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 8: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j= 9: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j=10: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j=11: 1 1 1 1 1 1 1 1 1 1 1 1 1 1
j=12: 1 1 1 1 1 1 1 1 1 1 1 1 1 1

We can see that the j=0 cells are still nan after interpolation. With the bilinear algorithm, if j=-1 and j=0 have values on lev=0, the j=0 of lev=1 should be a valid value after interp. However, right now it seems that the only way to fix this issue is to provide a effective bndry_func, as what has been commented in the functor.

I am wondering whether there is a way to fix this, so that the j=0 cells on lev=1 can get a wanted value. The reason is, usually we need to determine the j=-1 values by using the j=0 cells, other than set it with an explicit value. This problem should also exist in the :FillPatchTwoLevel() function.

[WeiqunZhang]

With the bilinear algorithm, if j=-1 and j=0 have values on lev=0, the j=0 of lev=1 should be a valid value after interp.

The coarse level data might be on a different process. Unless we communicate, we don't have access to it. But communication has cost. So it's usually better to try to fill it locally.

The reason is, usually we need to determine the j=-1 values by using the j=0 cells, other than set it with an explicit value. This problem should also exist in the :FillPatchTwoLevel() function.

Do you see a problem with no access to j=0 data or are you just worried about this could be an issue? If you do see a problem, let us know with a reproducer.

How do you fill j=-1 with j=0 data? It might have already been supported. If not, you have to write your own.

amrex/Src/Base/AMReX_BC_TYPES.H

Line 67 in 5b19359

enum mathematicalBndryTypes : int {

[pkufourier]

Thanks a lot for your reply. And I still have some additional questions.

1 Indeed, filling in the ghost cell’s data usually needs the neighbor block’s information, and if the neighbor is in another process, communication is required. But I am wondering what is the Array4 that BC functor(i.e. gpu_bndry_func(AmrCoreFill{})) implemented with? Do the FillPatch functions define another MultiFab and pass its data into the BC functor? The reason is, even if I filled the ghost cells in the BC functor, like data(iv) =10.0, the original Mulfifab’s ghost values are not changed.

2 Yes，the AMReX_BC_TYPES.H have provided several options for most situtations. But in some cases, the BC cells need to be adjusted in a more sophisticated way or even not locally (usually for precision or stability considerations). Consequently, not all the BCs can be written in the BC functor, or could be, but with a rather complicated manner. So, it will be more helpful if the Multifabs ghost cell values can be used directly. Of course, if the developer provided an empty BC functor, they must take care of the ghost cells before/after entering the FillPatch APIs.

3 I have tested the cell_cons_interp interpolator，and find out that in this manner, the lev=1, j=0 cells can be correctly interpolated even with an empty BC functor. I have looked up into the AMReX_Interpolator.H/cpp and AMReX_MFInterp_2D/3D.H. The CellConservativeLinear class have used several combinations of limiter and interp methos. So do we have any reference that described the detailed algorithm?

4 Meanwhile, how is the member parameter, i.e. do_linear_limiting in CellConservativeLinear determined? It seems that when calling the FillPatch APIs, no info about the limiter is passed in. So how can I specify this parameter, and where is it passed into the CellConservativeLinear object in the code (I mean in the FillPatch functions other than the args in CellConservativeLinear’s constructor)?

[WeiqunZhang]

1. FillPatch functions create temporary MultiFabs internally.
2. It might open a can of worms. So you should try to use the current functionalities.
3. I think most of them can be found in standard textbooks.
4. We use global variables to store objects with different runtime parameters. For example

CellConservativeLinear    lincc_interp;
CellConservativeLinear    cell_cons_interp(false);

In AMReX_Interpolater.cpp. Both lincc_interp and cell_const_interp are CellConservativeLinear objects. When you call FillPatch, you tell it which object to use, not just a type name.

[asalmgren]

Closing this issue due to lack of activity. Please re-open it if you feel there is is still an issue!