Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set guard cells in corners with mixed boundary conditions #3842

Open
wants to merge 23 commits into
base: development
Choose a base branch
from

Conversation

dpgrote
Copy link
Member

@dpgrote dpgrote commented Apr 15, 2023

In the evolve loop, whenever fields are updated, the WarpX specific boundary conditions are applied first (e.g. PEC) and then the periodic (in a call to the appropriate FillBoundary routine). However, the FillBoundary routines will only fill guard cells using valid cells. For mixed BCs then, the guard cells in the corners don't get set to the periodic repeat of the side guard cells along the periodic direction. So, since WarpX applies its BCs first, the changes that are made in the side guard cells are not copied to the corners along the periodic direction, leaving them either zero or with incorrect values.

The fix is to call the PEC boundary routines after the FillBoundary routines, moving them out of the EvolveE and EvolveB routines and making direct calls to them.

Fix #3900

Tasks:

  • Move ApplyEfieldBoundary for FDTD
  • Move ApplyBfieldBoundary for FDTD
  • Do the same changes of PSATD
  • Fix CI benchmarks as needed. With careful checking, the tests were failing because the development version was not correct, with the code using corner cells that had not been updated.

@dpgrote dpgrote changed the title [WIP]Set guard cells in corners with mixed boundary conditions Set guard cells in corners with mixed boundary conditions May 6, 2023
@WeiqunZhang
Copy link
Member

There are amrex functions for filling ghost cells at the corner of periodic/non-periodic boundaries. https://github.com/AMReX-Codes/amrex/blob/68aa69a11750b3f6ac36ec13b26249f0be80d527/Src/Base/AMReX_FabArray.H#LL1098C35-L1098C43 Not sure if that could be an easier fix than the changes in this PR.

There is also a known issue with PEC boundaried. #3900. Fixing that by passing the correct number of ghost cells to SumGuargCells should also fix the corners.

(Admittedly, I have not read the changes in this PR in detail.)

@dpgrote
Copy link
Member Author

dpgrote commented May 8, 2023

@WeiqunZhang FillBoundary is already being called with the number of ghost cells specified. This seems to be the same as calling EnforcePeriodicity with nghost since both are calling FBEP_nowait. One of the reasons I like my method is that it puts the handling of the boundary in one place, rather than scattered in different routines. In fact, I think I'll update this pull request to put the Apply*FieldBoundary in the 'FillBoundary` calls to reduce code duplication.

The fix for #3900 may be the same as is done here.

@ax3l ax3l added bug Something isn't working bug: affects latest release Bug also exists in latest release version component: parallelization Guard cell exchanges and particle redistribution component: boundary PML, embedded boundaries, et al. labels May 8, 2023
@ax3l ax3l requested a review from RemiLehe May 8, 2023 23:06
@ax3l ax3l requested a review from WeiqunZhang May 8, 2023 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug: affects latest release Bug also exists in latest release version bug Something isn't working component: boundary PML, embedded boundaries, et al. component: parallelization Guard cell exchanges and particle redistribution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PEC boundaries: Copy values in guard cells
4 participants