New Implicit Solver interface with options to select Picard or Newton (JFNK) for the nonlinear solver. #4736
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ctions used with the implicit solvers. Only has ComputeRHSE() and ComputeRHSB() functions thus far.
…unctions in place of EvolveE and EvolveB functions.
…ions used by implicit solvers. Moved several functions from WarpXOneStepImplicitPicard.cpp to here.
… input file using algo.nonlinear_solver = picard, for example. Default is picard. It is not used yet.
…mplicit and SemiImplicit, respectively. This affects parsing from input file. algo.evolve_scheme = implicit_picard ==> theta_implicit. Have not updated Docs yet.
…instance of this class. OneStepImplicitEM is now done here. This class owns all things pertaining to the implicit integrator. Also added a NonlinerSolvers class, but it is not used yet.
…step is to move it there.
…e the nonlinear equation.
…ains all of the necessary functions required by the linear operator template parameter for AMReX_GMRES.
…ves in the implicit solver class. This ensures that 1 and only 1 instance of this object will be defined. dotProduct and norm can now be called through the implicit sovler class.
…::Define() and into Newton::Define()
… Picard particle solver parameters in ImplicitSolverEM class. Using a new formula for the epsilon used in the finite-difference Jacobian action calculation that is suitable for large absolute norms of the solution vector.
…/ImplicitSolvers/
|
@JustinRayAngus Sorry for the delay in reviewing this PR. I am finally giving it a finally round of review. |
RemiLehe
reviewed
May 29, 2024
RemiLehe
reviewed
May 29, 2024
I believe this has been done. Let me know if it looks like anything else should be moved. |
RemiLehe
reviewed
May 31, 2024
RemiLehe
reviewed
May 31, 2024
RemiLehe
reviewed
May 31, 2024
RemiLehe
reviewed
May 31, 2024
RemiLehe
reviewed
May 31, 2024
RemiLehe
reviewed
May 31, 2024
RemiLehe
reviewed
May 31, 2024
RemiLehe
reviewed
May 31, 2024
…ne added to the declaration.
for more information, see https://pre-commit.ci
RemiLehe
approved these changes
Jun 2, 2024
|
Thanks a lot for this PR!! |
Haavaan
pushed a commit
to Haavaan/WarpX
that referenced
this pull request
Jun 5, 2024
… (JFNK) for the nonlinear solver. (ECP-WarpX#4736) * created Source/FieldSolver/WarXImplicitFieldsEM.cpp file to store functions used with the implicit solvers. Only has ComputeRHSE() and ComputeRHSB() functions thus far. * OneStep_ImplicitPicard routine now uses ComputeRHSE and ComputeRHSB functions in place of EvolveE and EvolveB functions. * created Source/Evolve/WarpXImplicitOps.cpp file to store common functions used by implicit solvers. Moved several functions from WarpXOneStepImplicitPicard.cpp to here. * added NonlinearSolver:: enum. Options are picard and newton. Set from input file using algo.nonlinear_solver = picard, for example. Default is picard. It is not used yet. * changed EvolveScheme::ImplicitPicard and SemiImplicitPicard to ThetaImplicit and SemiImplicit, respectively. This affects parsing from input file. algo.evolve_scheme = implicit_picard ==> theta_implicit. Have not updated Docs yet. * NonlinearSolver ==> NonlinearSolverType * intermediate commit. Added an ImplicitSolversEM class. WarpX owns an instance of this class. OneStepImplicitEM is now done here. This class owns all things pertaining to the implicit integrator. Also added a NonlinerSolvers class, but it is not used yet. * cleaning up ImplicitSolverEM::OneStep(). * more refactoring of ImplicitSovlerEM class. * WarpXFieldVec ==> WarpXSolverVec * removed depricated functions. WarpXSolverVec has zero ghost. * ImplicitSolverEM::OneStep now looks exactly like Picard solver. Next step is to move it there. * ImplicitSolverEM::OneStep() now uses the Picard solver object to solve the nonlinear equation. * changed where implicit solver parameters are printed. * refactoring of WarpXImplicitOps.cpp * added NewtonSolver.H file. Doesn't work yet. * adding more functionality to WarpXSovlerVec class. * added JacobianFunctionJFNK.H file in NonlinarSolvers/ folder. It contains all of the necessary functions required by the linear operator template parameter for AMReX_GMRES. * dotMask object used for dot product from Linear Function class now lives in the implicit solver class. This ensures that 1 and only 1 instance of this object will be defined. dotProduct and norm can now be called through the implicit sovler class. * moved temporary linear_function and GMRES testing lines out of Picard::Define() and into Newton::Define() * intermediate commit. JFNK almost ready. * small refactoring of PreRHSOp() and PostUpdateState() functions. * cleaning things up. * Newton solver runs. GMRES runs. Next step is to do Particle-suppressed (PS) part. * minor clean up. * fixed typo in convergence message for Newton and Picard solvers. * changed how PostUpdateState() is used in implicit solver. Now parsing Picard particle solver parameters in ImplicitSolverEM class. Using a new formula for the epsilon used in the finite-difference Jacobian action calculation that is suitable for large absolute norms of the solution vector. * Picard method for particle is now being used. PS-JFNK works. * moved WarpXImplicitOps.cpp from Source/Evolve/ to Source/FieldSolvers/ImplicitSolvers/ * minor cleanup. PostUpdateState() ==> UpdateWarpXState(). * Moved the particle convergence check into its own function. * added increment function to WarpXSolverVec class. * removed some commented out lines in JacobianFunctionJFNK.H * removed a_tol condition for print message when maximum iterations reached in nonlinear solvers. Newton solver iter is base zero now. * cleaned up picard method for self-consistent particle update. added ablastr warning message for particles that don't converge after the maximum number of iterations. * fixed small bug in PicardSolver related to absolute tolerance check for convegence. Default absolute tolerance values are set to zero. * the mask used to compute the dot product of a WarpXSolverVec is now owned by the WarpXSolverVec class rather then the time solver class. It is a static member to avoid multiple definitions. * defined accessors for Efield_fp and Bfield_fp Vectors owned by WarpX. ImplicitSolver is no longer a friend class of WarpX. Small tidy for PicardSolver.H. * SemiImplicitEM and ThetaImplicitEM are now their own independent derived classes from the base ImplicitSolver class. * added algorithm descriptions to the top of the SemiImplicitEM.cpp and ThetaImplicitEM.cpp files. * updated appropriate files in Examples and Regression folders to reflect new changes. * updating docs. * JacobianFunctionJFNK.H ==> JacobianFunctionMF.H * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * need to call clear on the static vector m_dotMask owned by the WarpXSovlerVec class to avoid malloc_consolidate(): unaligned fastbin chunk detected error messages at simulation finish. * moved WarpXSovlerVec class from ../ImplicitSolvers/Utils/ to ../ImplicitSolvers/. Utils directory is deleted. * ImplicitPushXP: GPU Support for Convergence Test * cleaning up. * Atomic: Fix type of `1` * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bug fix. * Update Source/FieldSolver/ImplicitSolvers/WarpXSolverVec.H Co-authored-by: Revathi Jambunathan <41089244+RevathiJambunathan@users.noreply.github.com> * Remove Zero * adding Copyright lines to top of files. Removed commented code. * Prevent calling the implicit solver if not initialized * More robust if conditions * set implicit verbose to warpx verbose * Update Source/FieldSolver/ImplicitSolvers/ThetaImplicitEM.cpp * Simplify call to updates of E * changed benchmarks_json file names as needed. * using warpx.verbose * clang-tidying * changed header names. * clang-tyding * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * more clang-tyding * clang tidy again * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * UpdateElectricFiled ==> SetElectricFieldAndApplyBCs * clang tidy * more clang tidy * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * prohibiting copy and move constructors for solver classes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed bug in move constructor in WarpXSovlerVec. * slight refactoring of ThetaImplicitEM class. * reducing divisions in Picard method for particles. * small cosmetic changes to implicit solvers. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed commented out code. * updating Docs and adding briefs. * Fix HIP compilation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix const * fixed indent. updated comments. * New Python test: Particle-Boundary interaction (ECP-WarpX#4729) * enable the diagnostic of ParticleScraping in Python * Update picmi.py * Update picmi.py * new test * python update * modification of the script * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update PICMI_inputs_rz.py * json * update * Update PICMI_inputs_rz.py * Update particle_boundary_interaction.json * Update PICMI_inputs_rz.py * Update PICMI_inputs_rz.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update PICMI_inputs_rz.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix hanging script in parallel * Make the test executable * Update analysis script * Update particle_containers.py * Update PICMI_inputs_rz.py * Update analysis.py * Update analysis.py * Update particle_containers.py * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Adding normal components to regular boundary buffer (ECP-WarpX#4742) * first draft * adding normal only * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update ParticleBoundaryBuffer.cpp * Update ParticleBoundaryBuffer.cpp --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add function to set domain boundary potentials from Python (ECP-WarpX#4740) * add function to set domain boundary potentials from Python * switch function arguments to form `potential_[lo/hi]_[x/y/z]` and add to docs * clean up `ablastr/fields` (ECP-WarpX#4753) * move PoissonInterpCPtoFP to Interpolate.H * concatenate nested namespaces * Split clang-tidy CI test into 4 to improve performances (ECP-WarpX#4747) * split clang-tidy checks to improve performances * rename folders and tests * fix concurrency * Simplify --------- Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Replace links to learn git (ECP-WarpX#4758) * Replace links to learn git * Bugfix in `fields.py` for GPU run without `cupy` (ECP-WarpX#4750) * Bugfix in `fields.py` for GPU run without `cupy` * apply suggestion from code review * Release 24.03 (ECP-WarpX#4759) * AMReX: 24.03 * pyAMReX: 24.03 * WarpX: 24.03 * Implement stair-case Yee solver with EB in RZ geometry (ECP-WarpX#2707) * Allow compilation with RZ EB * Do not push cells for RZ Yee solver, when covered with EB * Fix compilation errors * Fix additional compilation errors * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix additional compilation errors * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add automated test * Add automated test * Fix path in tests * Enable parser in RZ * Update example script * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Clean-up PR * Initialize EB quantities * Modified EM field initialization in 2D with EB * Typo fix * Typo fix * Ignoring unused variables correctly * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Correct condition for updating E * Correct update of B * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update B push * Update input script * Revert "Update input script" This reverts commit 5087485. * Update initialization * Updated test * Move test to a different folder * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add test for WarpX-test.ini * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix path for tests * Update test description * Update test metadata * Add checksum file * Revert changes * Revert changes * Change lx to lr * Revert "Change lx to lr" This reverts commit be3039a. * Change lx to lr --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: lgiacome <lorenzo.giacome@cern.ch> * AMReX/pyAMReX/PICSAR: Weekly Update (ECP-WarpX#4763) * AMReX: Weekly Update * pyAMReX: Weekly Update * clean up (ECP-WarpX#4761) * Fix compilation * updating some function names to contain Implicit. * fixed bug that caused segfault on GMRES restart. * parsing GMRES restart length from input file. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update field accessor * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed +, -, and * operators from WarpXSolverVec class. These operators encourage inefficient vector arithmetic. * fix/workaround to field accessor issue. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * changing implicit-scheme related header files in WarpX.H and WarpX.cpp * WarpX::max_particle_iterations ==> WarpX::max_particle_its_in_implicit_scheme and WarpX::particle_tolerance ==> WarpX:particle_tol_in_implicit_scheme * updating docs. * updating docs again. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding comments about the template parameters Vec and Ops used in the NonlinearSolver class. * adding comments. addressing comments from PR. * Ensure that laser particles can converge in the implicit solver * Add braces to make clang-tidy happy * moving nonlinear solver parameters to base ImplicitSolver class * mirrors not to be used with implicit schemes. * moved more to base implicit solver class. adding comments. * removed some WarpXSolverVec functions. Updated comments. * clang tidy complains when removing default copy constructor in WarpXSolverVec.H * amrex::ExecOnFinalize(WarpXSolverVec::clearDotMask) * WarpXSolverVec (const WarpXSolverVec&) = delete * updating briefs for nonlinear solvers. * adding loop over levels. * static cast amrex::Vector.size() to int * updating docs for nonlinear solvers. * adding gmres.restart_length to docs. * fixed typos in docs. * Removed PreRHSOp() call from nonlinear solvers. * clang tidy. * Prohibit = operator for WarpXSolverVec. Using Copy() instead. * Document PICMI function `LoadInitialField` * updating comments in WarpXImplicitOps.cpp * moved static member m_dotMask definition to the header file with inline added to the declaration. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed indent. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Debojyoti Ghosh <debojyoti.ghosh@gmail.com> Co-authored-by: Revathi Jambunathan <41089244+RevathiJambunathan@users.noreply.github.com> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> Co-authored-by: Weiqun Zhang <weiqunzhang@lbl.gov> Co-authored-by: Eya D <81635404+EyaDammak@users.noreply.github.com> Co-authored-by: Roelof Groenewald <40245517+roelof-groenewald@users.noreply.github.com> Co-authored-by: Arianna Formenti <ariannaformenti@lbl.gov> Co-authored-by: Luca Fedeli <luca.fedeli@cea.fr> Co-authored-by: lgiacome <lorenzo.giacome@cern.ch>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are two new folders:
Source/FieldSolver/ImplicitSolvers/andSource/NonlinearSolvers/. All of the containers needed by the implicit time solver and corresponding nonlinear/linear solvers are now owned by the Implicit Solver and nonlinear/ linear solver class rather than by WarpX. WarpX owns a pointer to ImplicitSolver class object. The base functions in the implicit solver class are those needed to interact with WarpX ( such as OneStep ) and with the nonlinear/linear solvers. The current choices for implicit time solvers are SemiImplicitEM and ThetaImplicitEM. The current nonlinear solver choices are Picard and Newton. The Newton solver is coupled to AMReX_GMRES for solving the linear system needed to obtain the Newton step at each Newton iteration in the standard JFNK method. Both the semi-implicit and theta-implicit algorithms can use particle-suppressed (PS)-JFNK for the nonlinear solver. However, it is not yet setup to use mass matrices or preconditioning, and thus it is not yet efficient.