I/O performance hints for Summit #2495
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ax3l
added
component: documentation
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
ax3l
reviewed
Nov 11, 2021
ax3l
reviewed
Nov 15, 2021
ax3l
reviewed
Nov 15, 2021
ax3l
reviewed
Nov 15, 2021
ax3l
reviewed
Nov 15, 2021
ax3l
reviewed
Nov 15, 2021
ax3l
reviewed
Nov 15, 2021
ax3l
approved these changes
Nov 15, 2021
ax3l
reviewed
Nov 15, 2021
ax3l
reviewed
Nov 15, 2021
roelof-groenewald
added a commit
to ModernElectron/WarpX
that referenced
this pull request
Nov 19, 2021
* Add Python Wrappers for F,G in PML (ECP-WarpX#2481) * Add Python Wrappers for F,G in PML * Add Getters for F,G Nodal Flags * Fix Bug in <F,G>FPPMLWrapper (Default Level) * Fix Bug in F,G Nodal Flags * Use GetPML Method for F,G Nodal Flags * PICMI: Add max_grid_size, blocking_factor in (x,y,z) (ECP-WarpX#2524) * PICMI: Add amr.max_grid_size_<x,y,z> * Set All Flags in Python * PICMI: Add amr.blocking_factor_<x,y,z> * added wrappers to get particle structs for the particles in the boundary buffers (ECP-WarpX#2498) * Doxygen: Fix Docs (ECP-WarpX#2526) * Fix Bug with Tilebox for G in PML (ECP-WarpX#2527) * Fix Bug with Tilebox for G in PML * Reset Benchmark * AMReX/PICSAR: Weekly Update (ECP-WarpX#2533) Weekly update to latest AMReX. Weekly update to latest PICSAR (no changes). ``` ./Tools/Release/updatePICSAR.py ./Tools/Release/updateAMReX.py ``` * Install pre-commit (ECP-WarpX#2532) * Add pre-commit Add basis for automated pre-commit checks. Install locally via: ```bash python3 -m pip install -U pre-commit pre-commit install ``` See: https://pre-commit.com * Cleanup: Whitespaces * Cleanup: requirements.txt order * Summit: Update Numpy Hints (ECP-WarpX#2535) Make sure `numpy` can be rebuilt when and were needed. To achieve that, move numpy-specific installation hints on OpenBLAS to the WarpX profile. * Fix some issues with Fujitsu compiler (ECP-WarpX#2529) * make some code compilable with Fujitsu compiler in clang mode * update documentation * Fix ECP-WarpX#2522: Gaussian beam positions do not change with warpx.random_seed (ECP-WarpX#2523) * Draw Gaussian beam position with amrex random engine * Update benchmarks * Update tolerance in space-charge tests * Update benchmark for space charge initialization test * Update benchmarks * Update benchmark * Clean-up code * Update benchmarks * ECP-WarpX#2534: Don't access position vector values beyond the configured dimension (ECP-WarpX#2536) * ECP-WarpX#2534: Don't access position vector values beyond the configured dimension * Fix particle position component used in XZ configuration * Handle 1D case * Move values only used in scraping function into inside-boundary condition * Error out if scraping from EB in RZ * Spack Development: macOS & GNUmake (ECP-WarpX#2545) - Add macOS hints for OpenMP - Add hints for running GNUmake regression tests locally * Regression Tests: OMP on (ECP-WarpX#2548) We generally run only with one OpenMP threads at the moment, but disabling OpenMP altogether causes an extra compile, which slows down CI. * Tests: numthreads to 1 (ECP-WarpX#2546) * Tests: numthreads to 1 We already hack this option to read `numthreads = 1` already for benchmarks, thus we remove the confusing other values now. * Prepare for CI: Do not Overwrite `numthreads` * std::ifstream: Defensive Patterns (ECP-WarpX#2547) Add failure handling if inputs in `std::ifstream`s cannot be opened or have problems seek-ing through them. This should catch I/O errors early. * openPMD: 0.14.3 (ECP-WarpX#2551) Automatically copy and compile openPMD-api 0.14.3, but still supporting the 0.14.2+ range (ECP-WarpX#2150). The 0.14.3 release solves ABI incompatibilities in C++14/17 mixed builds, among other issues (mainly read). * Add Ar and Xe to pre-defined particle types. (ECP-WarpX#2549) * Added Ar and Xe to pre-defined particle types * Added Boltzmann's constant to warpx parser * Updated documentation * 2D EM solver with EB (ECP-WarpX#2401) * adding the FieldProbe * adding missing file * updating makefile * fixing host-device problem * Revert "fixing host-device problem" This reverts commit 801e6fc. * fixing host-device problem * making some variables const * adding a few comments * Adding the FieldProbe to the documentation * making the probe mpi-safe * added field probe to reduced diag test * added field probe to reduced diag analysis * using cell-centered fields in probe diag * removed a few typos * Interpolating to the point instead oof cell center * bug fix * improved a comment * updated documentation * Undone an outdated change * improving some variable names * improving the box extraction * making the interpolation order an input parameter * fix a typo * setting the field values to zero if the point is not in the domain * skipping the communication if probe proc is IO prcessor * Fixed typo in documentation Co-authored-by: Neïl Zaim <49716072+NeilZaim@users.noreply.github.com> * Updating an header * Added a comment on the probe position * tidying up the analysis script * fixed a comment * removing an unused include * improving the parsing of parameters * fixing some comments * making some variables const * changed some ParticleReal into Real * using better tags in MPI communication * Making field probe work in 2D * making a variable const * initializing y_probe only in 3D * tidying up a line which is common to 2D and 3D * making a variable constexpr * adding a _rt * checking that the probe location is in one of the processors * removing a useless if condition * Fixing the initialization in 2D * Avoiding scrape particles in 2D (it segfaults) * Adding a test for 2D EB * Fixed the areas initialization * Initializing to zero some multifabs * Modified the ECT solver to make it work in 2D * Modified the cell extensions to make them work in 2D * Improved 2D cube test * Added 2D rotated cube test * Adding the 2d analysis script and CI * Removed an unused import from the analysis script * Ignoring some unused variables * Fixing the number of dimensions in the 2d test * Added missing analysis for ECT * Enabled again 2d particles scraping * Fixing the test_name with the general logic * Fixing the test_name with the general logic * Removed some commented code * Modified several preprocessor directives to check consistency EB-dimension * Added missing semicolons * Fixed a preprocessor directivew * Fix typo: WARPX_DIM_XZ * Improving some comments Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Adding some more consistency checks * Adding some more consistency checks * Fixed a typo Co-authored-by: Neïl Zaim <49716072+NeilZaim@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Fix Instability in PML with PSATD (ECP-WarpX#2558) * Fix Instability in PML with PSATD Damping in PML should be applied before the communications between the regular grids and PML, and between PML grids take place, otherwise the ghost cells are filled with lagged information, which results in an instability. Closes ECP-WarpX#2525. * Update checksum of the pml_psatd_dive_divb_cleaning test * Bugfix in load balancing routine (ECP-WarpX#2555) * add remake of phi_fp during load balancing RemakeLevel * added phi_cp remake to RemakeLevel function * revert changes from previous commit * I/O performance hints for Summit (ECP-WarpX#2495) * Fix conflict with upstream * Apply suggestions from code review * Remove space in the end of lines * Include suggestions from PR review * Generalize ROMIO Hints in Batch Scripts * Fix Comment * Fix Comment * Remove duplication * Formatting Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * AMReX/PICSAR: Weekly Update (ECP-WarpX#2559) * AMReX: Weekly Update * PICSAR: Weekly Update * Docs: Fix .rst Label in PML, rm .tex (ECP-WarpX#2537) Fix an auto-converted label in a `.rst` file for the manual. Remove the `PML.tex` file. * Add 2D circle EB test (ECP-WarpX#2538) * Added embedded_circle test * Add embedded_circle test files * Removed diag files * removed PICMI input file * Update to use default regression analysis * Added line breaks for spacing * Added description * Fixed benchmark file * Added load balancing to test * Commented out load_balancing portion of test. This will be added back in once load balancing is fixed. Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Fixes to the EB init (ECP-WarpX#2565) * Avoid code duplications in ECT face extension (ECP-WarpX#2557) * Refactoring the nborrow functions * Refactoring the one cell extension * Refactoring the eight cells extension * Enabling 2D * Bug fix * Some more improvements * Fixing templates * Switching the order of templates and AMREX_GPU_DEVICE * Adding the needed AMREX_GPU_DEVICE in WarpX.H * Fixing GPU related issues * Fixed a for loop bound * Making the new functions free * Suggestion from review * Suggestion from review Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Suggestion from review Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Improve loops over dimensions for 2D Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Enhanced inline documentation of EB related data (ECP-WarpX#2562) * Enhanced inline documentation of EB related data * Added ECT to the glossary * Made the EB documentation doxygen-compatible * Clean up input files for tests with MCC (ECP-WarpX#2552) * Added embedded_circle test * Add embedded_circle test files * Removed diag files * removed PICMI input file * Update to use default regression analysis * Added line breaks for spacing Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Added description * Added Ar and Xe to pre-defined particle types * added Boltzmann's constant to pre-defined constants and cleaned up the MCC CI test input * Added Boltzmann's constant to warpx parser * cleaned up embedded circle CI test input * Remove duplicate entry. Co-authored-by: kzhu-ME <kevin.zhu@modernelectron.com> Co-authored-by: Kevin Z. Zhu <86268612+KZhu-ME@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Cell Center Macroscopic Properties (ECP-WarpX#2530) * Cell Center Macroscopic Properties * Commit Suggestions from PR Review * Fix Error for 2D Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> * added superLU solver example to docs (ECP-WarpX#2567) * Fix: GNUmake Python Link -g (ECP-WarpX#2568) On CPU links of the GNUmake Python lib, we forgot our `-g`, which we add to all build and optimization types. THis is part of the `LINKFLAGS` variable. * Apply PEC to Split PML Fields (ECP-WarpX#2541) * WarpXMovingWindow.cpp: Add `amrex::` Prefix (ECP-WarpX#2579) * Bug fixes and cleanup in load balancing (ECP-WarpX#2563) * added helper function to rebuild MultiFabs and iMultiFabs during load balancing and included rebuilding of EB multifabs * added redistribute call for the particle boundary buffer during load balancing * consistently use DistribtionMap rather than dmap in ElectrostaticSolver.cpp * applied suggested changes from code review by Phil Miller * removed default argument for redistribute in RemakeMultiFab * removed RemakeMultiFab() as a member of WarpX * Only remake EB multifabs if they are used Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> * adapted existing particle scraping test (PICMI version) to also cover the redistribution of particle buffers from load balancing * added redeclaring of m_borrowing * Move redeclaring of m_borrow inside if statement for ECT solver algorihtm Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> * added calls to MarkCells and ComputeFaceExtensions * fixed issue causing CI test to fail and copied conditionals from WarpXInitData.cpp to recompute EB quantities * Guard cells communication for EB data when re-gridding (#105) * Add 2D circle EB test (ECP-WarpX#2538) * Added embedded_circle test * Add embedded_circle test files * Removed diag files * removed PICMI input file * Update to use default regression analysis * Added line breaks for spacing * Added description * Fixed benchmark file * Added load balancing to test * Commented out load_balancing portion of test. This will be added back in once load balancing is fixed. Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Added guard cells communication for EB data in regridding Co-authored-by: Kevin Z. Zhu <86268612+KZhu-ME@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * moved all EB grid data calculations to a new function InitializeEBGridData() which is now called by both WarpX::InitLevelData and WarpX::RemakeLevel * Fix typo in doc string. Co-authored-by: Phil Miller <unmobile+gh@gmail.com> Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> Co-authored-by: Kevin Z. Zhu <86268612+KZhu-ME@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Phil Miller <unmobile+gh@gmail.com> * WarpXComm.cpp: Use Explicit Types, not `auto` (ECP-WarpX#2578) * WarpXComm.cpp: Use Explicit Types, not `auto` * Use MultiFab* const instead of MultiFab* const& * Add WARPX_PROFILE calls to each python callback. (ECP-WarpX#2573) When python callbacks take some time, this is useful as otherwise many callbacks are lumped together in WarpX::Evolve::step. * LaserInjectionFromTXYEFile Test: Use MPI (ECP-WarpX#2577) * WarpX tests: All MPI Enable MPI for the one regression test that does not use it. Still uses one rank there. This saves a compile per CI run. * add numprocs * Bug fix in postprocessing yt data and other small changes (#104) * fixed bug in post-processing of field diagnostic yt data * fixed issue causing post processing of field diagnostics unit test to fail * added helper function to rebuild MultiFabs and iMultiFabs during load balancing and included rebuilding of EB multifabs * added redistribute call for the particle boundary buffer during load balancing * consistently use DistribtionMap rather than dmap in ElectrostaticSolver.cpp * applied suggested changes from code review by Phil Miller * removed default argument for redistribute in RemakeMultiFab * added load balance intervals as an optional input parameter to diode_setup.py * removed RemakeMultiFab() as a member of WarpX * Only remake EB multifabs if they are used Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> * changed plot_contours grid setting parameter from b to visible in accordance with matplotlib 3.5 changes * Revert change to WarpXRegrid.cpp * Suggested change from PS during code review Co-authored-by: Peter Scherpelz <31747262+peterscherpelz@users.noreply.github.com> Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> Co-authored-by: Peter Scherpelz <31747262+peterscherpelz@users.noreply.github.com> Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com> Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Luca Fedeli <luca.fedeli@cea.fr> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> Co-authored-by: Phil Miller <phil@intensecomputing.com> Co-authored-by: Kevin Z. Zhu <86268612+KZhu-ME@users.noreply.github.com> Co-authored-by: Lorenzo Giacomel <47607756+lgiacome@users.noreply.github.com> Co-authored-by: Neïl Zaim <49716072+NeilZaim@users.noreply.github.com> Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov> Co-authored-by: Jean Luca Bez <jeanlucabez@gmail.com> Co-authored-by: kzhu-ME <kevin.zhu@modernelectron.com> Co-authored-by: Revathi Jambunathan <41089244+RevathiJambunathan@users.noreply.github.com> Co-authored-by: Phil Miller <unmobile+gh@gmail.com> Co-authored-by: Peter Scherpelz <31747262+peterscherpelz@users.noreply.github.com>
dpgrote
pushed a commit
to dpgrote/WarpX
that referenced
this pull request
Nov 29, 2021
* Fix conflict with upstream * Apply suggestions from code review * Remove space in the end of lines * Include suggestions from PR review * Generalize ROMIO Hints in Batch Scripts * Fix Comment * Fix Comment * Remove duplication * Formatting Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update documentation with ROMIO hints for Summit that can provide performance improvements.