From 4d9bf4c1ec6dae49dbf1cd4b8e11288c80f807e1 Mon Sep 17 00:00:00 2001 From: Andrew Myers Date: Wed, 2 Nov 2022 14:33:02 -0700 Subject: [PATCH] Squashed commit of the following: commit 40b3d2176b17785191050482a2ead5539993fac6 Author: Weiqun Zhang Date: Wed Jul 13 13:24:15 2022 -0700 Add extra braces in initialization of GpuArray (#2876) It should not be needed since C++14. But some compilers seem to need the double braces. commit a633d2bff1db1a3335efd077a34b6a8dcfb4e793 Author: Luca Fedeli Date: Fri Jul 8 20:34:18 2022 +0200 Workaround to bypass issue observed at very large scale with Fujitsu MPI (#2874) We have observed some MPI issues at very large scale when WarpX is compiled using Fujitsu MPI (i.e., with the Fujitsu compiler). These issues seem to be related to the use of MPI Gatherv with MPI_Datatype. This PR implements a possible workaround, initially proposed by @WeiqunZhang . The idea is that, when WarpX is compiled with the Fujitsu compiler, simpler integer arrays instead of MPI_Datatype are used in the routine where the issue was observed. commit 7660c885d46779367344adf88af75e630a0bc77a Author: Weiqun Zhang Date: Fri Jul 8 08:48:14 2022 -0700 Allow zero components MultiFab and BaseFab (#2873) This is useful for particle I/O that does not have any mesh data. yt needs a header file associated with a MultiFab. commit c849dd1994388cebd78a6a1624e80bc3ab640970 Author: Weiqun Zhang Date: Fri Jul 8 08:06:37 2022 -0700 New EB optimization parameter: eb2.num_coarsen_opt (#2872) At the beginning of EB generation, we chop the entire finest domain into boxes and find out the type of the boxes. We then collect the completely covered boxes and cut boxes into two BoxArrays. This process can be costly because of the number of calls to the implicit functions. In this commit, we have introduced a new ParmParse parameter, eb2.num_coarsen_opt with a default value of zero. If for instance it is set to 3, we start the box type categorization at a resolution that is coarsened by a factor of 2^3. For the provisional cut boxes, we refine them by a factor of 2, Then we chop them into small boxes and categorize the new boxes. This process is performed recursively until we are at the original finest resolution. The users should be aware that, if eb2.num_coaren_opt is too big, this could produce in erroneous results because evaluating the implicit function on coarse boxes could miss fine structures in the EB. Thank Robert Marskar for sharing this algorithm. commit 557aae84902f63a84edc8b49831ee66af7d1a46a Author: Erik Date: Wed Jul 6 08:54:24 2022 -0700 point to new location of AMReX images, AMReX website repo (#2867) commit cbdc6580ee3d78cccdd37172e4ba077ee181f483 Author: Axel Huebl Date: Tue Jul 5 01:41:03 2022 +0200 SENSEI 4.0: Fix Build for Particles (#2869) ## Summary This part causes a compile error now in WarpX. cc @burlen @kwryankrattiger ## Additional background X-ref: Blocks WarpX 22.07 release https://github.com/ECP-WarpX/WarpX/pull/3211 Follow-up to: - #2785 - #2834 commit dc8b734b6a70583602150cfbee1b7d51f8dacdeb Author: Andrew Myers Date: Fri Jul 1 17:19:20 2022 -0700 Cache the neighbor comm tags for the CPU implementation of fillNeighbors. (#2862) * Cache the neighbor comm tags for the CPU implementation of fillNeighbors. * fix areMasksValid function commit 2b42fb56a96e752d301916ca23160098c5369386 Author: drangara <69211175+drangara@users.noreply.github.com> Date: Fri Jul 1 18:44:35 2022 -0400 Remove some hard checks in check_mvmc for 3D (#2864) Removing some hard checks in 3D coarsening logic as it appears that those are not necessarily bad states, and a soft failure to coarsen should suffice. commit 19c70685cdb0c3322712e9f442092b1140cfe7ec Author: Erik Date: Fri Jul 1 18:24:24 2022 -0400 Carry over fix for ngbxy.smallEnd typo (#2868) This a typo that got correct in other places but didn't get fixed here. commit d736ef299b724b96b34d41103dfc5318d0ecdee4 Author: Weiqun Zhang Date: Fri Jul 1 11:00:15 2022 -0700 Update CHANGES for 22.07 (#2866) commit be813d024e6b314e41c727734b8e53481898e08e Author: Weiqun Zhang Date: Fri Jul 1 10:29:13 2022 -0700 Hypre: add version check (#2865) These HYPRE_SetSp* are only available in hypre >= 22500. commit 8fb23ec17a58284af6bdafbcda3eea0d86d8ce69 Author: Jon Rood Date: Wed Jun 29 16:52:35 2022 -0600 Refactor Make.nrel to use MPT for MPI with the Intel compiler on Eagle. (#2861) commit 6f9a46c7e834046970d46d684927a078671355bc Author: PaulMullowney <60452402+PaulMullowney@users.noreply.github.com> Date: Wed Jun 29 11:09:57 2022 -0600 Adding control APIs and namespacing for core algorithm paths like SpGEMM, SpMV, and SpTrans. (#2859) Co-authored-by: Paul Mullowney commit e4c83cfddc8afb1bd091c45a6ad3040d23f019bc Author: Jon Rood Date: Wed Jun 29 11:08:42 2022 -0600 Add lib64 library location for ZFP since it may exist there instead of lib. (#2860) commit b2b9150ada12af878a07e0628be03668a9d17270 Author: Burlen Loring Date: Tue Jun 28 13:42:41 2022 -0700 update the SENSEI in situ coupling for SENSEI v4.0.0 (#2785) In this release, an install of VTK is no longer required. To compile AMReX w/ SENSEI use: ```cmake -DAMReX_SENSEI=ON -DSENSEI_DIR=//cmake ``` Note: may be `lib` or `lib64` or something else depending on your OS and is determined by CMake at configure time. See the CMake GNUInstallDirs documentation for more information. commit 2c5f475d451aede47fe2cad2bbd8681c9ca1f456 Author: Andrew Myers Date: Tue Jun 28 12:51:19 2022 -0700 Write runtime attribs to checkpoints on GPUs (#2856) commit d2cb54668b5e49fd35a60164f40ad6f36720f806 Author: Jon Rood Date: Tue Jun 28 13:27:02 2022 -0600 Fix gnu make on Crusher for mpi_gtl_hsa (#2857) Update environment variable at OLCF for mpi_gtl_hsa. commit 21fe4b3016a796b99c409760cfad7ae00a7475ba Author: Axel Huebl Date: Tue Jun 28 19:53:09 2022 +0200 CMake: FindDependency CUDAToolkit (#2849) If we install AMReX with CUDA support using a modern CMake, we need to repopulate targets such as `CUDA::curand` from `find_dependency` for downstream. Downstream users find us via `find_package` and that target link dependency showed up to be unpopulated in MFIX. commit 027f2ff77fed33a191cfc735d8adaabb42d21743 Author: Weiqun Zhang Date: Thu Jun 23 16:15:57 2022 -0700 Fix make help (#2854) This reverts the change in #2845, which fixed an issue with `make print-%`, but broke `make help`. This is now fixed in a different way. Both `make print-%` and `make help` should work now. commit 3d3ad213ca4b60421c9a80328e1316b23435958f Author: kngott Date: Thu Jun 23 13:39:59 2022 -0700 NERSC Programming Environment prototype (#2848) commit 487267625412e4f8a4fa1ab2492cb578955c4239 Author: Weiqun Zhang Date: Thu Jun 23 12:41:20 2022 -0700 GNU Make: No need to query mpif90 if Fortran is not used. (#2852) This minimize potential issues. commit fc0d6469f4ad590d576a7109d8719b018838dd86 Author: Weiqun Zhang Date: Thu Jun 23 12:23:55 2022 -0700 Remove f90doc (#2851) We no longer use it. commit 5188a6a28e64dc627c3333d13bebeb0d7250b506 Author: Weiqun Zhang Date: Thu Jun 23 11:09:15 2022 -0700 Explicitly invoke python3 (#2850) According to PEP 394, a python distributor may choose to not provide the python command. In fact, that's what recent versions of macOS do. commit 2d931f63cb4d611d0d23d694726889647f8a482d Author: Andrew Myers Date: Wed Jun 22 15:03:50 2022 -0500 Maintain the high end of the 'roundoff domain' in both float and double precision (#2839) * Maintain the high end of the 'roundoff domain' in both float and double precision * fix shadowing * fix warning * fix float conversion warning * fix logic * Update Src/Base/AMReX_Geometry.H * Update Src/Base/AMReX_Geometry.H --- Src/Base/AMReX_Geometry.H | 1 + Src/EB/AMReX_EB2.H | 1 - Src/Particle/AMReX_ParticleUtil.H | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Src/Base/AMReX_Geometry.H b/Src/Base/AMReX_Geometry.H index 890ec2e0f7e..110b7078d7f 100644 --- a/Src/Base/AMReX_Geometry.H +++ b/Src/Base/AMReX_Geometry.H @@ -248,6 +248,7 @@ public: return roundoff_lo_d; #endif } + GpuArray ProbHiArrayInParticleReal () const noexcept { #ifdef AMREX_SINGLE_PRECISION_PARTICLES return roundoff_hi_f; diff --git a/Src/EB/AMReX_EB2.H b/Src/EB/AMReX_EB2.H index def8d2de9e0..b140ffbb6f9 100644 --- a/Src/EB/AMReX_EB2.H +++ b/Src/EB/AMReX_EB2.H @@ -128,7 +128,6 @@ void Build (const Geometry& geom, bool extend_domain_face = ExtendDomainFace(), int num_coarsen_opt = NumCoarsenOpt()); - void BuildFromChkptFile (std::string const& fname, const Geometry& geom, int required_coarsening_level, diff --git a/Src/Particle/AMReX_ParticleUtil.H b/Src/Particle/AMReX_ParticleUtil.H index e0ec8944361..d1d57fc1807 100644 --- a/Src/Particle/AMReX_ParticleUtil.H +++ b/Src/Particle/AMReX_ParticleUtil.H @@ -580,7 +580,7 @@ bool enforcePeriodic (P& p, } // clamp to avoid precision issues; if (p.pos(idim) > rhi[idim]) { - p.pos(idim) = rhi[idim]; + p.pos(idim) = static_cast(rhi[idim]); } shifted = true; }