Merge pull request #44 from mwiesenberger/feltor3d

More Multistep

.gitignore

@@ -19,4 +19,10 @@ src/feltor/doc/
 *.dat
 *.eps
 *.swp
-*.txt
+*.txt
+*.pdf
+*.aux
+*.log
+*.png
+*.pvsm
+*.bak

CHANGELOG.md

@@ -1,12 +1,129 @@
 # Changelog
 All notable changes to this project will be documented in this file.
-
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
-We do not (yet) follow semantic versioning.
-Only code changes are reported here, we do not track changes in the
-doxygen documentation, READMEs or tex writeups.
 
-## [Unreleased]
+We agreed to updating the master branch on feltor-dev/feltor very seldomnly and only when significant changes merit the effort to brush everything up, document and make
+everyone update their codes as well. We mark those updates with a (pre-)release tag on
+github.
+> Creating a release on github will create an associated archived version
+  on zenodo, so **we reserve releases for when we publish an article**.
+
+We kind of make up our own version numbers right now. A new major version number is often associated with a journal publication, but other than that there is no defined mapping from the amount or kind of change to a version number.
+[Semantiv versioning](https://semver.org/) might serve as a guideline  but we are
+far away from strictly following it really.
+
+> Only changes in code are reported here, we do not track changes in the
+> doxygen documentation, READMEs or tex writeups.
+
+## [v5.2] More Multistep
+### Added
+ - M100 config file
+ - json utility functions `dg::file::get, dg::file::get_idx` in `dg/file/json_utilities.h` which adds a small abstraction layer that gives a user more control over what happens if a variable is not found
+ - json utility functions `dg::file::file2Json`, and `dg::file::string2Json` in json_utilities.h which adds a small abstraction layer that gives a user more control over what happens if an error happens during the parsing of a file
+ - "easy output" netcdf utility functions that are particularly useful for MPI output: either write data in parallel or funnel through the master thread
+ - new include files `dg/file/file.h`, `dg/file/json_utilities.h` and `dg/exblas/exblas.h`
+ - new class `dg::Gradient` for gradient
+ - new class `dg::Advection` for the upwind advection scheme
+ - new `dg::blas1::reduce` function for custom reductions
+ - new "exchangeable" `dg::x::DVec`, `dg::x::HVec`, ..., `dg::x::CartesianGrid2d`, ..., `dg::x::IHMatrix`, ... typedefs. The idea is that these resolve to either shared memory or mpi distributed memory versions depending on the MPI_VERSION macro. This helps merging shared and mpi programs into single ones.
+ - added "simple" mode to `dg::Average` computation, which is beneficial for GPU computing
+ - add `dg::integrate` that computes an indefinite integral of a function (essentially the opposite of the derivative)
+ - add` dg::ModalFilter` and tests thereof
+ - new `dg::compose` function that concatenates two or more functors to one
+ - add `dg::cooRZP2X` coordinate transformation functions to easily transform between Cylindrical and Cartesian coordinates
+ - interpolate function has an additional `dg::space` parameter to indicate nodal or modal values
+ - Grid classes now have `host_vector` and `host_grid` member typedefs
+ - new tensor functions `dg::tensor::scalar_product2d` and `dg::scalar_product3d` that can compute uE2 in one go
+ - new extended tensor functions `dg::tensor::multiply2d` and `dg::multiply3d` that can compute uE2 in one go
+ - new single step timestepper `dg::ShuOsher` including new `dg::ShuOsherTableau` and `dg::ConvertsToShuOsherTableau` classes to hold corresponding coefficients
+ - new ShuOsher tableaus SSPRK
+ - new Runge Kutta embedded tableaus tsitouras, the default timesteppers in Julia
+ - new implicit RK tableau trapezoidal, implicit midpoint and sdirk-2-1-2
+ - new class Simpsons that implements Simpsons rule for (time) integration
+ - new implicit timesteppers `dg::DIRKStep` and `dg::ImplicitRungeKutta`
+ - Redesign of multistep time steppers consistent with Runge-Kutta ones in terms of `dg::MultistepTableau` and `dg::ConvertsToMultistepTableau`
+ - a host of new explicit, implicit and semi-implicit multistep tableaus
+ - experimental "filtered" multistep time-steppers that allow modal filtering (first tests are not promising though)
+ - new experimental multigrid solvers involving Chebyshev iterations as smoother (but none are better than nested iterations so they remain experimental)
+ - new class `dg::EVE` that computes the largest Eigenvalue of a matrix (courtesy of Eduard Reiter)
+ - new class `dg::ChebyshevIterations` and `dg::ChebyshevPreconditioner` (for chebyshev iterations)
+ - new solvers `dg::LGMRES`, `dg::BICGSTABL`, and d`g::AndersonAcceleration` (courtesy of Aslak Poulsen)
+ - new `dg::FixedPointSolver` and `dg::AndersonSolver` for nonlinear problems in time
+ - a host of new functors for the evaluate and pullback functions
+ - `dg::geo::FluxSurfaceIntegral`, `dg::geo::FluxVolumeIntegral` and `dg::geo::SafetyFactorAverage` classes
+ - new implementation: `dg::geo::ds_centered_bc_along_field` and `dg::geo::dss_centered_bc_along_field` that implement boundary condition "Stegmeir" style along the magnetic field lines
+ - new Fieldaligned member functions `integrate_between_coarse_grid` and `interpolate_from_coarse_grid` that allow field-aligned interpolations
+ - `dg::geo::Periodify` class and `dg::geo::periodify` function to extend flux-functions periodically beyond grid boundaries
+ - new `dg::geo::findCriticalPoint` function that generalizes X-point and O-point identification
+ - new classes `dg::geo::SquareNorm` and `dg::geo::ScalarProduct` that work on cylindrical vector fields
+ - new set utility functors `dg::geo::SetUnion`, `dg::geo::SetIntersection`, and `dg::geo::SetNot` that help construct damping regions
+ - `dg::geo::createMagneticField` and `dg::geo::createModifiedField` with associated utility functions and classes that generalize the creation of magnetic flux functions and wall and sheath regions
+ - new polynomial expansion and associated `dg::Horner2d` functor for magnetic flux functions that can in particular approximate any experimental equilibrium
+ - new equilibrium, modifier and description fields for tokamak magnetic fields
+ - Sign reversal of magnetic field and associated flux functions is now possible
+ - new utility `dg::geo::createAlignmentTensor` and `dg::geo::createProjectionTensor` with respect to the magnetic unit vector
+### Changed
+ - namespace file changed to **dg::file** and exblas changed to **dg::exblas** (for consistency reasons, everything should go into the dg namespace, which in particular reduces the chance for name-clashes to just one, namely 'dg')
+ - Moved **variation** member function into **dg::Elliptic** (previously in ArakawaX and Poisson)
+ - **std=c++14** We use the C++-14 standard now (previously 11)
+ - vectorclass dependency changed to vectorclass/version1 (previously we used a custom upload on feltor-dev repository)
+ - default cuda compute capability bumped to sm-61 (previously sm-35)
+ - marconi config now uses jsoncpp module (previously manually installed)
+ - `dg::blas1::dot` and `dg::blas2::dot` and corresponding exblas functions now detect NaN and Inf errors
+ - `dg::blas1::dot` and `dg::blas2::dot` now both do not accumulate rest of multiplication (inconsistent before)
+ - All blas1 functions that do not read or alias their result vector now overwrite NaN and Inf
+ - all our mpi communications on GPUs now fall-back to host2host communication for cuda-unaware mpi-installations
+ - swapped input and output parameters in `dg::blas1::evaluate` first subroutine
+ - the fast_interpolation and fast_projection functions now can also double / divide the polynomial coefficient consistent with the grids
+ - change `shift_topologic()` shared RealTopology member functions to `shift()` and have an additional `negative` parameter that indicates sign swaps
+ - clarify and unify the behaviour of the interpolation functions when points lie outside the grid boundaries
+ - split and join functions have an additional real_type template parameter
+ - improved stopping criterion for `dg::bisection1d` function
+ - implicit time steppers give access to solver
+ - multistep time-stepper now initialize with Runge-Kutta timesteppers of corresponding order
+ - Multigrid nested iteration algorithm now allows accuracies for each stage separately (which can give a significant speed-up)
+ - `dg::inverse( bc)` function is now a free-standing function to invert a boundary condition
+ - `dg::Elliptic` classes now have `jump_weighting` member function
+ - `dg::CG` operator now has a `test-frequency` parameter to control the number of times the error condition is evaluated
+ - `dg::Extrapolation` class now has a `derive` member function to interpolate the derivative of the interpolating polynomial
+ - Adapt all src and diag projects to changed file and json utilities and the moved variation member
+ - Rename all input files with correct json file-ending
+ - Complete redesign of src/feltor and src/lamb_dipole
+ - Merge toefl_hpc with old toefl_mpi program
+ - bump Doxygen version to 1.8.17
+ - DS forward, backward, centered and dss functions are now free-standing, only requiring a fielaligned object, plus, and minus applications (this allows to reduce the number of times the plus and minus interpolation has to be applied)
+ - changed Fieldaligned members hp_inv to hbp
+ - changed name `dg::forward_transform` function (previously `dg::create::forward_transform`)
+ - new `dg::geo::MagneticFieldParameters` struct to unify the representation of Meta-data in the `dg::geo::TokamakMagneticField` class (simplifies construction)
+
+### Deprecated
+ - `dg::Karniadakis` time-stepper is now superceded by the `dg::ImExMultistep` class
+### Removed
+ - remove diag/feltordiag.cu
+ - remove dg::MemoryTraits and associated dimensionality and memory_category traits in favor of direct host_vector and host_grid typedefs in topology classes
+ - old txt input files
+ - `dg::geo::DeltaFunction` and `dg::geo::Alpha` for the computation of flux-surface averages no longer needed
+ - dg::blas1::transfer (previously marked deprecated)
+### Fixed
+ - Fix bug: race condition in `dg::blas1::dot` and `dg::blas2::dot` on GPUs that led to hard to reproduce and seemingly unreasonable crashes
+ - Fix bug: std namespace in diag/probes.h
+ - Fix bug: const in `exblas::cpu::get_element`
+ - Fix bug: correct  indices in `exblas::cpu::make_vcl_vec8d`
+ - Fix bug: infinite creation of MPI communicators in `exblas::mpi_reduce_communicator` . Lead to MPI crashes due to memory overflow.
+ - Fix bug: correct capture of cuda-aware mpi in configuration
+ - Fix bug: test for no-communication in mpi_communicator.h (indicated false positives)
+ - Fix bug: coefficient and initialization in `dg::Extrpolate`
+ - Fix bug: Fpsi safety-factor in case nan is encountered still works
+ - Fix bug: Fpsi safety-factor works up to the O-point
+ - Fix bug: `dg::pushForwardPerp` on functors computed wrong result (only affects `dg::geo::Hector`)
+ - Fix bug(s): several bugs in `dg::geo::Hector` which computed wrong grid (happened probably when we changed the grid design to polymorphic)
+ - Fix bug: in perpendicular grid of MPI Curvlinear grid
+ - Fix bug: missing direction initialization in mpi fieldaligned class
+ - Fix bug: host mpi code compiled with nvcc
+ - Fix bug: non-zero parallel boundary condition in mpi fieldaligned
+ - Fix bug: GPU symv on X-point grids
+
+## [v5.1] Adaptive Timesteppers
 ### Added
 - dg::Elliptic3d: a three-dimensional version of dg::Elliptic
 - Add 4 parameter symv member to dg::Elliptic class
@@ -45,7 +162,7 @@ doxygen documentation, READMEs or tex writeups.
 - default constructor of MPI\_Vector constructs empty communicator instead of MPI\_COMM\_WORLD
 - set\_communicator in MPI\_Vector takes three arguments now to avoid group
   creation
-- Configure cuda-aware mpi test a warning instead of an error
+- cuda-aware mpi no longer a requirement, fall-back to traditional mpi implemented
 - rewrite feltordiag.cu merging ncdiag and filamentdiag
 - Remove container argument from dg::geo::SafetyFactor constructor (Since it always takes Alpha)
 - More general interface for geometries/init.h functors including Nprofile and ZonalFlow (Old Psi functors are now regular functors)
@@ -66,12 +183,13 @@ doxygen documentation, READMEs or tex writeups.
 - Optimization: implement fast EllSparseBlockMat kernel for z derivative
 - Optimization: change buffer layout in dg::NearestNeighborComm and CooSparseBlockMat kernels to avoid slow scatter/gather operations in mpi matrix-vector multiplication
 - Optimization: implement faster kernels for CooSparseBlockMat symv kernel to accelerate mpi symv with low computation to communication ratio
+- separate modification of fluxfunctions into mod namespace that works on flux functions in general (previously only solovev)
 
 ### Deprecated
 - dg::blas1::transfer (replaced by the more general dg::assign and dg::construct)
-- the header geometries/geometries.h (now dg/geometries/geometries.h is
+- the header geometries/geometries.h (now **dg/geometries/geometries.h** is
   preferred for unified access and easier recognition as a dg header file)
-- the header file/nc\_utilities.h ( now dg/file/nc\_utilities.h is preferred for
+- the header file/nc\_utilities.h ( now **dg/file/nc_utilities.h** is preferred for
   unified access and easier recognition as a dg header file)
 - the Helmholtz2 class (it's much faster to use Helmholtz twice)
 

README.adoc

@@ -29,6 +29,10 @@ installation. Please read it before you proceed to the https://feltor-dev.github
 We also present how to use the FELTOR software package,
 which requires additional external libraries to be installed on the system.
 ____
+If you run into trouble during the setup, there is a chance that we already know of the problem and have an entry in our https://feltor-dev.github.io/troubleshooting/[troubleshooting] page.
+If you cannot find a solution there, please inform us of the issue.
+____
+____
 The first part assumes a Linux operating system. If you want to work
 on Windows, jump to <<sec_windows,Using Feltor on Windows>>.
 ____
@@ -44,18 +48,19 @@ git clone https://www.github.com/feltor-dev/feltor
 
 You also need to clone https://github.com/thrust/thrust[thrust] and
 https://github.com/cusplibrary/cusplibrary[cusp] distributed under the
-Apache-2.0 license. Also, we need Agner Fog's https://github.com/feltor-dev/vcl[vcl] library (GPL-3.0). So again in a folder of your choice
+Apache-2.0 license. Also, we need Agner Fog's https://github.com/vectorclass/version1[vcl] library (Apache 2.0). So again in a folder of your choice
 
 [source,sh]
 ----
 git clone https://www.github.com/thrust/thrust
 git clone https://www.github.com/cusplibrary/cusplibrary
-git clone https://www.github.com/feltor-dev/vcl
+git clone https://www.github.com/vectorclass/version1 vcl
 ----
 
 ____
 Our code only depends on external libraries that are themselves openly
 available.
+ We use version1 of the vectorclass library, as version2 requires C{plus}{plus}-17 and does not work with the intel compiler.
 ____
 
 .System requirements [[tab_requirements]]
@@ -64,15 +69,15 @@ ____
 |=======================================================================
 |    | Minimum system requirements  | Recommended system requirements
 | *CPU*     | Any         |support for AVX and FMA instruction set
-| *Compiler*| gcc-4.9 or msvc-15 or icc-15.0 (C{plus}{plus}-11 standard)| OpenMP-4 support, avx, fma instruction set flags
-| *GPU*     | - | NVidia GPU with compute-capability > 6 and nvcc-8.0
-| *MPI*     | - | mpi installation compatible with compiler (must be cuda-aware in case hybrid MPI+GPU is the target system)
+| *Compiler*| gcc-5.1 or msvc-15 or icc-17.0 (C{plus}{plus}-14 standard)| gcc-9.3, OpenMP-4 support, avx, fma instruction set flags
+| *GPU*     | - | NVidia GPU with compute-capability > 6 and nvcc-11.0
+| *MPI*     | - | mpi installation compatible with compiler (ideally cuda-aware in case hybrid MPI+GPU is the target system)
 |=======================================================================
 ____
 Our GPU backend uses the
 https://developer.nvidia.com/cuda-zone[Nvidia-CUDA] programming
 environment and in order to compile and run a program for a GPU a user
-needs at least the nvcc-7.5 compiler (available free of charge) and a NVidia
+needs at least the nvcc-8.0 compiler (available free of charge) and a NVidia
 GPU. However, we explicitly note here that due to the modular design of
 our software a user does not have to possess a GPU nor the nvcc
 compiler. The CPU version of the backend is equally valid and provides
@@ -199,7 +204,7 @@ To compile and run this code for a GPU use (assuming the external libraries are
 
 [source,sh]
 ----
-nvcc -x cu -std=c++11 -Ipath/to/feltor/inc -Ipath/to/include test.cpp -o test
+nvcc -x cu -std=c++14 -Ipath/to/feltor/inc -Ipath/to/include test.cpp -o test
 ./test
 ----
 
@@ -208,7 +213,7 @@ functions you can also use
 
 [source,sh]
 ----
-g++ -std=c++11 -fopenmp -mavx -mfma -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_OMP -Ipath/to/feltor/inc -Ipath/to/include test.cpp -o test
+g++ -std=c++14 -fopenmp -mavx -mfma -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_OMP -Ipath/to/feltor/inc -Ipath/to/include test.cpp -o test
 export OMP_NUM_THREADS=4
 ./test
 ----
@@ -251,7 +256,7 @@ Compile e.g. for a hybrid MPI {plus} OpenMP hardware platform with
 
 [source,sh]
 ----
-mpic++ -std=c++11 -mavx -mfma -fopenmp -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_OMP -Ipath/to/feltor/inc -Ipath/to/include test_mpi.cpp -o test_mpi
+mpic++ -std=c++14 -mavx -mfma -fopenmp -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_OMP -Ipath/to/feltor/inc -Ipath/to/include test_mpi.cpp -o test_mpi
 export OMP_NUM_THREADS=2
 mpirun -n 4 ./test_mpi
 ----
@@ -352,7 +357,7 @@ In Visual Studio we suggest to create a Property Sheet for FELTOR.
 The Property Sheet can then be conveniently added to any project that includes
 the FELTOR library headers `dg/algorithm.h` and/or `dg/geometries/geometries.h`
 
-* Open an existing solution in Visual Studio or create a new project with `File -> New -> Projet ...` selecting `Empty Project` in Visual C++.
+* Open an existing solution in Visual Studio or create a new project with `File -> New -> Projet ...` selecting `Empty Project` in Visual C{plus}{plus}.
 * In the Solution Explorer change to the `Property Manager` tab, then click on `Add New Project Property Sheet`, name it `FeltorPropertySheet.props` and save it
 to a convenient location.
 * Double click on `FeltorPropertySheet` (expand your solution and any of the Debug or Release tabs to find it)
@@ -436,7 +441,7 @@ compile the tex file(s) of the documentation.
 
 The
 http://feltor-dev.github.io/doc/dg/html/modules.html[documentation]
-of the dG library was generated with
+of the dg library was generated with
 http://www.doxygen.org[Doxygen] and LateX. You can generate a local
 version including informative pdf writeups on implemented numerical
 methods directly from source code. This depends on the `doxygen`,
@@ -455,8 +460,8 @@ For details on how FELTOR's internal Makefiles are configured please see the lin
 
 == 3. Authors, Acknowledgements, Contributions
 
-FELTOR has been developed by Matthias Wiesenberger and Markus Held. Please see the list of https://feltor-dev.github.io/about[contributors]
-and funding.
+FELTOR has been developed by Matthias Wiesenberger and Markus Held. Please see the https://feltor-dev.github.io/about/[Acknowledgements] section on our homepage
+for a list of contributors and funding.
 Also check out our https://feltor-dev.github.io[homepage]
 for general information, wiki pages,
 troubleshooting and guides on how to contribute.