Version 3.2

CHANGELOG

@@ -7,12 +7,166 @@
 
                                http://mfem.org
 
+Version 3.2, released on June 30, 2016
+======================================
+
+Dynamic AMR with parallel load balancing, derefinement of non-conforming meshes
+-------------------------------------------------------------------------------
+- Parallel non-conforming meshes can now be load balanced at any time by calling
+  ParMesh::Rebalance(). Elements of the mesh are redistributed in such a way
+  that each processor gets approximately the same number of elements (plus minus
+  one element). Partitioning is done by splitting a sequence of space-filling
+  (Hilbert) curves defined on the refinement octrees.
+
+- Isotropically refined non-conforming meshes can now be derefined, both in
+  serial and in parallel, based on a per-element error measure and a
+  derefinement threshold. See the class ThresholdDerefiner.
+
+- Following an arbitrary mesh change (uniform/general conforming/non-conforming
+  refinement, derefinement, load balancing), the FiniteElementSpace and
+  associated GridFunctions can be updated by interpolating or redistributing the
+  previous function values based on the new state of the mesh. (Internally this
+  is implemented through a transformation matrix that is constructed in the
+  FiniteElementSpace.) The user interface is quite simple:
+
+     pmesh.Rebalance();  // or GeneralRefinement, or GeneralDerefinement
+     fespace.Update();   // calculate a transformation matrix (by default)
+     x.Update();         // apply the transformation to the GridFunction
+     z.Update();         // apply it again
+
+- New abstractions are available for error estimation and general mesh
+  operations such as refinement and derefinement. See the base classes
+  ErrorEstimator and MeshOperator and their descendants.
+
+- The above features are illustrated in the new Example 15 (see also Example 6).
+
+Tensor-based high-performance FEM operator assembly and evaluation
+------------------------------------------------------------------
+- Added support for high-performance, tensor-based efficient assembly and
+  evaluation of high-order operators.
+
+- A number of new header files have been added to the fem/, linalg/ and mesh/
+  directories. They start with the prefix "t" to indicate the (heavy) use of C++
+  templating, similar to how the prefix "p" denotes "parallel". All the code for
+  the new HPC FE assembly/evaluation algorithms is fully implemented in these
+  header files. Note that the new interface is optional and only enabled if the
+  mfem-performance.hpp header is included instead of mfem.hpp. This is an
+  initial, reference implementation.
+
+- Similarly to the serial-to-parallel (ex1.cpp-to-ex1p.cpp) transition, an
+  existing MFEM-based applications has to be transitioned to the new HPC
+  interface. This is illustrated in two new example codes which are the
+  high-performance versions of Example 1/1p. See miniapps/performance.
+
+- The new interface reduces local operator assembly/evaluation to batched small
+  dense tensor contraction operations. For high performance, the sizes of these
+  contractions should be known at compile time, so the BilinearForm object needs
+  to have detailed knowledge about the mesh, the finite element space, the
+  quadrature rule and the integrator to be assembled. This required a new
+  interface, that supports a subset of the current (general) coefficients and
+  bilinear form integrators, including variable coefficients and mass and
+  diffusion integrators. It is possible to use the old and the new HPC interface
+  side-by-side, see the HPC version of Example 1/1p in miniapps/performance.
+
+Advanced FEM on parallel non-conforming meshes
+----------------------------------------------
+- Added support for discontinuous Galerkin methods on parallel non-conforming
+  meshes, see Examples 9p and 14p.
+
+- Added support for hybridization on parallel non-conforming meshes, see
+  Example 4p.
+
+New and improved linear solvers
+-------------------------------
+- Added a wrapper for the real-valued, double precision solver in SuperLU_DIST
+  which is a sparse direct solver for distributed memory architectures. As such
+  it can only be enabled along with MFEM_USE_MPI. When MFEM is configured with
+  MFEM_USE_SUPERLU, one also needs to alter the version of METIS, since SuperLU
+  requires ParMETIS (which comes packaged with a serial version of METIS).  See
+  http://http://crd-legacy.lbl.gov/~xiaoye/SuperLU for SuperLU_DIST details.
+
+- Added a wrapper for the KLU solver in SuiteSparse see
+  http://faculty.cse.tamu.edu/davis/suitesparse.html for details of KLU.
+  If MFEM was configured with MFEM_USE_SUITESPARSE, one must now also link
+  against the klu and btf libraries in SuiteSparse, see config/defaults.mk.
+
+New and updated examples and miniapps
+-------------------------------------
+- Four new serial and parallel example codes that demonstrate:
+  * high-performance finite element operator assembly/evaluation (Example 1/1p
+    in miniapps/performance)
+  * adaptive refinement, derefinement and load balancing (in parallel) on
+    non-conforming meshes (Example 15/15p)
+
+- Examples 4p now supports hybridization on non-conforming meshes.
+
+- Examples 9p and 14p now work on non-conforming meshes.
+
+- Example 11p now has optional support for the SuperLU parallel direct solver.
+
+- Added several new options and example runs in the Volta and Tesla miniapps,
+  including support for Halbach arrays of permanent magnets.
+
+Miscellaneous
+-------------
+- Added "check" and "test" targets to the top-level makefile. The former does a
+  quick check by running Example 1/1p, while the latter does a more thorough
+  verification of the build by running all example codes and miniapps.
+
+- Added support for 2D and 3D meshes generated by Gmsh (http://gmsh.info), both
+  in ASCII and binary formats.
+
+- Added a reader for Cubit meshes in the Genesis (NetCDF) format. Currently
+  supported are linear and quadratic tet and hex meshes.
+
+- Added support for boundary bilinear form integrators when using hybridization.
+
+- Added support for Robin boundary conditions for DG in BoundaryMassIntegrator.
+
+- Moved all reference element connectivity descriptions, such as element-edge,
+  element-face, etc. to the template class Geometry::Constants<Geometry::Type>.
+
+- Added support for secure socket communications in class socketstream based on
+  the GnuTLS library, see INSTALL for more details.
+
+- Renamed config/user.mk.in to config/defaults.mk and moved all the default
+  build settings from the makefile there.
+
+- Added configurable variables AR, ARFLAGS, and RANLIB in the build system. The
+  defaults for Mac OS X will suppress the "has no symbols" warnings.
+
+- Various other simplifications, extensions, and bugfixes in the code.
+
+API changes
+-----------
+- Changes in class Mesh
+  * Two-level state functionality was removed, including: UseTwoLevelState(int),
+    SetState(int), GetState(), GetNumFineElems(int), GetRefinementType(int),
+    GetFineElem(int, int) and GetFineElemTrans(int, int).
+
+- Changes in class FiniteElementSpace
+  * BuildElementToDofTable() is now protected, and it is always called.
+  * GlobalRestrictionMatrix(FiniteElementSpace*, int) was removed, but the
+    prolongation operator can still be accessed via GetUpdateOperator() after
+    mesh refinement and a call to Update(true).
+
+- Changes in methods related to non-conforming meshes and spaces
+  * The methods LinearForm::ConformingAssemble, BilinearForm::ConformingAssemble
+    and GridFunction::ConformingProlongate/ConformingProject are now hidden
+    inside (Par)BilinearForm::FormLinearSystem and RecoverFEMSolution.
+  * The conforming prolongation/restriction matrices can still be accessed via
+    FiniteElementSpace::GetConformingProlongation()/GetConformingRestriction().
+
+- Changes in classes GridFunction and ParGridFunction
+  * Renamed Update((Par)FiniteElementSpace*, Vector&, int) to MakeRef.
+  * Renamed Update((Par)FiniteElementSpace*) to SetSpace.
+
 
 Version 3.1, released on Feb 16, 2016
 =====================================
 
 Substantially improved non-conforming adaptive mesh refinement
----------------------------------------------------------------
+--------------------------------------------------------------
 - Added support for parallel non-conforming mesh refinement, including a new
   example code with adaptive mesh refinement for the Laplace problem (Example
   6p). Most of the example codes can now work on non-conforming meshes in serial
@@ -108,7 +262,6 @@ New and updated examples
 
 - Example 3/3p now works in both 2D and 3D.
 
-
 New miniapps
 ------------
 - Electromagnetic miniapps:
@@ -460,4 +613,3 @@ Version 1.0, released on Jul 21, 2010
 - Uploaded to http://mfem.googlecode.com.
 
 - Initial release.
-

INSTALL

@@ -39,6 +39,9 @@ Example codes (serial/parallel, depending on the build):
 Build everything (library, examples and miniapps) with current configuration:
    make all -j 4
 
+Quick-check the build by running Example 1/1p (optional):
+   make check
+
 
 Building with GNU make
 ======================
@@ -53,11 +56,12 @@ The OPTIONS are of the form VARIABLE=VALUE. Detailed description of the
 configuration options is given below. Alternatively, the options can be
 specified with an input file:
 
-   cp config/user.mk.in config/user.mk
+   cp config/defaults.mk config/user.mk
    (edit config/user.mk)
    make config
 
-The default configuration file (config/user.mk) can be changed with
+Note that config/user.mk, if present, is loaded after config/defaults.mk and
+its path/name can be changed with
 
    make config USER_CONFIG=user_config_file
 
@@ -77,6 +81,19 @@ are also defined:
 Note that any of the above shortcuts accept configuration options, either at the
 command line or through a user configuration file.
 
+The build can be quick-tested by running
+
+   make check
+
+which will simply compile and run Example 1/1p. For more extensive tests that
+check the results from all the serial/parallel MFEM examples and miniapps use:
+
+   make test
+
+Note that running all the tests may take a while. Implementation details about
+the check and test targets can be found in the top-level makefile and the
+config/test.mk file.
+
 An optional installation of the library and the headers can be performed with
 
    make install [PREFIX=<dir>]
@@ -154,15 +171,6 @@ MFEM_THREAD_SAFE = YES/NO
 MFEM_USE_OPENMP = YES/NO
    Enable (basic) experimental OpenMP support. Requires MFEM_THREAD_SAFE.
 
-MFEM_USE_MESQUITE = YES/NO
-   Enable MFEM functionality based on the Mesquite library. When enabled, this
-   option uses the MESQUITE_* library options, see below.
-
-MFEM_USE_SUITESPARSE = YES/NO
-   Enable MFEM functionality based on the SuiteSparse library. Currently, this
-   option adds the class UMFPackSolver (a sparse direct solver). When enabled,
-   this option uses the SUITESPARSE_* library options, see below.
-
 MFEM_USE_MEMALLOC = YES/NO
    Internal MFEM option: enable batch allocation for some small objects.
    Recommended value is YES.
@@ -176,6 +184,41 @@ MFEM_TIMER_TYPE = 0/1/2/3/NO
       3  - use QueryPerformanceCounter from <windows.h>
       NO - use option 3 if the compiler macro _WIN32 is defined, 0 otherwise
 
+MFEM_USE_MESQUITE = YES/NO
+   Enable MFEM functionality based on the Mesquite library. When enabled, this
+   option uses the MESQUITE_* library options, see below.
+
+MFEM_USE_SUITESPARSE = YES/NO
+   Enable MFEM functionality based on the SuiteSparse library. Currently, this
+   option adds the classes UMFPackSolver and KLUSolver (both sparse serial
+   direct solvers). When enabled, this option uses the SUITESPARSE_* library
+   options, see below.
+
+MFEM_USE_SUPERLU = YES/NO
+   Enable MFEM functionality based on the SuperLU_DIST library. Currently, this
+   option adds the classes SuperLUSolver (a parallel sparse direct solver) and
+   SuperLURowLocMatrix a distributed CSR matrix class needed by SuperLU. When
+   enabled, this option uses the SUPERLU_* library options, see below.
+
+MFEM_USE_GNUTLS = YES/NO
+   Enable secure socket support in class socketstream, using the auxiliary
+   GnuTLS_* classes, based on the GnuTLS library. This option may be useful in
+   multi-user environment to prevent users from sending/receiving visualization
+   data to/from other users. When this option is enabled, the default behavior
+   in class socketstream is to use secure sockets, e.g. when connecting to a
+   GLVis visualization server. In order for this to work, one needs to generate
+   GLVis server/client key pairs (in ~/.config/glvis), similar to ssh keys --
+   the script 'glvis-keygen.sh' in the main GLVis directory can be used to do
+   that:
+      bash glvis-keygen.sh ["Your Name"] ["Your Email"]
+   When MFEM_USE_GNUTLS is enabled, the additional build options, GNUTLS_*, are
+   also used, see below.
+
+MFEM_USE_NETCDF = YES/NO
+   NetCDF is the library that is used by the SNL Cubit mesh generator to create
+   Genesis mesh files. This option enables a reader for these files, which
+   requires that NetCDF be installed, see the NETCDF_* build options below.
+
 MFEM_BUILD_TAG = (any value)
    An optional tag to characterize the build. Exported to config/config.mk.
    Can be used to identify the MFEM build from other makefiles.
@@ -214,6 +257,10 @@ The specific libraries and their options are:
 - OpenMP (optional), usually part of compiler, used when MFEM_USE_OPENMP = YES.
   Options: OPENMP_OPT, OPENMP_LIB.
 
+- High-resolution POSIX clocks: when using MFEM_TIMER_TYPE = 2, it may be
+  necessary to link with a system library (e.g. librt.so).
+  Option: POSIX_CLOCKS_LIB (default = -lrt).
+
 - Mesquite (optional), used when MFEM_USE_MESQUITE = YES.
   URL: http://trilinos.org/oldsite/packages/mesquite
   Options: MESQUITE_OPT, MESQUITE_LIB.
@@ -222,9 +269,23 @@ The specific libraries and their options are:
   URL: http://faculty.cse.tamu.edu/davis/suitesparse.html
   Options: SUITESPARSE_OPT, SUITESPARSE_LIB.
 
-- High-resolution POSIX clocks: when using MFEM_TIMER_TYPE = 2, it may be
-  necessary to link with a system library (e.g. librt.so).
-  Option: POSIX_CLOCKS_LIB (default = -lrt).
+- SuperLU_DIST (optional), used when MFEM_USE_SUPERLU = YES. Note that
+  SuperLU_DIST requires ParMETIS which, in turn, requires METIS (see above URL).
+  URL: http://crd-legacy.lbl.gov/~xiaoye/SuperLU
+  Options: SUPERLU_OPT, SUPERLU_LIB.
+
+- GnuTLS (optional), used when MFEM_USE_GNUTLS = YES. On most Linux systems,
+  GnuTLS is available as a development package, e.g. gnutls-devel. On Mac OS X,
+  one can get the library through the Homebrew package manager (http://brew.sh).
+  URL: http://gnutls.org
+  Options: GNUTLS_OPT, GNUTLS_LIB.
+
+- NetCDF (optional), used when MFEM_USE_NETCDF = YES, required for reading Cubit
+  mesh files. Also requires installation of HDF5 and ZLIB, as explained at the
+  NetCDF web site. Note that we use the plain vanilla "C" version of NetCDF, you
+  don't need the C++ or parallel versions.
+  URL: www.unidata.ucar.edu/software/netcdf
+  Options: NETCDF_OPT, NETCDF_LIB.
 
 
 Building without GNU make

config/config.hpp.in

@@ -32,18 +32,9 @@
 // Enable experimental OpenMP support. Requires MFEM_THREAD_SAFE.
 // #define MFEM_USE_OPENMP
 
-// Enable MFEM functionality based on the Mesquite library.
-// #define MFEM_USE_MESQUITE
-
-// Enable MFEM functionality based on the SuiteSparse library.
-// #define MFEM_USE_SUITESPARSE
-
 // Internal MFEM option: enable group/batch allocation for some small objects.
 // #define MFEM_USE_MEMALLOC
 
-// Enable functionality based on the Gecko library
-// #define MFEM_USE_GECKO
-
 // Which library functions to use in class StopWatch for measuring time.
 // The available options are:
 //   0 - use std::clock from <ctime>
@@ -53,6 +44,24 @@
 // If not defined, an option is selected automatically.
 // #define MFEM_TIMER_TYPE @MFEM_TIMER_TYPE@
 
+// Enable MFEM functionality based on the Mesquite library.
+// #define MFEM_USE_MESQUITE
+
+// Enable MFEM functionality based on the SuiteSparse library.
+// #define MFEM_USE_SUITESPARSE
+
+// Enable MFEM functionality based on the SuperLU library.
+// #define MFEM_USE_SUPERLU
+
+// Enable functionality based on the Gecko library
+// #define MFEM_USE_GECKO
+
+// Enable secure socket streams based on the GNUTLS library
+// #define MFEM_USE_GNUTLS
+
+// Enable functionality based on the NetCDF library (reading CUBIT files)
+// #define MFEM_USE_NETCDF
+
 // Windows specific options
 #ifdef _WIN32
 // Macro needed to get defines like M_PI from <cmath>. (Visual Studio C++ only?)

config/config.mk.in

@@ -16,11 +16,14 @@ MFEM_DEBUG           = @MFEM_DEBUG@
 MFEM_USE_LAPACK      = @MFEM_USE_LAPACK@
 MFEM_THREAD_SAFE     = @MFEM_THREAD_SAFE@
 MFEM_USE_OPENMP      = @MFEM_USE_OPENMP@
+MFEM_USE_MEMALLOC    = @MFEM_USE_MEMALLOC@
+MFEM_TIMER_TYPE      = @MFEM_TIMER_TYPE@
 MFEM_USE_MESQUITE    = @MFEM_USE_MESQUITE@
 MFEM_USE_SUITESPARSE = @MFEM_USE_SUITESPARSE@
-MFEM_USE_MEMALLOC    = @MFEM_USE_MEMALLOC@
+MFEM_USE_SUPERLU     = @MFEM_USE_SUPERLU@
 MFEM_USE_GECKO       = @MFEM_USE_GECKO@
-MFEM_TIMER_TYPE      = @MFEM_TIMER_TYPE@
+MFEM_USE_GNUTLS      = @MFEM_USE_GNUTLS@
+MFEM_USE_NETCDF      = @MFEM_USE_NETCDF@
 
 # Compiler, compile options, and link options
 MFEM_CXX       = @MFEM_CXX@