Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functor material properties #18395

Merged
merged 83 commits into from Oct 12, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
a9902f7
Create functor material property
lindsayad Jul 1, 2021
23b8fd7
Remove block insertion check
lindsayad Jul 20, 2021
d1e8c09
Set _subproblem parameter for executioner
lindsayad Jul 20, 2021
797df91
Add GenericConstantFunctorMaterial
lindsayad Jul 20, 2021
26fc5a9
Move to functor material property for mu in (P)INSFV
lindsayad Jul 20, 2021
a049ee8
Allow default Real values in inputs for functor properties
lindsayad Jul 21, 2021
f994f88
Make rho a material property in (P)INSFV
lindsayad Jul 21, 2021
8ca8366
Create and define the FunctorInterface
lindsayad Jul 21, 2021
54ce2f8
INSFVMomentumGravity -> NSFVMomentumGravity in (some) inputs
lindsayad Jul 21, 2021
a688488
Add wcnsfv implementation of natural convection benchmark
lindsayad Jul 21, 2021
e1570b4
Create RhoFromPTFunctorMaterial
lindsayad Jul 21, 2021
2c59dc6
variable() API in FVBoundaryCondition
lindsayad Jul 22, 2021
d0ba4cf
getFunctor method in Coupleable
lindsayad Jul 22, 2021
bb61c49
Use const reference in GenericConstantMaterial
lindsayad Jul 21, 2021
6e089eb
Adapt FV framework and test objects to functor properties
lindsayad Jul 21, 2021
e6e49f8
Clearer variable naming in FV::interpolate limiting overload
lindsayad Jul 22, 2021
8ae451c
Query cache in getExtrapolatedBoundaryFaceValue
lindsayad Jul 22, 2021
49d10e1
Add Limiter based interpCoeffs overload
lindsayad Jul 22, 2021
d445330
Changes/additions to Limiter API
lindsayad Jul 22, 2021
df24c14
Adapt (P)INSFV to functor props
lindsayad Jul 21, 2021
bf6034b
Create FunctorMatPropElementalAux
lindsayad Jul 23, 2021
1bf65cf
Create PINSFV and PWCNSFV analogs of PCNSFVStrongBC
lindsayad Jul 24, 2021
b5a0439
Make sure to query interpCoeffs for boundary faces
lindsayad Jul 24, 2021
72165e1
Add no-diffusion PINSFV 1D MMS tests
lindsayad Jul 24, 2021
2e571e2
Working wcnsfv natural convection example
lindsayad Jul 24, 2021
a1faa73
Restore aggregate interpolation in FVMatAdvection
lindsayad Jul 26, 2021
a896d93
Allow functor property creation from getFunctorMaterialProperty
lindsayad Aug 4, 2021
84562bd
Restore aggregate interpolations in NS objects
lindsayad Aug 4, 2021
bcdcd18
Add two qp-based functor overloads
lindsayad Aug 5, 2021
7c52535
Adapt (P)INSFV to functor props, part 2
lindsayad Aug 4, 2021
79ae545
Make functor properties thread-safe
lindsayad Aug 5, 2021
d9e7765
Add pwcnsfv mms test
lindsayad Jul 24, 2021
6550f20
Remove PWCNSFVStrongBC
lindsayad Aug 6, 2021
e095777
Documentation
lindsayad Aug 6, 2021
16a9bd5
Apply suggestions from code review
lindsayad Aug 10, 2021
59db6dd
Create FunctorMaterial so we can do multiple definition checking
lindsayad Aug 20, 2021
257d146
Remove GenericFunctor
lindsayad Aug 21, 2021
6bf472d
Restrict periodically failing contact threads test
lindsayad Aug 23, 2021
fb693f9
Make all functor material property lambdas block restricted
lindsayad Aug 23, 2021
3ba76fc
Add caching to functor properties
lindsayad Aug 5, 2021
c8edc01
No reason to link libmesh libs into gtest
lindsayad Aug 25, 2021
9040131
Address Robert review
lindsayad Aug 25, 2021
6a84446
Address Guillaume navier-stokes portion review
lindsayad Aug 25, 2021
a253ebf
Move caching down into FunctorInterface
lindsayad Aug 25, 2021
aaa62bb
Add time argument to functors
lindsayad Aug 26, 2021
6ace575
Add quadrature rule to QpArg
lindsayad Aug 26, 2021
c0df195
Demonstrate use of Functions as functors
lindsayad Aug 26, 2021
58eb930
Apply suggestions from code review
lindsayad Aug 30, 2021
c4c2550
Elucidate meaning of ElemFromFaceArg
lindsayad Aug 31, 2021
9ce8e6c
Overhaul FaceArg functor overload
lindsayad Aug 31, 2021
0691fe7
Update FunctorInterface doxygen
lindsayad Sep 1, 2021
83d4b4e
Add ElemSideQpArg overload
lindsayad Sep 1, 2021
40a83c6
Resize qp cache all at once with qrule->n_points()
lindsayad Sep 2, 2021
a5b0280
Allow creation of deriatives of FunctorMaterialProperty
lindsayad Sep 2, 2021
226eb28
Test functor material property derivative creation
lindsayad Sep 3, 2021
ed7d970
Complete coverage of FunctorInterface
lindsayad Sep 7, 2021
ec79f69
shorten caching tests
lindsayad Sep 10, 2021
f9d9e90
Function FunctorInterface overload testing
lindsayad Sep 12, 2021
d54ef7a
git sed FunctorInterface -> Moose::Functor
lindsayad Sep 16, 2021
9217f42
Create general functors container and add getFunctor
lindsayad Sep 16, 2021
7bbd2a0
Add back FunctorInterface in traditional MOOSE interface form
lindsayad Sep 16, 2021
543e805
getFunctorMaterialProperty -> getFunctor
lindsayad Sep 16, 2021
5095d90
Update NS to getFunctor
lindsayad Sep 17, 2021
422fad9
Make SubProblem::getFunctor error message cleaner
lindsayad Sep 22, 2021
0050797
Create FunctionTempl
lindsayad Sep 22, 2021
dd6fd5f
Add functor version of 'u' to FVElementalKernel
lindsayad Sep 21, 2021
a762e32
Separate compressible and incompressible implementations of friction/…
lindsayad Sep 27, 2021
4df5341
Create GenericConstantVectorFunctorMaterial
lindsayad Sep 23, 2021
060e2af
Cover PINSFVMomentumFriction and GenericConstantVectorFunctorMaterial
lindsayad Oct 1, 2021
0bc54be
more documentation
lindsayad Oct 4, 2021
66557dd
Get rid of NonMathFVUtils
lindsayad Oct 4, 2021
2778094
Remove default execute on for GenericFunctionFunctorMaterial
lindsayad Oct 4, 2021
433a4bc
Split limiter change: framework/unit/core NS
lindsayad Oct 4, 2021
b9a921d
Remove FunctorMatPropElementalAux
lindsayad Oct 4, 2021
eef4a49
Apply suggestions from code review
lindsayad Oct 4, 2021
6fe4847
Test declaration/retrieval of conflicting types
lindsayad Oct 4, 2021
d4d9c0f
Test new interpolate method with vector arguments
lindsayad Oct 4, 2021
6bfd4e6
Unit test all supported functor name type retrieval
lindsayad Oct 4, 2021
c7f3cab
Move PINSFVFunctorBC to test objects
lindsayad Oct 4, 2021
9584906
Test advected mat prop in FE contexts in InternalVolumetricFlowRatePP
lindsayad Oct 4, 2021
2566c29
Delete momentum_name code-path from incompressible friction object
lindsayad Oct 4, 2021
e5c5bfe
Further stratification of (P)INSFV and (P)CNSFV
lindsayad Oct 4, 2021
0653f11
Test swapping out of functor material property types
lindsayad Oct 5, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions framework/build.mk
Expand Up @@ -74,6 +74,7 @@ endif
# machine in question (from config.guess, i.e. @host@ in
# contrib/utils/Make.common.in) and the $(METHOD).
obj-suffix := $(libmesh_HOST).$(METHOD).lo
no-method-obj-suffix := $(libmesh_HOST).lo

# The libtool script used by libmesh is in different places depending on
# whether you are using "installed" or "uninstalled" libmesh.
Expand Down Expand Up @@ -119,6 +120,11 @@ pcre%.$(obj-suffix) : pcre%.cc
@$(libmesh_LIBTOOL) --tag=CXX $(LIBTOOLFLAGS) --mode=compile --quiet \
$(libmesh_CXX) $(libmesh_CPPFLAGS) $(CXXFLAGS) $(libmesh_CXXFLAGS) $(ADDITIONAL_CPPFLAGS) $(app_INCLUDES) $(libmesh_INCLUDE) -w -DHAVE_CONFIG_H -MMD -MP -MF $@.d -MT $@ -c $< -o $@

gtest%.$(no-method-obj-suffix) : gtest%.cc
@echo "Compiling C++ "$<"..."
@$(libmesh_LIBTOOL) --tag=CXX $(LIBTOOLFLAGS) --mode=compile --quiet \
$(libmesh_CXX) $(ADDITIONAL_CPPFLAGS) $(CXXFLAGS) -w -MMD -MP -MF $@.d -MT $@ -c $< -o $@

%.$(obj-suffix) : %.cc
@echo "Compiling C++ (in "$(METHOD)" mode) "$<"..."
@$(libmesh_LIBTOOL) --tag=CXX $(LIBTOOLFLAGS) --mode=compile --quiet \
Expand Down
@@ -0,0 +1,17 @@
# GenericConstantFunctorMaterial

!syntax description /Materials/GenericConstantFunctorMaterial

## Overview

This object is very similar to [GenericConstantMaterial.md] and takes the exact
same input file syntax. The difference is that this object creates
[functor material properties](Materials/index.md#functor-props), e.g. properties
that get evaluated on-the-fly, as opposed to traditional "static" material
properties, e.g. material properties that are pre-evaluated.

!syntax parameters /Materials/GenericConstantFunctorMaterial

!syntax inputs /Materials/GenericConstantFunctorMaterial

!syntax children /Materials/GenericConstantFunctorMaterial
@@ -0,0 +1,22 @@
# GenericConstantVectorFunctorMaterial

!syntax description /Materials/GenericConstantVectorFunctorMaterial

The functor version of [GenericConstantVectorFunctorMaterial.md], this can be
used to quickly create simple constant anisotropic material properties, for
testing, for initial survey of a problem or simply because the material
properties do not vary much over the domain explored by the simulation.

## Example Input File Syntax

In this example, we create a `GenericConstantVectorFunctorMaterial` for two
anisotropic friction factors in a porous media flow simulation. Note the syntax
for declaring two material properties and their values in the same material.

!listing modules/navier_stokes/test/tests/finite_volume/pins/channel-flow/2d-rc-friction.i block=Materials/darcy

!syntax parameters /Materials/GenericConstantVectorFunctorMaterial

!syntax inputs /Materials/GenericConstantVectorFunctorMaterial

!syntax children /Materials/GenericConstantVectorFunctorMaterial
Expand Up @@ -8,10 +8,11 @@ domain explored by the simulation.

## Example Input File Syntax

In this example, we create a `GenericConstantVectorMaterial` for two anisotropic friction factors in a porous media flow simulation.
Note the syntax for declaring two material properties and their values in the same material.
In this example, we create a `GenericConstantVectorMaterial` to generate an
anisotropic vector diffusivity and then compute the integral of the diffusive
flux through a specified boundary on the mesh.

!listing modules/navier_stokes/test/tests/finite_volume/pins/channel-flow/2d-rc-friction.i block=Materials/darcy
!listing test/tests/postprocessors/side_diffusive_flux_integral/side_diffusive_flux_integral.i block=Materials/mat_props_vector

!syntax parameters /Materials/GenericConstantVectorMaterial

Expand Down
@@ -0,0 +1,31 @@
# GenericFunctionFunctorMaterial

!syntax description /Materials/GenericFunctionFunctorMaterial

## Overview

This class template is the functor material property version of
[GenericFunctionMaterial.md]. It evaluates the function at the requested location,
which can be the element centroid, an element face centroid, a quadrature point,
or any defined overload of the functor argument.

By default this class caches function evaluations
and clears the cache at the beginning of every timestep. Cache clearing behavior can be
controlled by setting the `execute_on` parameter.

## Example Input File Syntax

In this example, `ADGenericFunctionMaterial` is used to define a linearly varying in space
diffusion coefficient for this finite volume diffusion calculation.
We add the prefix `AD` as this simulation is making use of automatic differentiation to compute the Jacobian exactly.
The diffusion coefficient is retrieved as a `Moose::Functor<ADReal>`, the base class
of `FunctorMaterialProperty<ADReal>`, by the diffusion kernel. The diffusion kernel can
then obtain the diffusion coefficient directly on the faces when evaluating the face flux.

!listing test/tests/materials/boundary_material/fv_material_quadrature.i block=Materials/k1

!syntax parameters /Materials/GenericFunctionFunctorMaterial

!syntax inputs /Materials/GenericFunctionFunctorMaterial

!syntax children /Materials/GenericFunctionFunctorMaterial
Expand Up @@ -8,12 +8,11 @@ from the defined function over the domain explored by the simulation.

## Example Input File Syntax

In this example, `ADGenericFunctionMaterial` is used to define a linearly varying in space
diffusion coefficient for this finite volume diffusion calculation.
We add the prefix `AD` as this simulation is making use of automatic differentiation to compute the Jacobian exactly.
The diffusion coefficient is retrieved as an `ADMaterialProperty` by the diffusion kernel.
In this example, `GenericFunctionMaterial` is used to define a diffusion
coefficient that is inversely proportional to time. The diffusion coefficient is
retrieved as a `MaterialProperty` by the diffusion kernel.

!listing test/tests/materials/boundary_material/fv_material_quadrature.i block=Materials/k1
!listing test/tests/functions/generic_function_material/generic_function_material_test.i block=Materials/gfm

!syntax parameters /Materials/GenericFunctionMaterial

Expand Down
16 changes: 16 additions & 0 deletions framework/doc/content/source/materials/VarFunctorMaterial.md
@@ -0,0 +1,16 @@
# VarFunctorMaterial

!syntax description /Materials/VarFunctorMaterial

## Overview

Creates a functor material property with name corresponding to the value
provided for the parameter `mat_prop_name`. The functor material property
evaluation will be equivalent to the evaluation of the coupled variable `var` at
the provided geometric argument.

!syntax parameters /Materials/VarFunctorMaterial

!syntax inputs /Materials/VarFunctorMaterial

!syntax children /Materials/VarFunctorMaterial
29 changes: 29 additions & 0 deletions framework/doc/content/source/variables/MooseVariableBase.md
Expand Up @@ -168,6 +168,35 @@ gradient. Some of these methods are exemplified below:
+multi-component+ `VectorMooseVariable`) and returns the curl of the finite element solution
at the quadrature points (`VectorVariableCurl`)


### Variable functor evaluation id=functor-vars

Derived field classes of `MooseVariableBase`, e.g. derivatives of the class
template `MooseVariableField<T>` inherit from the
`Moose::Functor`. Quadrature-based overloads of the `evaluate` method are
implemented in `MooseVariableField<T>`. The `ElemQpArg` and `ElemSideQpArg` `evaluate` overloads do
true on-the-fly computation of the solution based on the information contained
within the argument, e.g. they perform calls to libMesh `FE::reinit` methods
after attaching the quadrature rule provided withing the calling argument. The
`ElementType` overload, however, simply queries methods like `adSln()`,
`slnOld()`, `slnOlder()`, `adSlnNeighbor()`, and `slnOldNeighbor()`. The success
of this latter overload depends on the fact that the variable has already been
reinit'd on the requested element or neighbor type. If a user is unsure whether
this precondition will be met, then they should call the likely slower but more
flexible `ElemQpArg` overload. For an overview of the different spatial
overloads available for functors, please see [Materials/index.md#spatial-overloads].

Finite-volume-centric `evaluate` overloads are individually implemented in
`MooseVariableFE<T>` and `MooseVariableFV<T>` class templates. The finite
element "implementations" currently just error out at run-time if called, but
these could be non-trivially implemented if on-the-fly evaluation of FE
variables coupled into FV physics becomes important. `MooseVariableFV<T>`
implementations of the finite-volume-centric `evaluate` overloads leverage
pre-existing methods like `getExtrapolatedBoundaryFaceValue`,
`getInternalFaceValue`, and `getDirichletBoundaryFaceValue` when called with
face-like arguments, and `getElemValue` and `getNeighborValue` when called with
element-like arguments.

GiudGiud marked this conversation as resolved.
Show resolved Hide resolved
!syntax parameters /Variables/MooseVariableBase

!syntax inputs /Variables/MooseVariableBase
Expand Down
156 changes: 156 additions & 0 deletions framework/doc/content/syntax/Materials/index.md
Expand Up @@ -205,6 +205,162 @@ e.g. the producer material will execute before the consumer material. If a
cyclic dependency is detected between two materials, then MOOSE will produce an
error.

## Functor Material Properties id=functor-props

Functor material properties are properties that are evaluated
on-the-fly. E.g. they can be viewed as functions of the current location in
space (and time). Functor material properties provide several overloads of the
`operator()` method for different "geometric quantities". One example of a
"geometric quantity" is a `const Elem *`, e.g. for an `FVElementalKernel`, the
value of a functor material property in a cell-averaged sense can be obtained by
the syntax

- `_foo(_current_elem)`

where here `_foo` is a functor material property data member of the kernel. The
functor material property system introduces APIs very similar to the traditional
material property system for declaring and getting properties. To declare a
functor property:

- `declareFunctorProperty<TYPE>`

where `TYPE` can be anything such as `Real, ADReal, RealVectorValue, ADRealVectorValue`
etc. To get a functor material property:

- `getFunctor<TYPE>`

It's worth noting that whereas the traditional regular material property system
has different methods to declare/get non-AD and AD properties, the new functor
system has single APIs for both non-AD and AD property types.
lindsayad marked this conversation as resolved.
Show resolved Hide resolved

Currently, functor material property evaluations are defined using the API:

```c++
template <typename T>
template <typename PolymorphicLambda>
void FunctorMaterialProperty<T>::
setFunctor(const MooseMesh & mesh,
const std::set<SubdomainID> & block_ids,
PolymorphicLambda my_lammy);
```

where the first two arguments are used to setup block restriction and the last argument is a lambda
defining the property evaluation. The lambda must be callable with two arguments, the first
corresponding to space, and the second corresponding to time, and must return the type `T` of the
`FunctorMaterialProperty`. An example of setting a constant functor material property that returns
an `ADReal` looks like:

```c++
_constant_unity_prop.setFunctor(
_mesh, blockIDs(), [](const auto &, const auto &) -> ADReal { return 1.; });
```

An example of a functor material property that depends on a nonlinear variable would look like

```c++
_u_prop.setFunctor(_mesh, blockIDs(), [this](const auto & r, const auto & t) -> ADReal {
return _u_var(r, t);
});
```

In the above example, we simply forward the calling arguments along to the variable. Variable
functor implementation is described in [MooseVariableBase.md#functor-vars]. A test functor material
class to setup a dummy Euler problem is shown in

!listing test/src/materials/ADCoupledVelocityMaterial.C

In the following subsections, we describe the various spatial arguments that functor (material
properties) can be evaluated at. Almost no functor material developers should have to concern
themselves with these details as most material property functions should just appear as functions of
space and time, e.g. the same lambda defining the property evaluation should apply across all
spatial and temporal arguments. However, in the case that a functor material developer wishes to
create specific implementations for specific arguments (as illustrated in `IMakeMyOwnFunctorProps`
test class) or simply wishes to know more about the system, we give the details below.

Any call to a functor (material property) looks like the following
`_foo(const SpatialArg & r, const TemporalArg & t)`. Below are the possible type overloads of
`SpatialArg`.

### FaceArg id=spatial-overloads

A typedef defining a "face" evaluation calling argument. This is composed of

- a face information object which defines our location in space
- a limiter which defines how the functor evaluated on either side of the face should be
interpolated to the face
- a boolean which states whether the face information element is upwind of the face
- a pair of subdomain IDs. These do not always correspond to the face info element subdomain
ID and face info neighbor subdomain ID. For instance if a flux kernel is operating at a
subdomain boundary on which the kernel is defined on one side but not the other, the
passed-in subdomain IDs will both correspond to the subdomain ID that the flux kernel is
defined on

### ElemFromFaceArg

People should think of this geometric argument as corresponding to the location in space of the
provided element centroid, *not* as corresonding to the location of the provided face
information. Summary of data in this argument:

- an element, whose centroid we should think of as the evaluation point. It is possible that
the element will be a nullptr in which case, the evaluation point should be thought of as
the location of a ghosted element centroid
- a face information object. When the provided element is null or for instance when the
functoris a variable that does not exist on the provided element subdomain, this face
information object will be used to help construct a ghost value evaluation
- a subdomain ID. This is useful when the functor is a material property and the user wants
to indicate which material property definition should be used to evaluate the functor. For
instance if we are using a flux kernel that is not defined on one side of the face, the
subdomain ID will allow us to compute a ghost material property evaluation

### ElemQpArg

Argument for requesting functor evaluation at a quadrature point location in an element. Data
in the argument:

- The element containing the quadrature point
- The quadrature point index, e.g. if there are `n` quadrature points, we are requesting the
evaluation of the ith point
- The quadrature rule that can be used to initialize the functor on the given element

If functor material properties are functions of nonlinear degrees of freedom, evaluation with this
argument will likely result in calls to libMesh `FE::reinit`.

### ElemSideQpArg

Argument for requesting functor evaluation at quadrature point locations on an element side.
Data in the argument:

- The element
- The element side on which the quadrature points are located
- The quadrature point index, e.g. if there are `n` quadrature points, we are requesting the
evaluation of the ith point
- The quadrature rule that can be used to initialize the functor on the given element and side

If functor material properties are functions of nonlinear degrees of freedom, evaluation with this
argument will likely result in calls to libMesh `FE::reinit`.

### Functor caching

By default, functor material properties are always (re-)evaluated every time they are called with
`operator()`. However, the base class that `FunctorMaterialProperty` inherits from,
`Moose::Functor`, has a
`setCacheClearanceSchedule(const std::set<ExecFlagType> & clearance_schedule)` API that allows
control of evaluations. Supported values for the `clearance_schedule` are any combination of
`EXEC_ALWAYS`, `EXEC_TIMESTEP_BEGIN`, `EXEC_LINEAR`, and `EXEC_NONLINEAR`. These will cause cached
evaluations of functor (material properties) to be cleared always (in fact not surprisingly in this
case we never fill the cache), on `timestepSetup`, on `residualSetup`, and on `jacobianSetup`
respectively. If a functor is expected to depend on nonlinear degrees of freedom, then the cache
should be cleared on `EXEC_LINEAR` and `EXEC_NONLINEAR` (the default `EXEC_ALWAYS` would obviously also work) in
order to achieve a perfect Jacobian. Not surprisingly, if a functor evaluation is cached, then
memory usage will increase.

!alert note title=Caching Implementations
Functor caching is only currently implemented for `ElemQpArg` and `ElemSideQpArg` spatial
overloads. This is with the idea that calls to `FE::reinit` can be fairly expensive whereas for the
other spatial argument types, evaluation of the functor (material property) may be relatively
inexpensive compared to the memory expense incurred from caching. We may definitely implement
caching for other overloads, however, if use cases call for it.

## Advanced Topics

### Evaluation of Material Properties on Element Faces
Expand Down
4 changes: 3 additions & 1 deletion framework/include/auxkernels/AuxKernel.h
Expand Up @@ -29,6 +29,7 @@
#include "MooseVariableInterface.h"
#include "ElementIDInterface.h"
#include "UserObject.h"
#include "FunctorInterface.h"

// forward declarations
template <typename ComputeValueType>
Expand Down Expand Up @@ -74,7 +75,8 @@ class AuxKernelTempl : public MooseObject,
public Restartable,
public MeshChangedInterface,
protected VectorPostprocessorInterface,
public ElementIDInterface
public ElementIDInterface,
protected FunctorInterface
{
public:
static InputParameters validParams();
Expand Down
2 changes: 1 addition & 1 deletion framework/include/base/Moose.h
Expand Up @@ -111,7 +111,7 @@ extern const ExecFlagType EXEC_SAME_AS_MULTIAPP;
extern const ExecFlagType EXEC_PRE_MULTIAPP_SETUP;
extern const ExecFlagType EXEC_TRANSFER;
extern const ExecFlagType EXEC_PRE_KERNELS;

extern const ExecFlagType EXEC_ALWAYS;
namespace Moose
{

Expand Down