Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SIDE_HIERARCHIC support #20249

Merged
merged 4 commits into from
Feb 9, 2022
Merged

Conversation

roystgnr
Copy link
Contributor

@roystgnr roystgnr commented Feb 8, 2022

Reason

To enable storage of flux or other per-side variables, support for Discontinuous Petrov-Galerkin or other fancy mixed methods, etc.

Design

libMesh supports a SIDE_HIERARCHIC FE class; the changes here merely make it possible for Moose users to request that variable type, and add a regression test with a couple such variables.

Impact

This adds new options, no impact on users who don't specifically request a SIDE_HIERARCHIC variable.

We may have some future behavior changes in store for such users, as we try to figure out what the best way to visualize these things is, or if we discover any bugs that the libMesh unit test coverage didn't trigger. Derek tells me @YaqiWang is ready to try these out, so best to get this PR enabled ASAP before getting bogged down in the question of what proper Exodus output would look like.

Refs #20191

@moosebuild
Copy link
Contributor

Job Precheck on 75cb660 wanted to post the following:

Your code requires style changes.

A patch was auto generated and copied here
You can directly apply the patch by running, in the top level of your repository:

curl -s https://mooseframework.inl.gov/docs/PRs/20249/style.patch | git apply -v

Alternatively, with your repository up to date and in the top level of your repository:

git clang-format db314bead5d74ab94880069dfbcd0967183f2805

libmesh Outdated Show resolved Hide resolved
@moosebuild
Copy link
Contributor

Job Precheck on 75cb660 wanted to post the following:

A change of the following file(s) triggered this check:

libmesh

The following file(s) are unchanged:

conda/mpich/conda_build_config.yaml
conda/libmesh/meta.yaml

The libmesh submodule or configuration was changed but the conda build config was not

@moosebuild
Copy link
Contributor

Job Precheck on 976f0a6 wanted to post the following:

A change of the following file(s) triggered this check:

libmesh

The following file(s) are unchanged:

conda/mpich/conda_build_config.yaml
conda/libmesh/meta.yaml

The libmesh submodule or configuration was changed but the conda build config was not

libmesh Outdated Show resolved Hide resolved
@moosebuild
Copy link
Contributor

moosebuild commented Feb 8, 2022

Job Documentation on f49b213 wanted to post the following:

View the site here

This comment will be updated on new commits.

@YaqiWang
Copy link
Contributor

YaqiWang commented Feb 8, 2022

@roystgnr few quick questions:

  1. Can we block restrict or side-set restrict the side variables?
  2. Can we couple a side variable in objects operating on sides, such as DGKernel, InterfaceKernel, InternalSideUserObject, InterfaceUserObject, BoundaryCondition, SideUserObject, etc.?
  3. How is a side built? Is it build always based on upwinding element and its local side ID? I guess this affect how DoFs on sides are interpreted.
  4. Are we supporting distributed mesh with side variables?

libmesh Outdated Show resolved Hide resolved
@moosebuild
Copy link
Contributor

Job Precheck on 171d6c9 wanted to post the following:

A change of the following file(s) triggered this check:

libmesh

The following file(s) are unchanged:

conda/mpich/conda_build_config.yaml
conda/libmesh/meta.yaml

The libmesh submodule or configuration was changed but the conda build config was not

@roystgnr
Copy link
Contributor Author

roystgnr commented Feb 8, 2022

Can we block restrict or side-set restrict the side variables?

You can restrict them to a set of subdomains. On any interface between two elements, there will be a side variable if either one element or the other or both belongs to a subdomain that has the side variable on it. However, if you're on an interface where one element belongs to a subdomain defining the side variable and the other doesn't, that variable will only be easily visible from the subdomain defining it (where you'll be able to see it in any elem dof_indices vector) and more difficult to access from the subdomain not defining it (where frankly the best way to would would be to just take the neighbor element and now you're on the right subdomain...)

Can we couple a side variable in objects operating on sides, such as DGKernel, InterfaceKernel, InternalSideUserObject, InterfaceUserObject, BoundaryCondition, SideUserObject, etc.?

In theory, yes. Internally a side variable looks like an ordinary interior variable, just with discontinuities; you'll only want to evaluate it on quadrature points on interiors-of-sides, because that's the only place it makes sense, but the same side reinit code that will give you a C_ZERO variable's values on a side quadrature rule's points will also give you a SIDE_DISCONTINUOUS variable's values there.

In practice? Let me know, I guess? I've tested libMesh projection code on this, but my first user switched to a different scheme before I finished so I've never run a proper kernel on it. It's also possible that it's working in libMesh but that we're missing something in the Moose shims (e.g. the missing pieces that required this PR to begin with).

How is a side built? Is it build always based on upwinding element and its local side ID?

It's not built. Elements with nodes on their sides store side variable DoFs there, the same way a C0 variable will store DoFs at vertices despite us not having a special Vertex object built. And just like you can access the same vertex Node from any element attached to it, you can access the same side Node from both upwind and downwind.

I guess this affect how DoFs on sides are interpreted.

It shouldn't affect any existing low-level interpretations. If you have some one-size-fits-all code that doesn't actually fit all, that could require some extension, like way back in the bad old days when too much Moose code assumed variables were all Lagrange. (Not to pick on Moose - we have the same sort of issue now in libMesh, where our viz code hits the "build a nodal interpolant" method and here it has to just cry because there's no obvious way to nodally interpolate something this discontinuous)

At a high level, of course interpretation is totally physics dependent. A side variable could be a Lagrange multiplier for a different discontinuous variable in one formulation, and a normal flux in a different formulation, and an index into a vector of data structures in a third formulation. That sort of interpretation is up to you.

Are we supporting distributed mesh with side variables?

Yes. As far as distributed mesh was concerned, "FE that has variables only on side nodes" was just a subset of "FE that has variables on an arbitrary subset of its nodes", already supported.

@roystgnr
Copy link
Contributor Author

roystgnr commented Feb 8, 2022

Can't believe I'd done a "commit -a" without thinking.

@YaqiWang
Copy link
Contributor

YaqiWang commented Feb 8, 2022

I should have been more specific on the second question: will coupledValue(var_name),coupledDofValues(var_name) and coupledNeighborValue(var_name),coupledNeighborDofValues(var_name) work for side variables?

So the dofs on an element is like normal var dofs; side var dofs on side 0; side var dofs on side 1; ...? On the boundary, we want the dofs seen on element and also dofs just for the boundary, how do we store the dofs just for the boundary?

If we want to do a trace operation for a normal variable to populate a side variable, does libMesh provide a quick way of doing that or we will have to loop through sides to do the projection on our own? If we have a C1 variable and do a trace operation, will the dofs stored on an element and its neighbor for a side variable are the same?

@roystgnr
Copy link
Contributor Author

roystgnr commented Feb 8, 2022

Your question was specific enough; I'm afraid that doesn't make the answer any more specific. I don't know of any reason why any of those methods should fail, but until we have test coverage for them there may be some reasons I don't know.

So the dofs on an element is like normal var dofs; side var dofs on side 0; side var dofs on side 1; ...?

If the side vars are the last variables added to the system, then yes; if not, then no; if the answer is important than that's some fragile user code, because it would be much safer to use the lookup APIs that take a variable index (or at least to query the DofObject on each side, if trying to optimize?), rather than to hard-code any belief about what they would return.

On the boundary, we want the dofs seen on element and also dofs just for the boundary, how do we store the dofs just for the boundary?

Right now the only way to do that is still to create boundary elements with their own subdomain (or a boundary mesh, if you don't need it tightly coupled) and add a (subdomain-restricted, in the former case) variable to that. The side variable type here assigns DoFs to element sides, not boundary sides. The way libMesh FE variables work would make a boundary-side-only variable much harder to implement.

If we want to do a trace operation for a normal variable to populate a side variable, does libMesh provide a quick way of doing that or we will have to loop through sides to do the projection on our own?

The latter. That'd be a reasonable thing to add to libMesh, though.

If we have a C1 variable and do a trace operation, will the dofs stored on an element and its neighbor for a side variable are the same?

Even if you have a discontinuous variable, any operation you do will leave the side DoFs stored on an element equal to the side dofs stored on its neighbor, because they're the same DoFs. We don't keep redundant duplicates, any more than we keep 8 copies of every vertex DoF on a hex mesh. If you want something that stores different DoFs on each side-of-a-side, go ahead and let me know since that wouldn't actually be hard to add on top of this work, but this work isn't it.

We're definitely not doing Exodus output well for these things yet (but
we do spit out a warning message to that effect!), so we need to fix
that and we'll need to update the gold here with that libMesh update,
but at least this test verifies that we can instantiate SIDE_HIERARCHIC
elements of various orders as both nonlinear solver and auxilliary
variables.
@roystgnr
Copy link
Contributor Author

roystgnr commented Feb 8, 2022

If the side vars are the last variables added to the system, then yes; if not, then no

My apologies; I spoke too hastily about this. If there's one side var and it's the last variable in the system then its DoFs will be last, and they will be in increasing side order; but if there's multiple side vars, they won't be interleaved "side var A side 0, side var B side 0, side var A side 1, side var B side 1, ...", they'll be blocked "side var A side 0, side var A side 1, ... side var B side 0, side var B side 1, ..."

@moosebuild
Copy link
Contributor

Job Coverage on f49b213 wanted to post the following:

Framework coverage

db314b #20249 f49b21
Total Total +/- New
Rate 82.31% 82.31% +0.00% 100.00%
Hits 73483 73486 +3 3
Misses 15795 15794 -1 0

Diff coverage report

Full coverage report

Modules coverage

Coverage did not change

Full coverage reports

Reports

This comment will be updated on new commits.

Copy link
Member

@lindsayad lindsayad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants