ADInterfaceKernel #15397

lindsayad · 2020-06-03T23:33:39Z

Support for both standard and vector variables.

This PR also contains a lot of const additions to Coupleable/NeighborCoupleable/MooseVariable* APIs since they should be such.

Closes #15307

moosebuild · 2020-06-04T00:37:01Z

Job Partial Modules Parallel Test (In progress) on e1789d9 : invalidated by @lindsayad

huh?

moosebuild · 2020-06-04T01:02:31Z

Job Documentation on f7aad99 wanted to post the following:

View the site here

This comment will be updated on new commits.

lindsayad · 2020-06-04T15:48:39Z

framework/src/interfacekernels/ADInterfaceKernel.C

+
+  std::size_t ad_offset;
+  if (type == Moose::ElementElement || type == Moose::NeighborElement)
+    ad_offset = jvar * _sys.getMaxVarNDofsPerElem();


@cticenhour I believe the key change was here. I think you mostly copy-pasted this from ADDGKernel maybe where this isn't a concern...but previously I believe you had _var.number(). So we were getting the jacobian wrong when computing the Jacobians with respect to the neighbor residual because we were always using the "elemental" variable. So the change was to go from _var.number() -> jvar.

Ah you're right - I originally had ivar here. I actually didn't look at ADDGKernel, but a combo of ADIntegratedBC and ADKernel and then built something up from what I saw there and my understanding of InterfaceKernel. My core misunderstanding was the concept of an ad_offset in the first place, it seems. Could you explain that a bit more?

I have to remind myself about this sometimes because I forget...

Let's just think about continuous finite elements for a second. We use a variable major numbering scheme, such that each variables indices are in a contiguous block. Let's imagine we have two variables, u and v, and we're on a QUAD4. The AD indices will be ordered like this:

u0, u1, u2, u3, v0, v1, v2, v3

_sys.getMaxVarNDofsPerElem should return for a QUAD4: 4. For a QUAD9, 9. HEX27, 27. Etc. For CFEM the offset will be simply that element number times the var number. So for u on a QUAD4: 4 * 0 = 0. For v: 4 * 1. So u indices start at index 0, v indices start at index 4.

With DFEM or interface kernels it's a little more complicated. We essentially already have an indices block that is num_vars * num_dofs_per_elem long, so in our two var, QUAD4 example: 4 * 2 = 8. So what we do is that if we are on a neighbor element, we do an additional offset by num_vars * num_dofs_per_elem. So now our derivative indices are ordered like this:

u0, u1, u2, u3, v0, v1, v2, v3, u0_neighbor, u1_neighbor, u2_neighbor, u3_neighbor, v0_neighbor, v1_neighbor, v2_neighbor, v3_neighbor

Does that make sense?

Very clear - thanks! :)

Let's just think about continuous finite elements for a second. We use a variable major numbering scheme, such that each variables indices are in a contiguous block. Let's imagine we have two variables, u and v, and we're on a QUAD4. The AD indices will be ordered like this:

u0, u1, u2, u3, v0, v1, v2, v3

_sys.getMaxVarNDofsPerElem should return for a QUAD4: 4. For a QUAD9, 9. HEX27, 27. Etc. For CFEM the offset will be simply that element number times the var number. So for u on a QUAD4: 4 * 0 = 0. For v: 4 * 1. So u indices start at index 0, v indices start at index 4.

With DFEM or interface kernels it's a little more complicated. We essentially already have an indices block that is num_vars * num_dofs_per_elem long, so in our two var, QUAD4 example: 4 * 2 = 8. So what we do is that if we are on a neighbor element, we do an additional offset by num_vars * num_dofs_per_elem. So now our derivative indices are ordered like this:

u0, u1, u2, u3, v0, v1, v2, v3, u0_neighbor, u1_neighbor, u2_neighbor, u3_neighbor, v0_neighbor, v1_neighbor, v2_neighbor, v3_neighbor

I would really appreciate it if you could add these sentences as comments into code.

moosebuild · 2020-06-04T18:04:16Z

Job Mac Test on 9ef963b : invalidated by @cticenhour

Unrelated test crash

lindsayad · 2020-06-04T20:24:09Z

@cticenhour feel free to review this although we'll need someone else to give approval since you have an important commit here

lindsayad · 2020-06-04T20:24:41Z

@rwcarlsen this has your adCoupledNeighborValue API

fdkong · 2020-06-09T02:43:40Z

framework/include/interfacekernels/ADInterfaceKernel.h

+
+  /// The ad version of JxW
+  const MooseArray<ADReal> & _ad_JxW;
+


Interesting.

fdkong · 2020-06-09T03:03:45Z

framework/src/interfacekernels/ADInterfaceKernel.C

+
+  std::size_t ad_offset;
+  if (type == Moose::ElementElement || type == Moose::NeighborElement)
+    ad_offset = jvar * _sys.getMaxVarNDofsPerElem();


Let's just think about continuous finite elements for a second. We use a variable major numbering scheme, such that each variables indices are in a contiguous block. Let's imagine we have two variables, u and v, and we're on a QUAD4. The AD indices will be ordered like this:

u0, u1, u2, u3, v0, v1, v2, v3

_sys.getMaxVarNDofsPerElem should return for a QUAD4: 4. For a QUAD9, 9. HEX27, 27. Etc. For CFEM the offset will be simply that element number times the var number. So for u on a QUAD4: 4 * 0 = 0. For v: 4 * 1. So u indices start at index 0, v indices start at index 4.

With DFEM or interface kernels it's a little more complicated. We essentially already have an indices block that is num_vars * num_dofs_per_elem long, so in our two var, QUAD4 example: 4 * 2 = 8. So what we do is that if we are on a neighbor element, we do an additional offset by num_vars * num_dofs_per_elem. So now our derivative indices are ordered like this:

u0, u1, u2, u3, v0, v1, v2, v3, u0_neighbor, u1_neighbor, u2_neighbor, u3_neighbor, v0_neighbor, v1_neighbor, v2_neighbor, v3_neighbor

I would really appreciate it if you could add these sentences as comments into code.

fdkong · 2020-06-09T03:07:40Z

framework/src/interfacekernels/ADInterfaceKernel.C

+    ad_offset = jvar * _sys.getMaxVarNDofsPerElem();
+  else
+    ad_offset = jvar * _sys.getMaxVarNDofsPerElem() +
+                (_sys.system().n_vars() * _sys.getMaxVarNDofsPerElem());


ad_offset has to match what you are doing seed? If so, it may be a good idea to create an API somewhere, so we can retrieve the same value

fdkong · 2020-06-09T03:11:36Z

Just minor things

rwcarlsen · 2020-06-09T19:30:00Z

I agree with @fdkong - that discussion about indexing for derivatives would be very helpful to have as comments in the code. Especially looking forward to FV reconstruction stuff I will probably end up working on eventually - where we will have to add in derivs. w.r.t. even more than just two elements.

lindsayad · 2020-06-11T18:30:17Z

@fdkong @rwcarlsen please see 258ecaf

rwcarlsen

The adOffset func and doc comments are fantastic - a huge improvement. It looks like you missed a couple of replacements though - in FV flux kernels/bcs.

lindsayad · 2020-06-11T20:10:35Z

Ok hopefully I got the framework holdovers in the latest commit. @joshuahansel please checkout the API I introduced in 0b8daa6 for computing the map you want

joshuahansel · 2020-06-11T22:27:28Z

@lindsayad Thanks, might take me a day or so to test the API, but it looks like it'll do the trick.

rwcarlsen · 2020-06-11T22:28:18Z

test/include/materials/ADCheckGlobalToDerivativeMap.h

+
+/**
+ * A material that couples a material property
+ */


Not a very useful doc comment.

Changed to something a bit more meaningful

rwcarlsen · 2020-06-11T22:34:24Z

framework/src/utils/ADUtils.C

+std::unordered_map<dof_id_type, Real>
+globalDofIndexToDerivative(const ADReal & ad_real,
+                           const SystemBase & sys,
+                           const Elem * elem,
+                           const ElementType elem_type /*=ElementType::Element*/)
+{


What do we anticipate this being used for in the future? Did you have anything in mind?

rwcarlsen · 2020-06-11T22:36:13Z

framework/src/utils/ADUtils.C

+    dof_map.dof_indices(elem, global_indices, var_num);
+
+    // determine the AD offset for the current var
+    const auto ad_offset = adOffset(var_num, max_dofs_per_elem, elem_type, num_vars);


What happens if elem isn't currently assembly's current elem or neighbor? Then we just get nonsense right? Maybe we should guard against that.

Or at least we need to know which dof indices were used when the dualnumbers that contributed to ad_real were seeded/initialized.

Might as well just remove elem from the argument list then and just take it from assembly

I went ahead and removed elem from the argument list, and am just using the assembly elem APIs to get the current elements

Also move ADOffset to be ADUtils Closes idaholab#15450

This is actually only really a test in debug mode

rwcarlsen

I anticipate more work being done here eventually. I expect as Josh starts using this, and as we eventually start trying to to fv gradient reconstruction, more ideas will pop out about how to further improve this situation. Wouldn't it be nice if we just stored the global dof id's straight in the dual numbers?

lindsayad · 2020-06-11T23:36:26Z

I anticipate more work being done here eventually. I expect as Josh starts using this, and as we eventually start trying to to fv gradient reconstruction, more ideas will pop out about how to further improve this situation. Wouldn't it be nice if we just stored the global dof id's straight in the dual numbers?

I definitely agree with you. We could do this right now with the sparse container...

moosebuild added the PR: Failed but allowed label Jun 4, 2020

lindsayad marked this pull request as draft June 4, 2020 04:39

lindsayad force-pushed the ad-interface-kernels-15307 branch from be221a4 to 9ef963b Compare June 4, 2020 14:54

moosebuild removed the PR: Failed but allowed label Jun 4, 2020

lindsayad commented Jun 4, 2020

View reviewed changes

moosebuild added the PR: Failed but allowed label Jun 4, 2020

lindsayad marked this pull request as ready for review June 4, 2020 20:23

lindsayad added the PR: Ready for review/merge label Jun 4, 2020

lindsayad requested a review from fdkong June 4, 2020 20:24

fdkong suggested changes Jun 9, 2020

View reviewed changes

cticenhour and others added 7 commits June 11, 2020 08:39

Add ADInterfaceKernel and ADVectorInterfaceKernel. Refs idaholab#15307

4056123

adCoupledNeighborValue API test (idaholab#15307)

261759d

Add a lot more const to Coupleable/MooseVariable APIs

a1c75f1

Create ADVectorMatchedValueBC

816d08f

adCoupledVectorNeighborValue API (idaholab#15307)

b969ae8

Test vector variable coupling in ADInterfaceKernel (idaholab#15307)

5cf6e3f

Add adOffset API

258ecaf

lindsayad force-pushed the ad-interface-kernels-15307 branch from 9ef963b to 258ecaf Compare June 11, 2020 18:28

moosebuild removed the PR: Failed but allowed label Jun 11, 2020

rwcarlsen suggested changes Jun 11, 2020

View reviewed changes

rwcarlsen previously approved these changes Jun 11, 2020

View reviewed changes

lindsayad marked this pull request as draft June 11, 2020 20:52

lindsayad dismissed rwcarlsen’s stale review via 6ee2830 June 11, 2020 21:54

lindsayad force-pushed the ad-interface-kernels-15307 branch 2 times, most recently from 65f9efc to 7727869 Compare June 11, 2020 22:24

lindsayad marked this pull request as ready for review June 11, 2020 22:24

rwcarlsen suggested changes Jun 11, 2020

View reviewed changes

lindsayad added 2 commits June 11, 2020 16:12

Create utility to generate map from global dof index to derivative

c4cc5ec

Also move ADOffset to be ADUtils Closes idaholab#15450

Test global map generation (idaholab#15450)

5f8a0e0

This is actually only really a test in debug mode

lindsayad force-pushed the ad-interface-kernels-15307 branch from 7727869 to 5f8a0e0 Compare June 11, 2020 23:15

rwcarlsen previously approved these changes Jun 11, 2020

View reviewed changes

moosebuild added the PR: Failed but allowed label Jun 12, 2020

Restore C++11 compatability (no std::make_unique)

f7aad99

lindsayad dismissed rwcarlsen’s stale review via f7aad99 June 12, 2020 07:33

moosebuild added PR: Failed but allowed and removed PR: Failed but allowed labels Jun 12, 2020

fdkong approved these changes Jun 12, 2020

View reviewed changes

fdkong merged commit 8ebd8b0 into idaholab:next Jun 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ADInterfaceKernel #15397

ADInterfaceKernel #15397

lindsayad commented Jun 3, 2020

moosebuild commented Jun 4, 2020

moosebuild commented Jun 4, 2020 •

edited

lindsayad Jun 4, 2020

cticenhour Jun 4, 2020

lindsayad Jun 4, 2020

cticenhour Jun 4, 2020

fdkong Jun 9, 2020

moosebuild commented Jun 4, 2020

lindsayad commented Jun 4, 2020

lindsayad commented Jun 4, 2020

fdkong Jun 9, 2020

fdkong Jun 9, 2020

fdkong Jun 9, 2020

fdkong commented Jun 9, 2020

rwcarlsen commented Jun 9, 2020 •

edited

lindsayad commented Jun 11, 2020

rwcarlsen left a comment

lindsayad commented Jun 11, 2020

joshuahansel commented Jun 11, 2020

rwcarlsen Jun 11, 2020

lindsayad Jun 11, 2020

rwcarlsen Jun 11, 2020

lindsayad Jun 11, 2020

rwcarlsen Jun 11, 2020

rwcarlsen Jun 11, 2020

lindsayad Jun 11, 2020

lindsayad Jun 11, 2020

rwcarlsen left a comment

lindsayad commented Jun 11, 2020 •

edited


		/// The ad version of JxW
		const MooseArray<ADReal> & _ad_JxW;

ADInterfaceKernel #15397

ADInterfaceKernel #15397

Conversation

lindsayad commented Jun 3, 2020

moosebuild commented Jun 4, 2020

moosebuild commented Jun 4, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moosebuild commented Jun 4, 2020

lindsayad commented Jun 4, 2020

lindsayad commented Jun 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fdkong commented Jun 9, 2020

rwcarlsen commented Jun 9, 2020 • edited

lindsayad commented Jun 11, 2020

rwcarlsen left a comment

Choose a reason for hiding this comment

lindsayad commented Jun 11, 2020

joshuahansel commented Jun 11, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rwcarlsen left a comment

Choose a reason for hiding this comment

lindsayad commented Jun 11, 2020 • edited

moosebuild commented Jun 4, 2020 •

edited

rwcarlsen commented Jun 9, 2020 •

edited

lindsayad commented Jun 11, 2020 •

edited