Exaconstit dev 4.0rc1 #1
Conversation
Cleanup [okina-cleanup]
Describe limitation of this operator when applied to H1 finite element spaces.
Update the LOR-to-HO interface
[okina] Adjustments to Okina's Atomics [artv3/okina/atomic-fix]
Some edits in the examples.
Re-introduce DeviceVector3 class Remove the mutable from MFEM_FORALL Simplified the dtensor class
Okina fallback kernels [okina-fallback-kernels]
…cts to be allocated.
Okina mm object [okina-mm-object]
Small updates in ex1, ex1p, ex6, ex6p.
parallel FE space.
Lambda const propagation [lambda-capture-problem]
Updates in the examples, class BilinearForm and related [okina-bilinearform-updates]
Okina GPU [okina]
Release candidate #1 for version 4.0
|
Compared to the previous release there have been several changes, not related to GPU/device support, that could introduce round-off level differences. Can you check the results with the Can you also provide more details about the problem you are solving? For example there have been some substantial changes in mesh refinement... FYI, we just tagged v4.0rc2: https://github.com/mfem/mfem/releases/tag/v4.0-rc2 |
|
I'll take a look at the The PDE I'm solving is simply Furthermore, I'm applying symmetry boundary conditions which means I'm only enforcing a zero boundary condition normal to the interior face, and then an applied boundary condition normal to the top surface. All of the other DOFs are free to move. Now as far as mesh generation/refinement goes for the specific problem I'm looking at our mesh is generated using the following command: mesh = new Mesh(toml_opt.nx, toml_opt.nx, toml_opt.nx, Element::HEXAHEDRON, 0, toml_opt.mx, toml_opt.mx, toml_opt.mx);where the toml_opt.nx is just our inputted desired number of elements along a given axis. Let me know if any of the above could be further clarified on for the description of our problem, and I'll do my best to either further expand or clarify on that point. edit: I forgot to mention that it's also just a monotonic tension test out to 1% strain. |
|
@tzanio It looks like the changes were introduced in the pre-okina tag. I'll probably need to take a look through the commit history to see if anything pops out to me that could be causing the slight differences. You did mention that there were some changes related to the mesh refinement which I could see playing a role in this if some nodes have slightly different starting positions from the old mesh we're using. If that's the case I could see this being something that could perturb the solution just enough to cause it to start to diverge from the older code base's results. edit: From looking at the mesh files it's hard to say whether or not there's any differences in the nodes, since the nodes that belong on each processor seem to have changed. However, I can see this contributing to some differences, since we are no longer performing reductions in the same order as previously. |
|
If I understand correctly, the differences between dd899e1 - (tag: pre-okina) Merge pull request mfem#843 from mfem/signed-char-fix (2 weeks ago) My guess is that the differences are due to mfem#697 or mfem#744. (You can check using the hashes above) Either way, if pre-okina and the 4.0-rc tags show no difference between them, the GPU code should be safe to merge. |
|
@tzanio It looks like it was due to mfem#697. I had to turn off the default setting to use the space-filling ordering in order to get back to the results I had previously. edit: Also, I forgot to mention that we're working on a set of questions related to the Okina branch that we have that we'll be sending out to you and the others on the MFEM team in the near future. |
The newer inline mesh generation introduced in the pre-okina tag of MFEM changed how elements were ordered according to a space-filling order. This option was made default in-order to satisfy some issues related to non-conforming meshes. It turns out this caused some diverging material responses in some of our standard monotonic tension tests out to 1% strain when using symmetric BCs. We therefore need to tell the inline mesh generator that we want to use the old lexicographic ordering.
I'm a little hesitant to merge the MFEM v4.0rc1 changes into the current ExaConstit branch. The reasons are that I'm finding some minor inconsistencies between the calculated residual for the initial time step even though nothing has changed within the ExaConstit framework. I'm also seeing differences in the outputted average stress tensor, but I'm pretty sure this more related to the difference in the residuals causing the solutions to be slightly different.
Here's an example from a small simulation of 1000 elements.
New version:
Old version:
I'm going to run a larger production run to see what the differences look like there before moving things forward.
edit: I finished one of the production runs, and the difference in that initial residual definitely had a larger effect by 1% strain.
Final average stress state master as of :
Final average stress state v4rc1 branch: