Getting tensor mechanics ready for parallel

[tophmatthews]

ref #2975

[tophmatthews]

FYI @permcody

[tophmatthews]

All of the tests in tensor_mechanics/test/tests/crystal_plasticity seem to be flagged for deprecation, but FiniteStrainCrystalPlasticity has not been deprecated. This is the only test that fails, so I just restrict it to be non-parallel only. I'm not sure if @permcody or @dschwen plan on yanking these out soon or not?

[moosebuild]

Job Documentation on 5e60037 wanted to post the following:

View the site here

This comment will be updated on new commits.

[permcody]

@tophmatthews - If you plan to switch the pc_type to lu, you really need to add superlu = true to the test spec. I suppose we need to actually test a configuration of PETSc where superlu isn't installed s you can see the problem. In that case, lu will complain that it is a serial only preconditioner. PETSc will run superlu when the number of CPUs is greater than one.

[permcody]

@milljm - Do we have any good way of running a version of PETSc without SuperLU? In Lieu of that, we could potentially add an extra PETSc config check to make sure that SuperLU is available when running in parallel.... That actually sounds a lot easier. @fdkong - Can you confirm the validity of my statement above?

[milljm]

@permcody we do not... All our scripts add that configuration switch. Looks like we need to provide some TLC to our scripts/update_and_rebuild_petsc.sh script. eg, allow a way to disable/add a configuration much like we would do with

update_and_rebuild_libmesh.sh --disable-shared --enable-static

for example.

[fdkong]

When getting any module parallel things in, it is better to test against petsc-alt as well.

[tophmatthews]

If you plan to switch the pc_type to lu, you really need to add superlu = true to the test spec.

I'm not sure lu is necessary, rather that asm was giving different answers parallel/serial, not because it's necessary. Maybe there is a better one to use? I'll play around.

[tophmatthews]

When getting any module parallel things in, it is better to test against petsc-alt as well.

Also, I found some tests failed depending on processor numbers, ie sometimes passed in -p2, but failed in -p3 or -p4.

Also, the heavy switch brought out some other failures.

[dschwen]

I'm phase field I'm seeing that I have to tighten tolerances quite a bit to make parallel tests pass with asm.

…

On Tue, May 7, 2019, 11:37 AM Topher Matthews ***@***.***> wrote: When getting any module parallel things in, it is better to test against petsc-alt as well. Also, I found some tests failed depending on processor numbers, ie sometimes passed in -p2, but failed in -p3 or -p4. Also, the heavy switch brought out some other failures. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13364 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABRMPT5E3QIRIECMEVTBLTPUG45BANCNFSM4HLD77KQ> .

[tophmatthews]

I'm phase field I'm seeing that I have to tighten tolerances quite a bit to make parallel tests pass with asm.

Tolerances work for most of the failures, but then it becomes a balancing act to get convergence in serial, I couldn't find an appropriate balance with just tolerances.

[tophmatthews]

@permcody can you add TM to Partial Modules Parallel Test (In progress)?

[permcody]

TM is added. I'm ok with adding lu as a preconditioner. I just want to make sure that we don't pick up regressions for PETSc builds that don't have SuperLU or Mumps installed. The only robust way for me to do this is to add a target that actually tests that configuration. We'll be working on that!

[tophmatthews]

Removed lu where possible, and added superlu = true to the test spec when unavoidable.

[dschwen]

but then it becomes a balancing act to get convergence in serial

Are you regolding with tighter tolerances?

[tophmatthews]

Are you regolding with tighter tolerances?

I was trying to avoid regolding? I didn't have to for some with tighter tolerances, and lu worked for the others

[tophmatthews]

Also, from looking at the exodiff, regolding wouldn't have worked, at least for one or two cases I was looking at.

[tophmatthews]

UGGGG SQA! Well if that's not motivation to not touch the tests file, I'm not sure what is