feat: hypre improvements

[victorapm]

• Replace internal GEOS SDC implementation with hypre's
• Remove tracking of certain SDC times
• Add hypre log level support
• Update hypre umpire pools naming
• Add missing coarse solver options in AMG
• Remove unused SDC variables in GEOS data structures
• Add separateComponents option to mechanics solver setup in MGR
• Move addCommaSeparators to StringUtilities
• Add unknowns and nonzeros info to linear solver message, e.g.:

Linear Solver | Success | Unknowns: 645,162 | Nonzeros: 42,898,950 | Iterations: 39 | Final Rel Res: 9.4042e-05 | Setup Time: 0.538 s | Solve Time: 0.933 s

Requires GEOS-DEV/thirdPartyLibs#286

[ryar9534]

Here is a quick test I ran on a version of SPE10 with burdens, compositional poromechanics, though with homogeneous properties, (so its just a high perm slab between two low perm boxes). Should be about 40 million DoF, and I ran on Dane.

Here were the results with develop a few weeks ago

Mpi ranks	Total Run Time (s)	Linear Solver Setup (s)	Linear Solver Solve (s)	Total GMRES Iterations (always 43 Total Newton iters)
64	2087	523	1966	1850
128	1451	343	1381	1833
256	963	234	662	1890
512	624	182	404	1974

Here are the results with the new Hypre

Mpi ranks	Total Run Time (s)	Linear Solver Setup (s)	Linear Solver Solve (s)	Total GMRES Iterations (always 43 Total Newton iters)
64	1568	233	1162	2281
128	1115	139	865	2229
256	723	85	572	2293
512	470	85	347	2366

Note that this does not double the number of nodes for each test, I will do that now

[ryar9534]

Old results for heterogeneous SPE10 compositional poromechanics on CPU (dane)

Dane cores (Mpi ranks)	Total Run Time (s)	Linear Solver Setup (s)	Linear Solver Solve (s)	Time steps/Nonlin iters / Lin iters
2 (224)	2409	488	1764	24/76/3464
4 (448)	1440	380	977	24/76/3862
8 (896)	1075	447	583	24/76/4286

New results for heterogeneous SPE10 compositional poromechanics on CPU (dane)

Dane cores (Mpi ranks)	Total Run Time (s)	Linear Solver Setup (s)	Linear Solver Solve (s)	Time steps/Nonlin iters / Lin iters
2 (224)	1770	185	1425	24/76/3882
4 (448)	952	152	719	24/76/3960
8 (896)	686	225	417	24/76/4228

[paveltomin]

can it be set by linear solver log level?

[victorapm]

Yes, but I did this way because the information provided here is mainly for developers. Other libraries such as umpire and rocsparse work this way as well (using env variable). I am open to change the strategy here if needeed

[paveltomin]

sounds reasonable, thanks, no need to change

[CusiniM]

let's not use environment variables please.

[victorapm]

I can remove this. What is the issue with env vars again?

[CusiniM]

I personally don't like them. It's hard to keep track of who sets them. Also we don't really do it for anything else in GEOS.

[victorapm]

Here's a place where it's being used:

GEOS/src/coreComponents/mainInterface/initialization.cpp

Line 141 in cd8175d

int columns = getenv( "COLUMNS" ) ? atoi( getenv( "COLUMNS" )) : 120;

HYPRE_LOG_LEVEL is an option supposed to be used by developers only. It's very convenient to turn it on/off if it's a env var wrt a XML input. I don't see any harm with using env var specifically, but I'm happy to remove it if that's the best solution

[victorapm]

@castelletto1 @CusiniM @rrsettgast Can you guys take it from here? It seems there's nothing else I can do. Thanks!

[CusiniM]

@castelletto1 @CusiniM @rrsettgast Can you guys take it from here? It seems there's nothing else I can do. Thanks!

is this basically ready? I mean it clearly needs an LvArray PR and a rebaseline but is it ready otherwise?

[victorapm]

Right, it's ready. Needs a LvArray PR and LC build, which I can't do. Rebaseline is due to field value change (amgNumFunctions)