Euclid does not support 64 bit indices

[lindsayad]

Hi, I was attempting to run Euclid through PETSc but ran into an error message that Euclid does not support 64 bit indices, which I need for a large application. How much work would it be to add 64 bit support for Euclid and are there any plans for the core development team to add this?

[victorapm]

Hi Alex,

we recommend using hypre's ILU, which is being actively developed by the team. It has 64-bit integer and GPU (CUDA/HIP) support. Here you can find information about it: https://hypre.readthedocs.io/en/latest/solvers-hypre-ilu.html

[victorapm]

PS: just realized your interest to use ILU through PETSc. However, PETSc currently does not have an interface to hypre's ILU. @oseikuffuor1 and @liruipeng might have something to add here

[adam-sim-dev]

Hi Alex,

we recommend using hypre's ILU, which is being actively developed by the team. It has 64-bit integer and GPU (CUDA/HIP) support. Here you can find information about it: https://hypre.readthedocs.io/en/latest/solvers-hypre-ilu.html

Hi @victorapm , the document is not complete as far as I see. Could you please provide me an ilut example? I want to have a try for my problem and replace pilut.

[victorapm]

Hi Adam,

thank you for pointing this out. The team will work on improving the documentation. Here are some instructions to get ILUT working in the meantime:

HYPRE_ILUSetType(solver, 1);
HYPRE_ILUSetMaxNnzPerRow(solver, max_nnz_row);
HYPRE_ILUSetDropThreshold(solver, threshold);

More details on the functions above:

hypre/src/parcsr_ls/HYPRE_parcsr_ls.h

Lines 4463 to 4480 in 8f6bdc6

	/**
	* Set the type of ILU factorization.
	*
	* Options for \e ilu_type are:
	* - 0 : BJ with ILU(k) (default, with k = 0)
	* - 1 : BJ with ILUT
	* - 10 : GMRES with ILU(k)
	* - 11 : GMRES with ILUT
	* - 20 : NSH with ILU(k)
	* - 21 : NSH with ILUT
	* - 30 : RAS with ILU(k)
	* - 31 : RAS with ILUT
	* - 40 : (nonsymmetric permutation) DDPQ-GMRES with ILU(k)
	* - 41 : (nonsymmetric permutation) DDPQ-GMRES with ILUT
	* - 50 : GMRES with RAP-ILU(0) using MILU(0) for P
	**/
	HYPRE_Int
	HYPRE_ILUSetType( HYPRE_Solver solver, HYPRE_Int ilu_type );

hypre/src/parcsr_ls/HYPRE_parcsr_ls.h

Lines 4404 to 4409 in 8f6bdc6

	/**
	* (Optional) Set the max non-zeros per row in L and U factors (for ILUT)
	* The default is 1000.
	**/
	HYPRE_Int
	HYPRE_ILUSetMaxNnzPerRow( HYPRE_Solver solver, HYPRE_Int nzmax );

hypre/src/parcsr_ls/HYPRE_parcsr_ls.h

Lines 4411 to 4417 in 8f6bdc6

	/**
	* (Optional) Set the threshold for dropping in L and U factors (for ILUT).
	* Any fill-in less than this threshold is dropped in the factorization.
	* The default is 1.0e-2.
	**/
	HYPRE_Int
	HYPRE_ILUSetDropThreshold( HYPRE_Solver solver, HYPRE_Real threshold );

[lindsayad]

PS: just realized your interest to use ILU through PETSc. However, PETSc currently does not have an interface to hypre's ILU.

Right. Yea I was curious what the actively developed option is. It might be helpful in the Euclid docs to direct the reader to ILU. With that knowledge, I may try to add the hooks in PETSc for hypre-ILU in the not-too-distant future.

I'm curious what is motivating the ILU development? I ask in case there is an overlap in motivation. I'm interested in potentially using parallel ILU for Schur complement preconditioning or potentially even for preconditioning the momentum equations in implicit transient Navier-Stokes (although I am also investigating AIR for the latter).

[victorapm]

It might be helpful in the Euclid docs to direct the reader to ILU.

Agreed! I noted this down.

With that knowledge, I may try to add the hooks in PETSc for hypre-ILU in the not-too-distant future.

Sounds great!

I'm curious what is motivating the ILU development?

Euclid and ParILUT have scalability issues at large scale and do not make full use of the newest hypre functions involving ParCSR matrices (which support GPUs). So, the team decided to start from scratch. @liruipeng and @oseikuffuor1 can say more. PS: Navier-Stokes is one of the applications for it, multiphysics problems at the subsurface, ILU being used through the MGR preconditioner, is another application.

I'm interested in potentially using parallel ILU for Schur complement preconditioning or potentially even for preconditioning the momentum equations in implicit transient Navier-Stokes

Nice! Keep us posted. We can setup a meeting and chat more if you want (paludettomag1 at llnl dot gov)

[lindsayad]

Great, thanks @victorapm. I may reach out at some point!

[victorapm]

I'm closing this issue since it seems resolved. Thanks!