Error while running CSC calculation (QE, W90)

[TommyHenz]

Hello everyone,
I'm writing to ask regarding an error that was occuring for me whilst I was following this tutorial about CSC calculation involving QE, W90 and TRIQS. I prepared all the files, but when I run the command mpirun solid_dmft > dmft.out &, I get an error saying:

There are not enough slots available in the system to satisfy the 25 slots 
that were requested by the application:
  python

Either request fewer slots for your application, or make more slots available
for use.

I'm running a Virtual Machine, so I believe it's bound to 3 slots, so if I change the n_cores parameter and set it to 3 in the dmft_config.ini file, the mpirun completes the SCF calculation, but crashes when it starts the NSCF calculation and says that an error occured at the point dmft=.true.

However, if I run singularily (with pw.x), my SCF, NSCF and W90 calculation run successfully (when I don't put csc=True in the dmft file). But I'm not sure if this would still be correct as I just get one iteration; I wanted to simulate as in the tutorial example of having 4 iterations shown at different points in the Brilluin Zone.

So, could you let me know how to fix the above issues?

Many thanks in advance

[ghost]

Could you try running initially with a single core and see if the issue persists?

mpirun -n 1 solid_dmft | tee dmft.out

[TommyHenz]

Hi, I tried running with
mpirun -n 1 solid_dmft | tee dmft.out
and also
mpirun -n 3 solid_dmft | tee dmft.out

With this, the SCF and NSCF calculations do run, however when it starts the W90 code, it fails giving the following error:

  solid_dmft: Running QE...
  solid_dmft: Starting scf calculation...
  solid_dmft: Starting nscf calculation...
  solid_dmft: Starting Wannier90 win_pp...
At line 98 of file ../wannier_prog.F90 (unit = 10, file = 'tio.werr')
Fortran runtime error: File cannot be deleted

Error termination. Backtrace:
At line 98 of file ../wannier_prog.F90 (unit = 10, file = 'tio.werr')
Fortran runtime error: File cannot be deleted

Error termination. Backtrace:
#0  0x7f2a08ed5ad0 in ???
#1  0x7f2a08ed6649 in ???
#2  0x7f2a08ed727f in ???
#3  0x7f2a09119be0 in ???
#4  0x5612fe728e4f in ???
#5  0x5612fe72a831 in ???
#6  0x7f2a08acdd8f in __libc_start_call_main
    at ../sysdeps/nptl/libc_start_call_main.h:58
#7  0x7f2a08acde3f in __libc_start_main_impl
    at ../csu/libc-start.c:392
#8  0x5612fe728a54 in ???
#9  0xffffffffffffffff in ???
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
#0  0x7ff2b526aad0 in ???
#1  0x7ff2b526b649 in ???
#2  0x7ff2b526c27f in ???
#3  0x7ff2b54aebe0 in ???
#4  0x558c7067be4f in ???
#5  0x558c7067d831 in ???
#6  0x7ff2b4e62d8f in __libc_start_call_main
    at ../sysdeps/nptl/libc_start_call_main.h:58
#7  0x7ff2b4e62e3f in __libc_start_main_impl
    at ../csu/libc-start.c:392
#8  0x558c7067ba54 in ???
#9  0xffffffffffffffff in ???
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[8331,1],0]
  Exit code:	2
--------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/bin/solid_dmft", line 45, in <module>
	run_solid_dmft.main(sys.argv)
  File "/usr/lib/python3.10/dist-packages/solid_dmft/main.py", line 103, in main
	csc_flow_control(general_params, solver_params, dft_params, advanced_params)
  File "/usr/lib/python3.10/dist-packages/solid_dmft/csc_flow.py", line 315, in csc_flow_control
	_run_qe(general_params, dft_params, iter_dmft, iteration_offset)
  File "/usr/lib/python3.10/dist-packages/solid_dmft/csc_flow.py", line 122, in _run_qe
	qe_w90 = start_qe(dft_params['n_cores'], 'win_pp')
  File "/usr/lib/python3.10/dist-packages/solid_dmft/csc_flow.py", line 101, in <lambda>
	start_qe = lambda n_cores, calc_type: qe.start(n_cores, calc_type, dft_params['dft_exec'],
  File "/usr/lib/python3.10/dist-packages/solid_dmft/dft_managers/qe_manager.py", line 250, in start
	qe_process_id = _fork_and_start_qe(mpi_exe, (arguments, qe_file_ext), env_vars, seedname)
  File "/usr/lib/python3.10/dist-packages/solid_dmft/dft_managers/qe_manager.py", line 199, in _fork_and_start_qe
	qe_process_id = subprocess.check_call(mpi_arguments + [seedname], env=env_vars, shell=False)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
	raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/local/bin/mpirun', '-np', '3', 'wannier90.x', '-pp', 'tio']' returned non-zero exit status 2.

If its not possible to run with a complete CSC using mpirun, what steps would be if I run each singularly for SCF, then NSCF then W90 since it works that way

[the-hampel]

Hi,
it looks as if either wannier90.x is not correctly found or it is not compiled with mpi support. Can you try to run the command solid_dmft tries to execute?

/usr/local/bin/mpirun -np 3 wannier90.x -pp tio

in the directory where it crashed? and then try to run it without mpirun just as:

wannier90.x -pp tio

If this is the problem you can either recompile wannier90 with MPI support, or change the way wannier90.x is called within solid_dmft. But let's first see what the problem is.

Best,
Alex

[TommyHenz]

Hi, it's as you said. Tried running with
/usr/local/bin/mpirun -np 3 wannier90.x -pp tio
and it gave a runtime error, but with
wannier90.x -pp tio

It works. This is why solid_dmft stops. How can I change the way wannier90 is called in it?

Many thanks!

[phibeck]

Hi. It should work if you simply change dft_params['n_cores'] to 1 in lines 122 and 124 of csc_flow.py:

solid_dmft/python/solid_dmft/csc_flow.py

Line 122 in 5ba4cf1

qe_w90 = start_qe(dft_params['n_cores'], 'win_pp')

This was actually the default, but we changed it to wannier90 with mpi support some months ago, see 0b34ee3.

[the-hampel]

If you want to use wannier90 with MPI you have to rebuild Quantum Espresso without wannier90, and build wannier90 manually. QE will always build w90 without MPI support atm. For this, build first wannier90 with COMMS=MPI and then build QE with the following cmake option turned off QE_WANNIER90_INTERNAL=OFF :
https://github.com/flatironinstitute/ccq-software-build/blob/635271bbe10e4cdf0b84c01954db828bf518936c/quantum_espresso/7.1_nix2_ompi/install.sh#L45

This will force QE to use your own wannier90.