Unable to build VASP 6.2.0 with GPU support

[robyelle]

Hello,

I am unable to build VASP 6.2.0 with VASPSol and GPU support. Everything seems to compile successfully, but at the linking stage I get this error:

mpif90 -mkl=sequential -lstdc++ -o vasp c2f_interface.o cuda_interface.o simd.o base.o base_gpu.o profiling.o string.o tutor.o version.o vhdf5_base.o incar_reader.o reader_base.o openmp.o openacc_struct.o mpi_gpu.o mpi_shmem.o mathtools.o hamil_struct.o radial_struct.o pseudo_struct.o mgrid_struct.o wave_struct_gpu.o nl_struct_gpu.o mkpoints_struct.o poscar_struct.o afqmc_struct.o phonon_struct.o fock_glb.o chi_glb.o smart_allocate.o xml.o extpot_glb.o constant.o vdwforcefield_glb.o jacobi.o main_mpi.o openacc.o scala.o asa.o lattice.o poscar.o ini.o mgrid.o setex_struct.o xclib.o vdw_nl.o xclib_grad_gpu.o setex.o radial.o pseudo.o gridq.o ebs.o symlib.o mkpoints.o wave_gpu.o wave_mpi_gpu.o wave_high_gpu.o bext.o spinsym.o symmetry.o lattlib.o random.o nonl.o nonlr_gpu.o nonl_high.o nonl_high_gpu.o dfast.o dfast_gpu.o choleski2_gpu.o mix.o hamil.o hamil_gpu.o xcgrad_gpu.o xcspin.o potex1.o potex2.o constrmag.o cl_shift.o relativistic.o LDApU.o paw_base.o metagga.o egrad.o pawsym.o pawfock.o pawlhf.o diis.o rhfatm.o hyperfine.o fock_ace.o paw.o mkpoints_full.o charge.o charge_gpu.o Lebedev-Laikov.o stockholder.o dipol.o solvation.o scpc.o pot.o pot_gpu.o tet.o dos.o elf.o hamil_rot.o chain.o dyna.o us.o core_rel.o aedens.o wavpre.o wavpre_noio_gpu.o broyden.o dynbr.o reader.o writer.o xml_writer.o brent.o stufak.o fileio.o vhdf5.o sphpro.o opergrid.o stepver.o chgloc.o fast_aug.o fock_multipole.o fock_gpu.o mkpoints_change.o subrot_cluster.o sym_grad.o mymath.o npt_dynamics.o subdftd3.o subdftd4.o internals.o dynconstr.o dimer_heyden.o dvvtrajectory.o vdwforcefield.o nmr.o pead.o k-proj.o subrot_gpu.o subrot_scf.o paircorrection.o rpa_force.o ml_interface.o force.o pwlhf.o gw_model.o optreal.o steep.o rmm-diis_gpu.o davidson_gpu.o david_inner.o root_find.o lcao_bare.o locproj.o electron_common.o electron_gpu.o rot.o electron_all.o shm.o pardens.o optics.o constr_cell_relax.o stm.o finite_diff.o elpol.o hamil_lr.o rmm-diis_lr.o subrot_lr.o lr_helper.o hamil_lrf.o elinear_response.o ilinear_response.o linear_optics.o setlocalpp.o wannier.o electron_OEP.o electron_lhf.o twoelectron4o.o gauss_quad.o m_unirnk.o minimax_ini.o minimax_dependence.o minimax_functions1D.o minimax_functions2D.o minimax_struct.o minimax_varpro.o minimax.o mlwf.o ratpol.o pade_fit.o screened_2e.o wave_cacher.o crpa.o chi_base.o wpot.o local_field.o ump2.o ump2kpar.o fcidump.o ump2no.o bse_te.o bse.o time_propagation.o acfdt.o afqmc.o main_gpu.o rpax.o chi.o acfdt_GG.o dmft.o GG_base.o greens_orbital.o lt_mp2.o rnd_orb_mp2.o greens_real_space.o chi_GG.o chi_super.o sydmat.o rmm-diis_mlr.o linear_response_NMR.o wannier_interpol.o wave_interpolate.o linear_response.o auger.o dmatrix.o phonon.o wannier_mats.o elphon.o core_con_mat.o embed.o extpot.o fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d_gpu.o fftmpiw_gpu.o main.o -Llib -ldmy -Lparser -lparser /usr/local/apps/intel/2019/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a -lmkl_blacs_openmpi_lp64 -Lparser -lparser -lstdc++ CUDA/lib/libCudaUtils_x86_64.a -L/usr/local/apps/cuda/10.2//lib64 -lnvToolsExt -lcudart -lcuda -lcufft -lcublas
solvation.o: In function pot_k_mp_vcorrection_': solvation.f90:(.text+0xb0de): undefined reference to m_sum_1_'
solvation.f90:(.text+0xb213): undefined reference to m_sum_1_' make[2]: *** [vasp] Error 1 make[2]: Leaving directory /nfs/hpc/apps/vasp/6.2.0/build/gpu'
cp: cannot stat ‘vasp’: No such file or directory
make[1]: *** [all] Error 1
make[1]: Leaving directory `/nfs/hpc/apps/vasp/6.2.0/build/gpu'

The non-GPU versions build without issue. Let me know what other information you need from me.

Thanks,

Rob

[shk11]

Hi Rob,

the GPU-Port of VASP doesn't support the VASPsol implementation. In order to use VASPsol within the GPU port, some additional subroutines have to be added. Therefore, I have created a corresponding patch for VASP 6.2.0 to make VASPsol work correctly with the GPU port:

VASPsol_VASP620.zip

1. Update the solvation routines by copying the VASPsol/src/solvation.F file to the src/ folder in the original VASP folder
2. Copy the VASPsol_VASP620.patch file (see attachment) into the src folder of VASP 6.2.0
3. Patch the pot.F, pot_gpu.F, solvation.F, mpi.F, and mpi_gpu.F file: patch -p0 < VASPsol_VASP620.patch
4. Compile the code as described in the VASP wiki (don't forget to set the CPP option -Dsol_compat in the makefile.include file)

The patch already includes the modifications from #43.
Consequently, the patch from the current VASPsol Github should not be applied at the same time as the attached patch.

The problem reported in #40 should also be fixed with this patch, as the pot_gpu.F has been extended to include the missing VASPsol routines.

This should fix the issues.

Best wishes,
Daniel

[robyelle]

Hi Daniel,

I was able to build VASP 6.2.0+VASPSol with this patch per your directions above.

Thanks!

Rob

[LongyuanZhang]

Hi Daniel,

I'm using VASP 5.4.4 and I also got the same problem with the GPU port. Can I use this same patch you uploaded here to make VASPsol work?

Best regards,
Longyuan

[shk11]

Hi Longyuan,

unfortunately, I have not tested it for VASP 5.4.4 yet. In principle, the same adjustments would have to be made within the mpi files, however, the VASP developers have introduced some changes in the code between VASP5 and VASP6. Thus, some minor tweaks to the patch file may be needed.

Best wishes,
Daniel

[LongyuanZhang]

Hi Daniel,

Thanks for your response. I'll try to upgrade my VASP.

Best,
Longyuan

[m3lab-zzl]

Hi, there. I can not compile VASP620 with VASPsol, it always complains as follows:

mpiifort -qmkl -o vasp c2f_interface.o nccl2for.o simd.o base.o profiling.o string.o tutor.o version.o vhdf5_base.o incar_reader.o reader_base.o openmp.o openacc_struct.o mpi.o mpi_shmem.o mathtools.o hamil_struct.o radial_struct.o pseudo_struct.o mgrid_struct.o wave_struct.o nl_struct.o mkpoints_struct.o poscar_struct.o afqmc_struct.o phonon_struct.o fock_glb.o chi_glb.o smart_allocate.o xml.o extpot_glb.o constant.o vdwforcefield_glb.o jacobi.o main_mpi.o openacc.o scala.o asa.o lattice.o poscar.o ini.o mgrid.o setex_struct.o xclib.o vdw_nl.o xclib_grad.o setex.o radial.o pseudo.o gridq.o ebs.o symlib.o mkpoints.o wave.o wave_mpi.o wave_high.o bext.o spinsym.o symmetry.o lattlib.o random.o nonl.o nonlr.o nonl_high.o dfast.o choleski2.o mix.o hamil.o xcgrad.o xcspin.o potex1.o potex2.o constrmag.o cl_shift.o relativistic.o LDApU.o paw_base.o metagga.o egrad.o pawsym.o pawfock.o pawlhf.o diis.o rhfatm.o hyperfine.o fock_ace.o paw.o mkpoints_full.o charge.o Lebedev-Laikov.o stockholder.o dipol.o solvation.o scpc.o solvation.o pot.o tet.o dos.o elf.o hamil_rot.o bfgs.o dynmat.o  instanton.o  lbfgs.o sd.o   cg.o dimer.o bbm.o fire.o lanczos.o neb.o  qm.o opt.o chain.o dyna.o fileio.o vhdf5.o sphpro.o us.o core_rel.o aedens.o wavpre.o wavpre_noio.o broyden.o dynbr.o reader.o writer.o xml_writer.o brent.o stufak.o opergrid.o stepver.o chgloc.o fast_aug.o fock_multipole.o fock.o fock_dbl.o fock_frc.o mkpoints_change.o subrot_cluster.o sym_grad.o mymath.o npt_dynamics.o subdftd3.o subdftd4.o internals.o dynconstr.o dimer_heyden.o dvvtrajectory.o vdwforcefield.o nmr.o pead.o k-proj.o subrot.o subrot_scf.o paircorrection.o rpa_force.o ml_interface.o force.o pwlhf.o gw_model.o optreal.o steep.o rmm-diis.o davidson.o david_inner.o root_find.o lcao_bare.o locproj.o electron_common.o electron.o rot.o electron_all.o shm.o pardens.o optics.o constr_cell_relax.o stm.o finite_diff.o elpol.o hamil_lr.o rmm-diis_lr.o subrot_lr.o lr_helper.o hamil_lrf.o elinear_response.o ilinear_response.o linear_optics.o setlocalpp.o wannier.o electron_OEP.o electron_lhf.o twoelectron4o.o gauss_quad.o m_unirnk.o minimax_ini.o minimax_dependence.o minimax_functions1D.o minimax_functions2D.o minimax_struct.o minimax_varpro.o minimax.o mlwf.o ratpol.o pade_fit.o screened_2e.o wave_cacher.o crpa.o chi_base.o wpot.o local_field.o ump2.o ump2kpar.o fcidump.o ump2no.o bse_te.o bse.o time_propagation.o acfdt.o afqmc.o rpax.o chi.o acfdt_GG.o dmft.o GG_base.o greens_orbital.o lt_mp2.o rnd_orb_mp2.o greens_real_space.o chi_GG.o chi_super.o sydmat.o rmm-diis_mlr.o linear_response_NMR.o wannier_interpol.o wave_interpolate.o linear_response.o auger.o dmatrix.o phonon.o wannier_mats.o elphon.o core_con_mat.o embed.o extpot.o fftmpiw.o fftmpi_map.o fft3dlib.o fftw3d.o main.o  -Llib -ldmy -Lparser -lparser /data/home/df101508/df101508/intel/oneapi/mkl/2022.1.0/lib/intel64/libmkl_scalapack_lp64.a -lmkl_blacs_intelmpi_lp64 -lstdc++ 
solvation.o: In function `pot_lpcm_k._':
solvation.f90:(.text+0x0): multiple definition of `pot_lpcm_k._'
solvation.o:solvation.f90:(.text+0x0): first defined here
solvation.o: In function `pot_lpcm_k_mp_shape_gradn_':
solvation.f90:(.text+0x10): multiple definition of `pot_lpcm_k_mp_shape_gradn_'
solvation.o:solvation.f90:(.text+0x10): first defined here
solvation.o:(.data+0x308): multiple definition of `pot_lpcm_k_mp_chgmin_'
solvation.o:(.data+0x308): first defined here
solvation.o: In function `pot_lpcm_k_mp_simplepoisson_':
solvation.f90:(.text+0x260): multiple definition of `pot_lpcm_k_mp_simplepoisson_'
solvation.o:solvation.f90:(.text+0x260): first defined here
solvation.o: In function `pot_lpcm_k_mp_pack_r2c_':
solvation.f90:(.text+0x1350): multiple definition of `pot_lpcm_k_mp_pack_r2c_'
solvation.o:solvation.f90:(.text+0x1350): first defined here
solvation.o: In function `pot_k._':
solvation.f90:(.text+0x1650): multiple definition of `pot_k._'
solvation.o:solvation.f90:(.text+0x1650): first defined here
solvation.o: In function `pot_k_mp_ionic_charge_':
solvation.f90:(.text+0x1660): multiple definition of `pot_k_mp_ionic_charge_'
solvation.o:solvation.f90:(.text+0x1660): first defined here
solvation.o:(.data+0x440): multiple definition of `pot_k_mp_core_c_'
solvation.o:(.data+0x440): first defined here
solvation.o:(.data+0x4a0): multiple definition of `pot_k_mp_sigma_rc_k_'
solvation.o:(.data+0x4a0): first defined here
solvation.o: In function `pot_k_mp_getrcore_':
solvation.f90:(.text+0x3720): multiple definition of `pot_k_mp_getrcore_'
solvation.o:solvation.f90:(.text+0x3720): first defined here
solvation.o: In function `pot_k_mp_jdftx_interface_':
solvation.f90:(.text+0x3b20): multiple definition of `pot_k_mp_jdftx_interface_'
solvation.o:solvation.f90:(.text+0x3b20): first defined here
solvation.o: In function `pot_k_mp_get_fermishift_':
solvation.f90:(.text+0x40c0): multiple definition of `pot_k_mp_get_fermishift_'
solvation.o:solvation.f90:(.text+0x40c0): first defined here
solvation.o: In function `pot_lpcm_k_mp_get_vac_level_':
solvation.f90:(.text+0x71c0): multiple definition of `pot_lpcm_k_mp_get_vac_level_'
solvation.o:solvation.f90:(.text+0x71c0): first defined here
solvation.o:(.bss+0x0): multiple definition of `pot_lpcm_k_mp_vacpot_pois_'
solvation.o:(.bss+0x0): first defined here
solvation.o: In function `solvation._':
solvation.f90:(.text+0x7700): multiple definition of `solvation._'
solvation.o:solvation.f90:(.text+0x7700): first defined here
solvation.o: In function `solvation_mp_sol_reader_':
solvation.f90:(.text+0x7710): multiple definition of `solvation_mp_sol_reader_'
solvation.o:solvation.f90:(.text+0x7710): first defined here
solvation.o:(.data+0x520): multiple definition of `solvation_mp_eifor_sol_'
solvation.o:(.data+0x520): first defined here
solvation.o:(.bss+0x48): multiple definition of `solvation_mp_ediel_sol_'
solvation.o:(.bss+0x48): first defined here
solvation.o:(.bss+0x60): multiple definition of `solvation_mp_lsol_'
solvation.o:(.bss+0x60): first defined here
solvation.o:(.data+0x508): multiple definition of `solvation_mp_sigma_k_'
solvation.o:(.data+0x508): first defined here
solvation.o:(.data+0x500): multiple definition of `solvation_mp_nc_k_'
solvation.o:(.data+0x500): first defined here
solvation.o:(.data+0x4f8): multiple definition of `solvation_mp_eb_k_'
solvation.o:(.data+0x4f8): first defined here
solvation.o:(.data+0x4f0): multiple definition of `solvation_mp_tau_'
solvation.o:(.data+0x4f0): first defined here
solvation.o:(.data+0x4e8): multiple definition of `solvation_mp_lambda_d_k_'
solvation.o:(.data+0x4e8): first defined here
solvation.o:(.bss+0x40): multiple definition of `solvation_mp_kappa2b_k_'
solvation.o:(.bss+0x40): first defined here
solvation.o:(.bss+0x58): multiple definition of `solvation_mp_lrhob_'
solvation.o:(.bss+0x58): first defined here
solvation.o:(.bss+0x50): multiple definition of `solvation_mp_lrhoion_'
solvation.o:(.bss+0x50): first defined here
solvation.o: In function `solvation_mp_sol_writer_':
solvation.f90:(.text+0x8da0): multiple definition of `solvation_mp_sol_writer_'
solvation.o:solvation.f90:(.text+0x8da0): first defined here
solvation.o: In function `solvation_mp_xml_write_sol_':
solvation.f90:(.text+0x8ec0): multiple definition of `solvation_mp_xml_write_sol_'
solvation.o:solvation.f90:(.text+0x8ec0): first defined here
solvation.o: In function `solvation_mp_sol_vcorrection_':
solvation.f90:(.text+0x9150): multiple definition of `solvation_mp_sol_vcorrection_'
solvation.o:solvation.f90:(.text+0x9150): first defined here
solvation.o: In function `pot_k_mp_sol_init_':
solvation.f90:(.text+0x245d0): multiple definition of `pot_k_mp_sol_init_'
solvation.o:solvation.f90:(.text+0x245d0): first defined here
solvation.o: In function `pot_k_mp_vcorrection_':
solvation.f90:(.text+0x9680): multiple definition of `pot_k_mp_vcorrection_'
solvation.o:solvation.f90:(.text+0x9680): first defined here
solvation.o:(.bss+0x38): multiple definition of `pot_k_mp_lsol_'
solvation.o:(.bss+0x38): first defined here
solvation.o:(.bss+0x20): multiple definition of `pot_k_mp_ldefaultpcm_'
solvation.o:(.bss+0x20): first defined here
solvation.o:(.bss+0x30): multiple definition of `pot_k_mp_lrhob_'
solvation.o:(.bss+0x30): first defined here
solvation.o:(.bss+0x28): multiple definition of `pot_k_mp_lrhoion_'
solvation.o:(.bss+0x28): first defined here
solvation.o: In function `pot_lpcm_k_mp_vcorrection_lpcm_':
solvation.f90:(.text+0xd410): multiple definition of `pot_lpcm_k_mp_vcorrection_lpcm_'
solvation.o:solvation.f90:(.text+0xd410): first defined here
solvation.o: In function `pot_k_mp_fcorrection_':
solvation.f90:(.text+0xb070): multiple definition of `pot_k_mp_fcorrection_'
solvation.o:solvation.f90:(.text+0xb070): first defined here
solvation.o:(.data+0x428): multiple definition of `pot_k_mp_pk_counter_'
solvation.o:(.data+0x428): first defined here
solvation.o:(.data+0x310): multiple definition of `pot_lpcm_k_mp_pk_counter_'
solvation.o:(.data+0x310): first defined here
solvation.o:(.data+0x320): multiple definition of `pot_lpcm_k_mp_core_c_'
solvation.o:(.data+0x320): first defined here
solvation.o:(.data+0x380): multiple definition of `pot_lpcm_k_mp_sigma_rc_k_'
solvation.o:(.data+0x380): first defined here
solvation.o:(.bss+0x8): multiple definition of `pot_lpcm_k_mp_lrhoion_'
solvation.o:(.bss+0x8): first defined here
solvation.o:(.bss+0x10): multiple definition of `pot_lpcm_k_mp_lrhob_'
solvation.o:(.bss+0x10): first defined here
solvation.o: In function `pot_lpcm_k_mp_createcavity_':
solvation.f90:(.text+0x10b40): multiple definition of `pot_lpcm_k_mp_createcavity_'
solvation.o:solvation.f90:(.text+0x10b40): first defined here
solvation.o: In function `pot_lpcm_k_mp_minimize_':
solvation.f90:(.text+0x18530): multiple definition of `pot_lpcm_k_mp_minimize_'
solvation.o:solvation.f90:(.text+0x18530): first defined here
solvation.o: In function `pot_lpcm_k_mp_unpack_c2r_':
solvation.f90:(.text+0x10850): multiple definition of `pot_lpcm_k_mp_unpack_c2r_'
solvation.o:solvation.f90:(.text+0x10850): first defined here
solvation.o: In function `pot_lpcm_k_mp_eps_pot_':
solvation.f90:(.text+0x1d6a0): multiple definition of `pot_lpcm_k_mp_eps_pot_'
solvation.o:solvation.f90:(.text+0x1d6a0): first defined here
solvation.o: In function `pot_lpcm_k_mp_my_d_prod_':
solvation.f90:(.text+0x1c1f0): multiple definition of `pot_lpcm_k_mp_my_d_prod_'
solvation.o:solvation.f90:(.text+0x1c1f0): first defined here
solvation.o: In function `pot_lpcm_k_mp_compute_rhob_':
solvation.f90:(.text+0x1f730): multiple definition of `pot_lpcm_k_mp_compute_rhob_'
solvation.o:solvation.f90:(.text+0x1f730): first defined here
solvation.o: In function `pot_lpcm_k_mp_compute_rhoion_':
solvation.f90:(.text+0x21a00): multiple definition of `pot_lpcm_k_mp_compute_rhoion_'
solvation.o:solvation.f90:(.text+0x21a00): first defined here
solvation.o:(.data+0x3e0): multiple definition of `pot_lpcm_k_mp_sol_phi_'
solvation.o:(.data+0x3e0): first defined here
solvation.o: In function `pot_lpcm_k_mp_nablan_':
solvation.f90:(.text+0x163c0): multiple definition of `pot_lpcm_k_mp_nablan_'
solvation.o:solvation.f90:(.text+0x163c0): first defined here
solvation.o: In function `pot_lpcm_k_mp_lapln_':
solvation.f90:(.text+0x14a10): multiple definition of `pot_lpcm_k_mp_lapln_'
solvation.o:solvation.f90:(.text+0x14a10): first defined here
solvation.o: In function `pot_lpcm_k_mp_leps_':
solvation.f90:(.text+0x1c330): multiple definition of `pot_lpcm_k_mp_leps_'
solvation.o:solvation.f90:(.text+0x1c330): first defined here
solvation.o: In function `pot_lpcm_k_mp_write_to_file_':
solvation.f90:(.text+0x20d80): multiple definition of `pot_lpcm_k_mp_write_to_file_'
solvation.o:solvation.f90:(.text+0x20d80): first defined here
solvation.o:(.bss+0x18): multiple definition of `pot_k_mp_ljdftx_'
solvation.o:(.bss+0x18): first defined here
make[2]: *** [vasp] Error 1

I followed the instruction in README.md, patched with patch_610, but it still failed as above. What should I do?

I am using oneAPI-2022.1.0 toolchain on Intel(R) Xeon(R) E5-2680 v4

[ericfonseca95]

Hello m3lab-zzl,

You may have patched the same file multiple times. Go into the files that the patches is being applied to (pot.F, mpi.F, and solvaiton.F) search for the keyword "!solvation" added by the patch, and make sure 1 copy of each function is present and then recompile. Let me know if this resolves your issue.

Eric

[m3lab-zzl]

Thank you for your advice, @ericfonseca95

I will check it later, and post my findings then.

[m3lab-zzl]

@ericfonseca95 You were right, this time I made vasp with vaspsol successfully.

The message in this repo's README.md may be updated, we have to patch inside the /src subfolder, otherwise it can not find files to path.

[huangj3]

For those who successfully build VASPsol with the gpu port of VASP.6.2.0, could you please share your key flags in the makefile.include?

Thanks and best regards,

huangj3

[ericfonseca95]

Hi Everyone,

I am a member of the Hennig Research Group. We are currently working on a VASPsol GPU patch for all VASP versions. 5.4.4 - 6.3.2. Thank you all for your patience and we will post updates as functionality becomes available.

Eric Fonseca