Instructions for installation (Ubuntu/WSL)

[AugustinLu]

At present, there is an installation section in the documentation, which describes how to compile the code in general.

However, it would be convenient and attractive for new users to have specific sections dedicated to specific systems besides the HPC ones, for instance MacOS, Linux or Windows. In particular, recent versions of Windows (10 and 11) have the Windows Subsystem for Linux, allowing for seamless use of a Linux virtual machine.

Here, we propose a detailed procedure to install required packages and compile CONQUEST on Ubuntu. The following have been tested so far:

• Works on Ubuntu, including with WSL in Windows [Ubuntu 22.04 and Windows 10/11 tested]
• Enables using CONQUEST in Windows through WSL (including MPI)

1. Install packages

sudo apt update
sudo apt upgrade

sudo apt install -y build-essential # gcc and other tools for software development

sudo apt install -y openmpi-bin libopenmpi-dev # MPI
sudo apt install -y libfftw3-dev # FFT
sudo apt install -y libblas-dev liblapack-dev libscalapack-openmpi-dev # Linear algebra

2. Install libxc

mkdir -p $HOME/local/src 
cd $HOME/local/src 

wget http://www.tddft.org/programs/libxc/down.php?file=6.2.2/libxc-6.2.2.tar.gz -O libxc.tar.gz
tar -zxf libxc.tar.gz 
cd libxc-6.2.2 && ./configure --prefix=$HOME/local
make -j4
make check && make install

3. Get CONQUEST source

cd $HOME/local/src
git clone https://github.com/OrderN/CONQUEST-release.git conquest_master
cd conquest_master/src

4. Copy the appropriate system.make makefile

To be provided as system.make.ubuntu or some similar name

#

# Set compilers
FC=mpif90
F77=mpif77

# Linking flags
LINKFLAGS= -L\${HOME}/local/lib -L/usr/local/lib -fopenmp
ARFLAGS=

# Compilation flags
# NB for gcc10 you need to add -fallow-argument-mismatch
COMPFLAGS= -O3 \$(XC_COMPFLAGS) -fallow-argument-mismatch
COMPFLAGS_F77= \$(COMPFLAGS)

# Set BLAS and LAPACK libraries
# MacOS X
# BLAS= -lvecLibFort
# Intel MKL use the Intel tool
# Generic
BLAS= -llapack -lblas

# Full library call; remove scalapack if using dummy diag module
LIBS= \$(FFT_LIB) \$(XC_LIB) -lscalapack-openmpi \$(BLAS)

# LibXC compatibility (LibXC below) or Conquest XC library

# Conquest XC library
#XC_LIBRARY = CQ
#XC_LIB =
#XC_COMPFLAGS =

# LibXC compatibility
# Choose LibXC version: v4 (deprecated) or v5/6 (v5 and v6 have the same interface)
# XC_LIBRARY = LibXC_v4
XC_LIBRARY = LibXC_v5
XC_LIB = -lxcf90 -lxc
#XC_COMPFLAGS = -I/usr/local/include
XC_COMPFLAGS = -I\${HOME}/local/include -I/usr/local/include

# Set FFT library
FFT_LIB=-lfftw3
FFT_OBJ=fft_fftw3.o

# Matrix multiplication kernel type
MULT_KERN = default
# Use dummy DiagModule or not
DIAG_DUMMY =

5. Compile CONQUEST

dos2unix ./makedeps # Some windows/unix incompatibilities may occur
make -j4 # Or make -j`nproc`  # Uses all cores available

This could also be useful for Windows/Ubuntu users at tutorial/hands-on sessions.

Any feedback is welcome.

Thank you.

[davidbowler]

Thank you for this suggestion. I think that there's a balance to be struck: general suggestions, which won't require frequent updating, are a good idea, but any more detailed instructions are likely to go out of date quite quickly. In this sense, I'm a little concerned about steps 1 and 2 in your instructions above but I'm happy to discuss this further. My feeling is that the Conquest installation instructions should make it clear what kind of packages are required, but not how to install these.

I think that it's an excellent idea to an appropriate system.make as an example (with comments at the top making it clear what it is).

[lionelalexandre]

I think this is a good idea to provide examples for installation to new users (particulary students). There are pros and cons for providing system-specific full install recipies including libraries. I would say that this can help even if there are expected to change quickly.

[AugustinLu]

It may also be put in a separate page for specific systems (ex: Ubuntu 22.04/23.04; Windows 10/11, MacOs14, etc.), with an archive page to refer to older systems. As written by @lionelalexandre, it would be useful for beginners, students or in general people would would like to test CONQUEST on their personal computer.

For production use on supercomputers by confirmed users, the main page in its current state is clear enough, I think.

[AugustinLu]

After rethinking a bit about it, here are some additional thoughts:

Thank you for this suggestion. I think that there's a balance to be struck: general suggestions, which won't require frequent updating, are a good idea, but any more detailed instructions are likely to go out of date quite quickly. In this sense, I'm a little concerned about steps 1 and 2 in your instructions above but I'm happy to discuss this further. My feeling is that the Conquest installation instructions should make it clear what kind of packages are required, but not how to install these.

What about creating specific pages for different systems in the installation section, like in ABINIT?

Then, there is no need to strike a balance if we have:

1. The main general installation page, which states in general what kind of package is required (this is the current documentation) and requires little maintenance.
2. System-specific installation pages, as what is done in Tensorflow, LAMMPS, etc., which give more detailed instructions. The version of the OS and the version of CONQUEST (or day) should be clearly indicated (ex: CONQUEST v1.2 on Ubuntu 23.04 (2024/01/15)).
3. (Optionally,) an archive page, where instructions for outdated systems are put when we find that they are no longer valid. This will typically happen with new OS versions, which would break the compilation.
4. (Optionally, maybe at a later stage) a compatibility page, similarly to what is done in CP2K.

I think that it's an excellent idea to an appropriate system.make as an example (with comments at the top making it clear what it is).

Such a system.make file is proposed in 5228d14. Maybe we could standardize the naming scheme for example system.make files in the src/system folder as system.make.[SystemLabel], for instance system.make.ubuntu2204 and use the first comment line to indicate the creation date and system version:

# (2024/01/28) system.make file for Ubuntu 22.04

[...]

Thank you for your feedback.

[davidbowler]

Thank you @AugustinLu for your continuing suggestions: they're really helpful. I think that the ABINIT example is a very good one, and I would be happy for us to go ahead along these lines.

In terms of system.make we need to have a discussion with @tkoskela and @ilectra who set up the present system which detects a system name and uses an existing file system.[SystemName].make for the compilation. We should agree a way for this and your suggestion to sit together.

[tkoskela]

An alternative would be using the spack package manager, following the instructions in #263 This automates the installation of dependencies.

The idea behind the current system of system.make files was to be able to provide a system.make file for (typically HPC) systems where the software environment is the same for all users. I'd be a bit cautious about providing system.make files for generic Ubuntu or Windows systems. There are so many unknowns that the user would have to manually edit the file in most cases anyways. The system/system.example.make is meant to be a template users can start from, it does need a few modifications on my Ubuntu machine before it will compile

1. Add -fallow-argument-mismatch to the compiler flags, which maybe should be the default since modern versions of gfortran are quite common
2. Include from -I/usr/include instead of -I/usr/local/include in XC_COMPFLAGS.
3. Link -lscalapack-openmpi instead of -lscalapack

I admit that is a lot to ask from a novice, but I'm not confident we can provide something that will always work out of the box. I don't have any experience with WSL but we can try to find out how different things look over there. Personally for non-expert users who just need to install and run a release version, I would recommend going down the spack route.

Another route would be using containers, where we provide a Ubuntu-22.04 (for example) image with CONQUEST and all dependencies pre-installed. This is not ideal for HPC systems, but could work on personal machines.

[AugustinLu]

Thank you for your comment.

I don't know about spack so I need to to my research to understand how it works. It seems that it would allow having a single-command method regardless of the OS and type of machine.

While I understand that in its current state, compilation should not be working out-of-the-box, I find it confusing to have a generic system.example.make file where instructions for Mac OS, openMP/non-openMP, gcc versions are all mixed together. To my opinion, having a slightly differentiated set of system.make files would help future users. For HPC users, I expect them to be experts so they typically don't need more than what is already in the main documentation page.

The system/system.example.make is meant to be a template users can start from, it does need a few modifications on my Ubuntu machine before it will compile
Add -fallow-argument-mismatch to the compiler flags, which maybe should be the default since modern versions of gfortran are quite common
Include from -I/usr/include instead of -I/usr/local/include in XC_COMPFLAGS.
Link -lscalapack-openmpi instead of -lscalapack

These are the typical instructions that would help new users if they can be found in a specific page of the documentation, which would require minimal effort to update the next year if the next release of the OS or compiler breaks it. Often, changing 1-2 options should fix it.

I admit that is a lot to ask from a novice, but I'm not confident we can provide something that will always work out of the box. I don't have any experience with WSL but we can try to find out how different things look over there. Personally for non-expert users who just need to install and run a release version, I would recommend going down the spack route.

Typically, Ubuntu in WSL is working like a regular Ubuntu (at least the 22.04 version I am currently using), thus expect the absence of the desktop environment, packages can be installed as usual and there is little to no difference between a native Ubuntu and the WSL virtual machine, when it comes to compiling and using CONQUEST. Personally, this is approach I am using for testing compilation and runs of CONQUEST before production runs on supercomputers.

Another route would be using containers, where we provide a Ubuntu-22.04 (for example) image with CONQUEST and all dependencies pre-installed. This is not ideal for HPC systems, but could work on personal machines.

This is a good approach for tutorials. Note that it is already available with MateriApps .