✨ NEW: Add initial implementation

[chrisjsewell]

No description provided.

[chrisjsewell]

Copied from Slack message to @ltalirz:

eurgh theres a problem with all this Conda business: to get conda activate to work you have to first run conda init bash, which adds things to .bashrc, then reload your terminal to trigger the .bashrc sourcing (which is normally only triggered for an interactive shell).
I can't get this to work at all when trying to test e.g. an abinit execution in ansible.
After runing conda init in a previous step, then trying e.g.:

- name: Run abinit test
  shell: bash -lic "{{ conda_code_exec }} activate {{ env.name }}; mpirun -np {{ conda_code_test_cpus }} '{{ conda_code_parent_dir }}/{{ conda_code_link_dir }}/envs/{{ env.name }}/bin/abinit' run.in"
  args:
    executable: /bin/bash
    chdir: /tmp/abinit_test/

fails with: bash: cannot set terminal process group (61075): Inappropriate ioctl for device, i.e. it won't work interactively
or trying shell: source ${HOME}/.bashrc; conda activate abinit doesn't work (CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.), and other combinations I've tried
(note if I login to the Docker container manually and do the activation then run abinit, it works fine)

this also makes me think; is this going to work when running an AiiDA code?
have you ever tried setting up a code to be run by an executable installed in a conda environment that first needs to be activated?

[chrisjsewell]

For reference this is what it adds to the Docker /root/.bashrc:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/root/.local/Miniconda3-py38_4.9.2-Linux-x86_64/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/root/.local/Miniconda3-py38_4.9.2-Linux-x86_64/etc/profile.d/conda.sh" ]; then
        . "/root/.local/Miniconda3-py38_4.9.2-Linux-x86_64/etc/profile.d/conda.sh"
    else
        export PATH="/root/.local/Miniconda3-py38_4.9.2-Linux-x86_64/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

[chrisjsewell]

this is what I cannot get to work

[chrisjsewell]

The other thing to note here, is that the size of the Conda folder is 1.3 Gb (with both abinit and cp2k envs installed, and after conda clean).
This will most likely increase the size of the Quantum Mobile VM, at least while there are still simulation codes and python environments that are not installed via conda (when there will be more possibility for codes to share dependencies)

It obviously vastly speeds up the build of the VM though; not having to run code compilations

[ltalirz]

To summarize our slack discussion: as you can verify with ldd, at least for the cp2k openmp and cp2k mpi executables from conda-forge, the dynamically linked libraries remain linked correctly even when the executables are called from outside the conda environment (I guess using a relative hardcoded rpath, didn't look into the mechanism further).

$ ldd /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/cp2k.ssmp
	linux-vdso.so.1 (0x00007fffbc6eb000)
	liblapack.so.3 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/liblapack.so.3 (0x000014775810f000)
	libfftw3.so.3 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libfftw3.so.3 (0x0000147757ed3000)
	libfftw3_omp.so.3 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libfftw3_omp.so.3 (0x000014775c4d4000)
	libxsmmf.so.1 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libxsmmf.so.1 (0x000014775c4c9000)
	libxsmm.so.1 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libxsmm.so.1 (0x0000147757b4e000)
	libgfortran.so.5 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libgfortran.so.5 (0x000014775c327000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00001477577b0000)
	libgomp.so.1 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libgomp.so.1 (0x000014775c2eb000)
	libgcc_s.so.1 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libgcc_s.so.1 (0x000014775779c000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000014775757d000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000014775718c000)
	/lib64/ld-linux-x86-64.so.2 (0x000014775c2b8000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x0000147756f84000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x0000147756d80000)
	libquadmath.so.0 => /home/ubuntu/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64/envs/aiida-zeopp/bin/../lib/libquadmath.so.0 (0x0000147756d46000)

When using these codes in AiiDA, it is therefore enough to use a prepend text

export PATH=<path/to/env/bin>:$PATH  # to get mpi wrapper + executable

and the codes run fine (no need to activate the conda environment).

[csadorf]

It's a hack, but you might be able to resolve this following the instructions shown here: conda/conda#7980 (comment)

[chrisjsewell]

Cheers, I'll check it out!

Note, as Leo mentions, technically it looks like you can use executables from the conda env, without activation. But obviously that is not ideal to stray from the intended use pattern, and it would be good to double check the canonical way to do this and understand any side-effects