Conda environment file not working for Singularity

[michael-sutherland]

Environment files are not working for Singularity. I get the error "EnvironmentFileNotFound: '/var/tmp/environment.yml' file not found". If I go into the generated ".def" file and change the location of the environment.yml file to be /environment.yml, it works. I'm not sure why, but the location /var/tmp does not work.

Minimum example:

"""
Conda Error Min Example

"""

from __future__ import absolute_import
from __future__ import unicode_literals
from __future__ import print_function

import hpccm

if __name__ == '__main__':

    ### Create Stage
    stage = hpccm.Stage()

    stage += hpccm.primitives.baseimage(image='nvidia/cuda:11.1-cudnn8-devel-centos8')

    ### Install Conda Python with environment
    stage += hpccm.building_blocks.conda(environment="environment.yml",
                                         eula=True,
                                         ospackages=['wget', 'ca-certificates', 'git'])

    ### Set container specification output format
    hpccm.config.set_container_format("singularity")

    ### Output container specification
    print(stage)

[samcmill]

Thanks for the issue report @michael-sutherland.

What version of Singularity are you using?

Can you please check whether /var/tmp/environment.yml exists on the host?

Some background: the reason HPCCM uses /var/tmp rather than /tmp is that Singularity mounts /tmp from the host into the container build process. Using /tmp would sometimes result in leftover files from the container build polluting the host /tmp or conflicts between paths with the same name. It looks like Singularity's behavior has changed and (at least as of 3.5.3) it also mounts /var/tmp from the host at build time.

[michael-sutherland]

I'm using singularity version 3.6.4-1.el8.

/var/tmp/environment.yml, does NOT exist on my host (although I see the miniconda installer among other files).

[samcmill]

@michael-sutherland can you please try #333?

Specifically, add hpccm.config.set_working_directory('/') to your script (before the conda building block).

Alternatively, add wd keyword argument to the conda building block:

    stage += hpccm.building_blocks.conda(environment="environment.yml",
                                         eula=True,
                                         ospackages=['wget', 'ca-certificates', 'git'],
                                         wd='/')

[VI-gha]

Another option could be to NOT use a separate environment.yml file but have all the conda channels and conda packages explicitly listed in the same building block using something like:

stage += hpccm.building_blocks.conda(channels=['conda-forge'], packages=['numpy'], eula=True, ospackages=['wget', 'ca-certificates', 'git'])

[samcmill]

Closing as fixed by #333. Please reopen if the issue persists.