Uninstall only removes modules when containers in distinct path

[marcodelapierre]

Describe the bug
As per description

To Reproduce
[update]: custom, distinct container_root in the shpc config

md@zeus-1:singularity-hpc$ shpc list autamus
        ghcr.io/autamus/beast2:2.6.4

md@zeus-1:singularity-hpc$ shpc uninstall ghcr.io/autamus/beast2:2.6.4
Are you sure you want to uninstall $module_base/ghcr.io/autamus/beast2/2.6.4, and all content below it? y

$module_base/ghcr.io/autamus/beast2/2.6.4 and all subdirectories been removed.
md@zeus-1:singularity-hpc$ ls containers/ghcr.io/autamus/beast2/2.6.4/
ghcr.io-autamus-beast2-2.6.4-sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif

Expected behavior
Both container and module are removed.

Version of Singularity and Singularity Registry HPC Client
Singularity 3.7.4
Latest SHPC as of 11 Aug 2021.

**Anything else?

[vsoch]

I think you are leaving out information - like having different directories?

[marcodelapierre]

Ooops sorry, indeed I forgot to mention that I have a custom container_root, distinct from module_root!
(I have also updated the description above accordingly)

[vsoch]

ah that's something new to try! So I have a custom config in my home with a custom container_base (not container_root, right?)
I set it to a random directory /tmp/containers:

$ shpc config get container_base
container_base                 /tmp/containers

now I'll install the module (default singularity and lmod). This shows that the container is pulled to /tmp/containers

$ shpc install  ghcr.io/autamus/beast2:2.6.4
singularity pull --name /tmp/containers/ghcr.io/autamus/beast2/2.6.4/ghcr.io-autamus-beast2-2.6.4-sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif docker://ghcr.io/autamus/beast2@sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d
INFO:    Using cached SIF image
INFO:    Using cached SIF image
/tmp/containers/ghcr.io/autamus/beast2/2.6.4/ghcr.io-autamus-beast2-2.6.4-sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif
Module ghcr.io/autamus/beast2:2.6.4 was created.

There it is! and the module is still in modules in the present working directory:

$ ls /tmp/containers/ghcr.io/autamus/beast2/2.6.4/ghcr.io-autamus-beast2-2.6.4-sha256\:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif 
/tmp/containers/ghcr.io/autamus/beast2/2.6.4/ghcr.io-autamus-beast2-2.6.4-sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif
(env) (base) vanessa@vanessa-ThinkPad-T490s:~/Desktop/Code/singularity-hpc$ ls modules/ghcr.io/autamus/beast2/2.6.4/
99-shpc.sh  module.lua

so now I'll list to see it

$ shpc list
        ghcr.io/autamus/beast2:2.6.4

Then I uninstall:

$ shpc uninstall ghcr.io/autamus/beast2:2.6.4
Are you sure you want to uninstall $container_base/ghcr.io/autamus/beast2/2.6.4, and all content below it? y
$container_base/ghcr.io/autamus/beast2/2.6.4 and all subdirectories been removed.
Are you sure you want to uninstall $module_base/ghcr.io/autamus/beast2/2.6.4, and all content below it? y
$module_base/ghcr.io/autamus/beast2/2.6.4 and all subdirectories been removed.

The top level folder for beast is there, but the version folder is gone:

$ ls /tmp/containers/ghcr.io/autamus/beast2/

and the module files are removed:

$ ls modules/ghcr.io/autamus/beast2/

shpc doesn't see that it is installed:

$ shpc list beast
You don't have any install modules. Try shpc show.

As for why your sif isn't removed, I can't reproduce. Can you try doing this from the main branch? I have a quick bugfix I want to do (not pushed / PRed yet) but it's not related to this issue. I wanted to address these before opening a PR.

[marcodelapierre]

Let's give it another go:

• latest commit as of 13 August 2021 (2ac8fe4617ea3a4c9385459d864ec1f49b260127)
• only changes to config

   shpc config set container_base:\$root_dir/containers
   shpc config set singularity_module:singularity
  

md@zeus-1:singularity-hpc$ shpc install ghcr.io/autamus/beast2:2.6.4
singularity pull --name /group/pawsey0001/mdelapierre/PSS/shpc-tests/test-13aug21/singularity-hpc/containers/ghcr.io/autamus/beast2/2.6.4/ghcr.io-autamus-beast2-2.6.4-sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif docker://ghcr.io/autamus/beast2@sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d
INFO:    Using cached SIF image
INFO:    Using cached SIF image
/group/pawsey0001/mdelapierre/PSS/shpc-tests/test-13aug21/singularity-hpc/containers/ghcr.io/autamus/beast2/2.6.4/ghcr.io-autamus-beast2-2.6.4-sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif
Module ghcr.io/autamus/beast2:2.6.4 was created.

md@zeus-1:singularity-hpc$ shpc list
        ghcr.io/autamus/beast2:2.6.4
md@zeus-1:singularity-hpc$ ls modules/ghcr.io/autamus/beast2/2.6.4/
99-shpc.sh  module.lua
md@zeus-1:singularity-hpc$ ls containers/ghcr.io/autamus/beast2/2.6.4/
ghcr.io-autamus-beast2-2.6.4-sha256:2c1b921b0c1e30e658a27cc22a7082f3367b508c964259f32310626b8b74cd0d.sif

md@zeus-1:singularity-hpc$ shpc uninstall ghcr.io/autamus/beast2:2.6.4
Are you sure you want to uninstall $container_base/ghcr.io/autamus/beast2/2.6.4, and all content below it? y
$container_base/ghcr.io/autamus/beast2/2.6.4 and all subdirectories been removed.
Are you sure you want to uninstall $module_base/ghcr.io/autamus/beast2/2.6.4, and all content below it? y
$module_base/ghcr.io/autamus/beast2/2.6.4 and all subdirectories been removed.

md@zeus-1:singularity-hpc$ ls modules/ghcr.io/autamus/beast2/
md@zeus-1:singularity-hpc$ ls containers/ghcr.io/autamus/beast2/

Ah! with this fresh installation it works!

Must have done something silly the other day -- apol for the issue -- closing now!