What if storage goes down ?

[mlmarius]

Hi.

I am testing the plugin and when the storage provider goes down, the volume inside the docker container disappears but it never reappears if the storage provider comes back up. How can this be handled ?

[vieux]

@mlmarius do you have a step by step way to reproduce ?

[mlmarius]

I am running this stack on a 2 node swarm:

version: "3"
services:
    bash:
        image: bash
        entrypoint: /usr/local/bin/bash
        volumes:
            - sshvol1:/sshvol1
        deploy:
            mode: global

        stdin_open: true
        tty: true


volumes:
    sshvol1:
        driver: vieux/sshfs
        driver_opts:
            sshcmd: user@ssh-host:/path/to/sshvol

Then I briefly stop ssh-host and restart it. When ssh-host goes down the volume is unmounted from the containers but when ssh-host comes back up or is reachable again the volume does not reappear on the containers.

[xelra]

It's quite reasonable to expect a behavior like this. SSH doesn't bring the link up automatically again.

It would be great if the container would use AutoFS or systemd to bring up the SSHFS connection and restore it on disconnect.

[victort]

additionally, consider the -o reconnect sshfs option when mounting your volume, which appears to have reconnected my destination after a reboot.

[alexanderkjeldaas]

@vieux where is the -o reconnect option supposed to go?

[vieux]

@alexanderkjeldaas docker volume create -d vieux/sshfs -o sshcmd=<user@host:path> -o reconnect <volume-name>

[victort]

Yeah, so far I've gotten the mount to restore on reboots with it. However I have not tested its resiliency further than that. so YMMV.

[xelra]

@victort For perfect resilience against network issues, the container would need to use AutoFS to mount the SSHFS.
I'm using AutoFS on my root servers to automatically mount and keep remote mounts alive. The same can be achieved with systemd and fstab, but at least on a server that has the drawback that the system cannot boot if the remote location isn't available during boot. I'm unsure how a Docker container would be affected by this, but at least on servers AutoFS is the superior solution to this problem.

[jamiejackson]

@xelra (or anybody else), I'm in the process of containerizing my stack, and one part of that stack was an autofs ssh mount. Am I going to have to continue to do that on the docker host and bind mount the host mount point into the container, or is there a better way?

[xelra]

@jamiejackson There's nothing really special about the container. SSHFS will drop its connection and the only way to reliably keep it alive or mount on demand is with AutoFS.

I'm not 100% sure what you're asking, but if you're asking whether you should include AutoFS, then the answer is yes.
And please share or PR if you base your container off this project.