mount gluster volume in docker service

[olafbuitelaar]

• Plugin version (or commit ref) : latest
  docker plugin ls
  ID NAME DESCRIPTION ENABLED
  96d24f765378 sapk/plugin-gluster:latest GlusterFS plugin for Docker true
  (Digest: sha256:776c684bad6b9d8600444c21ad40dd3e8d9987008f5a6ddc21f3a7379cf4248d)

• Docker version : 17.12.0-ce
  docker info
  Containers: 10
  Running: 1
  Paused: 0
  Stopped: 9
  Images: 12
  Server Version: 17.12.0-ce
  Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  Logging Driver: json-file
  Cgroup Driver: cgroupfs
  Plugins:
  Volume: local
  Network: bridge host macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
  Swarm: active
  NodeID: 78d8gy48o8d2oa8r16f4oxlsl
  Is Manager: true
  ClusterID: z85ihxm7afsllcxaiitkr1ne8
  Managers: 1
  Nodes: 1
  Orchestration:
  Task History Retention Limit: 5
  Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
  Dispatcher:
  Heartbeat Period: 5 seconds
  CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.0.45
  Manager Addresses:
  192.168.0.45:2377
  Runtimes: runc
  Default Runtime: runc
  Init Binary: docker-init
  containerd version: 89623f28b87a6004d4b785663257362d1658a729
  runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
  init version: 949e6fa
  Security Options:
  seccomp
  Profile: default
  Kernel Version: 3.10.0-693.11.1.el7.x86_64
  Operating System: CentOS Linux 7 (Core)
  OSType: linux
  Architecture: x86_64
  CPUs: 2
  Total Memory: 992.3MiB
  Name: lease-01.dc01.adsolutions
  ID: IHRB:OXLB:FH7Q:KAYH:QVP6:73N5:SVWP:S34P:V3CT:WMSY:3UG2:NBEP
  Docker Root Dir: /var/lib/docker
  Debug Mode (client): false
  Debug Mode (server): false
  Registry: https://index.docker.io/v1/
  Labels:
  Experimental: false
  Insecure Registries:
  127.0.0.0/8
  Live Restore Enabled: false

WARNING: bridge-nf-call-ip6tables is disabled

• Plugin type : managed
• Operating system: CentOS 7 3.10.0-693.11.1.el7.x86_64

Description

I'm trying to run the plugin as service:
docker service create --replicas 1 --name testing --mount type=volume,volume-opt="voluri=10.0.0.1:/docker",volume-driver=sapk/plugin-gluster,dst=/testing registry
(as side note adding multiple hosts as in: voluri=10.0.0.1,10.0.0.2:/docker seems to fail the docker cli to fail parsing on the comma (,)).

running a local the same command as local docker image works fine;
docker volume create --driver sapk/plugin-gluster --opt voluri="10.0.0.1,10.0.0.2:/docker" --name test
docker run -v test:/testing --rm -ti registry

I've also tried accessing the local volume via the service via;
docker service create --replicas 1 --name testing --mount type=volume,src=test,volume-driver=sapk/plugin-gluster,dst=/testing registry

Logs

The logs for creating the local docker container and the service look about the same;
Entering go-plugins-helpers mountPath" plugin=35970f8dcba65e81c467a5ded9daff8998c501cc042317676d4e81517e1357a2
"level=debug msg="Entering Mount: &{docker 4d7acdf887dea92f3e0290d6c5832018eaa2c9a70a958cff1626fa1a8f23e97f}"" plugin=35970f8dcba65e81c467a5ded9daff8998c501cc042317676d4e81517e1357a2
"level=debug msg="Entering MountExist: name: docker"" plugin=35970f8dcba65e81c467a5ded9daff8998c501cc042317676d4e81517e1357a2
"level=debug msg="Volume found: &{ docker %!s(int=0)}"" plugin=35970f8dcba65e81c467a5ded9daff8998c501cc042317676d4e81517e1357a2
"level=debug msg="Mount found: &{/var/lib/docker-volumes/gluster/docker %!s(int=0)}"" plugin=35970f8dcba65e81c467a5ded9daff8998c501cc042317676d4e81517e1357a2

Any clue why services refuse to mount?

[sapk]

I haven't try it lately in swarm but it should work.
What I can recommand you is to use a docker-compose file to define the volume more easely and do a docker stack deploy -c docker-compose.yml my-stack and make sure the the ip is unique and accesible inside swarm service.

[olafbuitelaar]

Thanks sapk, that seems to work.