Proposal: Volume refactor and external volume plugins

[calavera]

Extend the volume backend to accept configurations from an external process, aka plugin. It uses a pseudo rpc to get information from the plugin. Plugin discovery is performed by scanning the /usr/share/docker/plugins directory, looking for either a socket named like the volume driver specified or a file that contains the uri to the connection point.

This PR also includes a slim version of #13025 and crosbymichael#36.

TODO:

• More unit tests
• Integration tests
• Documentation

[wallnerryan]

considering storage drivers may do more then just provide a device, share or mount point, what about optional attributes that the driver may want like, performance requirements, failure domains, protection requirements like snapshots etc.?

[LK4D4]

But you talking about driver options itself, which should be handled on driver run, which isn't docker task.

[jvinod]

Would it make sense to extend the Create function to take in a set of opaque labels to be interpreted by the VolumeDriver. This would allow storage attributes (such as those mentioned by @wallnerryan) to be associated with the volume.

Create(name string, labels map[string]string) (err error)

[LK4D4]

Why is this public?

[LK4D4]

Oh, I see, this for restoring, right?

[calavera]

I think it doesn't need to be exported, I'll double check.

[calavera]

@jvinod I think it would be great if we can see a real example why you need those options for. I agree with @LK4D4 that volume configurations should be separated from usage, but I can change my mind if I understand better why you need them.

[cpuguy83]

@calavera e.g. requesting a volume of a specific size, minimum IOPS, etc.
These I would see as runtime requirements, and not driver init.

[nagendersoma]

@calavera the volume driver needs some context to configure perf and data mgmt (ex: snapshots) of the volumes. Having this information will help the driver make appropriate volume placements decisions to meet the SLOs. ex: choosing between ssd/sas/sata tiers for volume placement.

[icecrime]

Brace yourself: rebase is coming.

[wallnerryan]

@calavera +1 to @cpuguy83 & @nagendersoma. drivers may themselves understand these options differently, thus being able to pass this context allows the driver to make the decision during provision/request time for a volume.

[moxiegirl]

@icecrime Add a section in experimental.md file --- there is a template at the top. Link from that section to the details in the doc. Slack me if you need more help.

[moxiegirl]

LGTM

[icecrime]

As seen with @moxiegirl: docs content is there but needs to be properly "linked" from some experimental land page: we'll deal with this in a followup PR.

[icecrime]

#BOOM

[vaibhavkhanduja]

I am looking for documentation as it may be addressed somewhere. The issue of garbage collection i.e. as how storage is cleaned up in situations of errors and if such thing is required?

[moxiegirl]

@vaibhavkhanduja If yours is a general question, you should ask it in the Google Group or on IRC. You'll find information on how to get help with questions here: https://docs.docker.com/project/get-help/

This page in the documentation mentions garbage collection as it applies to volumes: https://docs.docker.com/userguide/dockervolumes/#data-volumes

[vaibhavkhanduja]

@moxiegirl .. my question is not a general ... in situations of failure or crashes are their any constructs to reclaim allocated storage? or it is responsibility of the volume driver to take care of gc?

[calavera]

it is responsibility of the volume driver to take care of gc?

@vaibhavkhanduja, yes, for now, it's responsibility of the volume driver, but we're open to suggestions.

[vaibhavkhanduja]

Volume driver seems to be the best place to implement this but I am wondering as how would it come to know state of the container? The volume api create seems to be passing just a name
+```
+{

• "Name": "volume_name"
  +}

this is suppose is not the docker container name considering there could be multiple volumes hooked into a container. Please correct me if I am wrong here?

Traditionally each mount point or file-system is associated with a unique number/entity name etc. The driver map of container run id and volume names could help in identifying which ones to be removed. Or a callback from docker to initiate GC or reclaim space when the container is removed (not stopped).

[cpuguy83]

@vaibhavkhanduja volume drivers get events for Create, Mount, Unmount, and Remove for each container that is using a given volume.
The name of a volume is globally addressable.

[vaibhavkhanduja]

@cpuguy83 not clear, can you elaborate more on the name? If I have a docker container with uuid how would know volumes associated with it? Does create volume api is called in context of a container passing id of the container? BTW I am talking about a GC, failure scenario ..

[cpuguy83]

@vaibhavkhanduja Please ask questions in docker-dev ML or #docker-dev on IRC, will be happy to address there.
Thanks.

[vaibhavkhanduja]

https://groups.google.com/forum/#!topic/docker-dev/swvOe6Qmrfw