Proposal: Container Storage Interface (CSI) proposed standard

[mgoelzer]

The Container Storage Interface (CSI) is a proposed new industry standard for cluster-wide volume plugins. This is a joint proposal from a group of us who work on Docker, Kubernetes, Mesosphere and Cloud Foundry. CSI is currently in the early draft stage, and we are seeking feedback from the community.

Draft spec here: https://docs.google.com/document/d/1JMNVNP-ZHz8cGlnqckOnpJmHF-DNY7IYP-Di7iuVhQI/edit

The goal of this standard is to have a single, cluster-level volumes plugin API that is shared by all orchestrators. So, for example, conformant storage plugins written for Docker would run unmodified in Kubernetes (and vice-versa).

The purpose of this issue is to ask the opinion of the Docker community on this proposed standard. Questions include:

• Would/should Docker Engine adopt this as the cluster-aware evolution of our Docker Volumes API?
• Should Docker suggest changes to the current draft before making that decision?
• More broadly, does the Docker community see value in adopting an industry standard volumes API? Or is it preferable for Docker to have its own independent volumes API?

/cc @docker/core-swarmkit-maintainers @docker/core-engine-maintainers

[AkihiroSuda]

My thought at first glance

• How to define FS type strings in portable way? Linux FS name? GPT GUID?
• For stateful set (for both k8s and future swarm), we might need to add some "stable ID" to AttachRequest. This would be similar to swarm task slot number, but unlike slots, no two containers in the cluster should share identical ID.
  docker/swarmkit#1741 A storage might provide different views according to the stable ID
• 👍 for snapshot API as well

[tiborvass]

• Would/should Docker Engine adopt this as the cluster-aware evolution of our Docker Volumes API?

YES

• Should Docker suggest changes to the current draft before making that decision?

We should make sure CSI works well for Docker.

• More broadly, does the Docker community see value in adopting an industry standard volumes API? Or is it preferable for Docker to have its own independent volumes API?

NO for independent volumes API, I would much rather adopt CSI as-is, after making sure it works well. If it means we slowly deprecate the volume API then so be it.

[ibuildthecloud]

@mgoelzer One small bit of confusion. Is CSI a proposal that is being discussed and formed in a different forum? Or is this the first public mention of it? Are you looking for feedback on CSI? Or just the adoption of CSI in Docker?

[tiborvass]

@ibuildthecloud I believe the reason he's opening this issue, is so that everyone contributes to CSI and make sure that it works well for docker and its ecosystem. My understanding is that the link is just a draft and it needs your feedback. So that if we can come to agreement, then we would adopt CSI in one form (expose it as-is?) or another (internal detail).

[ibuildthecloud]

@tiborvass Awesome thanks, I'll provide my feedback. Really excited to see this cross community effort in the storage area.

[ibuildthecloud]

Would/should Docker Engine adopt this as the cluster-aware evolution of our Docker Volumes API?

Yes. Phase out volumes API over time.

Should Docker suggest changes to the current draft before making that decision?

I made plenty of comments in the Google Docs

More broadly, does the Docker community see value in adopting an industry standard volumes API? Or is it preferable for Docker to have its own independent volumes API?

Yep, Nope.

[mgoelzer]

@ibuildthecloud I should have been clearer about the origins of this. A group of about eight of us from Docker, Kubernetes, Mesosphere and Cloud Foundry basically just sat down in a room and drew up this proposal. Now we are seeking feedback from our respective open source communities to improve the proposal. It would be cool if some day it improves to the point where maintainers decide they want to adopt it, but I'm not asking anyone to adopt the first draft unmodified.

I made plenty of comments in the Google Docs

Thank you! Exactly what we need.

[mykaul]

Would be interesting to compare to Cinder API (and support matrix[1]). I reckon many of Cinder's API is relevant here (apart from those 'from image' calls).
I'm quite sure Cinder has been through most of this discussion already, and while it may have some implementation challenges it tries to solve (active-active, ownership of the data, etc.) it already has most if not all the real life use cases of those calls already implemented, with security, etc. (which this draft ignores for the time being?).

[1] https://wiki.openstack.org/wiki/CinderSupportMatrix

[masonse]

I agree, Cinder is a well known API and using Cinder would eliminate the need for yet another API. I'm just referring to northbound API (what the application calls). What's southbound or under the covers, doesn't need to be Cinder.

[govint]

@mgoelzer would a multi-tenant model be worth including (allowing different sets of users to create and use volumes within defined storage pools), volume statistics as supported by the plugin, volume services - ability to create volume snaps/clones, ability to define and apply storage profiles?

[cpuguy83]

Just to be clear, the proposed API is the backend API for a cluster orchestrator to communicate with storage platforms, it has nothing to do with the frontend API that is exposed to users.

[e4jet]

I've added some comments and provided an example workflow for provisioning. The document doesn't propose an order of operations, so I took swag at what they might be for provisioning.

[e4jet]

Any chance of getting together to discus this live (this week) at Dockercon 2017?

[mgoelzer]

Yes, let's chat in person this week. mgoelzer@docker.com

[govint]

Please let know where the meeting may be held.

…

On Apr 18, 2017 12:34 PM, "Mike Goelzer" ***@***.***> wrote: Yes, let's chat in person this week. ***@***.*** — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#31923 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APHseLv-YwCpDrhd288HGf01IIC2P5gEks5rxPQ_gaJpZM4Mg1Rz> .

[crosbymichael]

I just wanted to comment here first before adding anything to the document to make sure I'm not going to stay anything stupid, so please stop me if i'm wrong.

The name is "container storage interface" but it really has nothing to do with "container storage". Its node storage, attaching/detaching volumes to nodes and other operations above a container level. It makes it very confusing because when I read the title, then read the contents, there is not much container specific functionality in there at all. I expected it to be a spec about defining container storage and apis for providing storage to containers but that does not seem to be true.

Am I understanding this correctly? Why isn't it called something else that is catchy like "cloud native storage interface"?

[GiorgioRegni]

So this is redundant with the existing volume plugin API? Is it a proposal for a replacement? https://docs.docker.com/engine/extend/plugins_volume/

[cpuguy83]

@crosbymichael Correct, it's more about getting storage to a node.

@GiorgioRegni Yes and yes. The existing API will most likely be enhanced to support the kind of things needed for cluster storage (specifically attach/detach similar to CSI), in addition to adding support for the spec (if we all agree that's what we want).

[Vanuan]

Meanwhile, v0.3 has been released
https://github.com/container-storage-interface/spec/releases