New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Container Storage Interface (CSI) proposed standard #31923

Open
mgoelzer opened this Issue Mar 17, 2017 · 19 comments

Comments

Projects
None yet
@mgoelzer
Contributor

mgoelzer commented Mar 17, 2017

The Container Storage Interface (CSI) is a proposed new industry standard for cluster-wide volume plugins. This is a joint proposal from a group of us who work on Docker, Kubernetes, Mesosphere and Cloud Foundry. CSI is currently in the early draft stage, and we are seeking feedback from the community.

Draft spec here: https://docs.google.com/document/d/1JMNVNP-ZHz8cGlnqckOnpJmHF-DNY7IYP-Di7iuVhQI/edit

The goal of this standard is to have a single, cluster-level volumes plugin API that is shared by all orchestrators. So, for example, conformant storage plugins written for Docker would run unmodified in Kubernetes (and vice-versa).

The purpose of this issue is to ask the opinion of the Docker community on this proposed standard. Questions include:

  • Would/should Docker Engine adopt this as the cluster-aware evolution of our Docker Volumes API?
  • Should Docker suggest changes to the current draft before making that decision?
  • More broadly, does the Docker community see value in adopting an industry standard volumes API? Or is it preferable for Docker to have its own independent volumes API?

/cc @docker/core-swarmkit-maintainers @docker/core-engine-maintainers

@AkihiroSuda

This comment has been minimized.

Member

AkihiroSuda commented Mar 17, 2017

My thought at first glance

  • How to define FS type strings in portable way? Linux FS name? GPT GUID?
  • For stateful set (for both k8s and future swarm), we might need to add some "stable ID" to AttachRequest. This would be similar to swarm task slot number, but unlike slots, no two containers in the cluster should share identical ID.
    docker/swarmkit#1741 A storage might provide different views according to the stable ID
  • 👍 for snapshot API as well
@tiborvass

This comment has been minimized.

Collaborator

tiborvass commented Mar 17, 2017

  • Would/should Docker Engine adopt this as the cluster-aware evolution of our Docker Volumes API?

YES

  • Should Docker suggest changes to the current draft before making that decision?

We should make sure CSI works well for Docker.

  • More broadly, does the Docker community see value in adopting an industry standard volumes API? Or is it preferable for Docker to have its own independent volumes API?

NO for independent volumes API, I would much rather adopt CSI as-is, after making sure it works well. If it means we slowly deprecate the volume API then so be it.

@ibuildthecloud

This comment has been minimized.

Contributor

ibuildthecloud commented Mar 17, 2017

@mgoelzer One small bit of confusion. Is CSI a proposal that is being discussed and formed in a different forum? Or is this the first public mention of it? Are you looking for feedback on CSI? Or just the adoption of CSI in Docker?

@tiborvass

This comment has been minimized.

Collaborator

tiborvass commented Mar 17, 2017

@ibuildthecloud I believe the reason he's opening this issue, is so that everyone contributes to CSI and make sure that it works well for docker and its ecosystem. My understanding is that the link is just a draft and it needs your feedback. So that if we can come to agreement, then we would adopt CSI in one form (expose it as-is?) or another (internal detail).

@ibuildthecloud

This comment has been minimized.

Contributor

ibuildthecloud commented Mar 17, 2017

@tiborvass Awesome thanks, I'll provide my feedback. Really excited to see this cross community effort in the storage area.

@ibuildthecloud

This comment has been minimized.

Contributor

ibuildthecloud commented Mar 17, 2017

Would/should Docker Engine adopt this as the cluster-aware evolution of our Docker Volumes API?

Yes. Phase out volumes API over time.

Should Docker suggest changes to the current draft before making that decision?

I made plenty of comments in the Google Docs

More broadly, does the Docker community see value in adopting an industry standard volumes API? Or is it preferable for Docker to have its own independent volumes API?

Yep, Nope.

@mgoelzer

This comment has been minimized.

Contributor

mgoelzer commented Mar 17, 2017

@ibuildthecloud I should have been clearer about the origins of this. A group of about eight of us from Docker, Kubernetes, Mesosphere and Cloud Foundry basically just sat down in a room and drew up this proposal. Now we are seeking feedback from our respective open source communities to improve the proposal. It would be cool if some day it improves to the point where maintainers decide they want to adopt it, but I'm not asking anyone to adopt the first draft unmodified.

I made plenty of comments in the Google Docs

Thank you! Exactly what we need.

@mykaul

This comment has been minimized.

mykaul commented Mar 26, 2017

Would be interesting to compare to Cinder API (and support matrix[1]). I reckon many of Cinder's API is relevant here (apart from those 'from image' calls).
I'm quite sure Cinder has been through most of this discussion already, and while it may have some implementation challenges it tries to solve (active-active, ownership of the data, etc.) it already has most if not all the real life use cases of those calls already implemented, with security, etc. (which this draft ignores for the time being?).

[1] https://wiki.openstack.org/wiki/CinderSupportMatrix

@masonse

This comment has been minimized.

masonse commented Mar 26, 2017

I agree, Cinder is a well known API and using Cinder would eliminate the need for yet another API. I'm just referring to northbound API (what the application calls). What's southbound or under the covers, doesn't need to be Cinder.

@govint

This comment has been minimized.

govint commented Mar 28, 2017

@mgoelzer would a multi-tenant model be worth including (allowing different sets of users to create and use volumes within defined storage pools), volume statistics as supported by the plugin, volume services - ability to create volume snaps/clones, ability to define and apply storage profiles?

@cpuguy83

This comment has been minimized.

Contributor

cpuguy83 commented Mar 28, 2017

Just to be clear, the proposed API is the backend API for a cluster orchestrator to communicate with storage platforms, it has nothing to do with the frontend API that is exposed to users.

@e4jet

This comment has been minimized.

e4jet commented Mar 28, 2017

I've added some comments and provided an example workflow for provisioning. The document doesn't propose an order of operations, so I took swag at what they might be for provisioning.

@e4jet

This comment has been minimized.

e4jet commented Apr 18, 2017

Any chance of getting together to discus this live (this week) at Dockercon 2017?

@mgoelzer

This comment has been minimized.

Contributor

mgoelzer commented Apr 18, 2017

Yes, let's chat in person this week. mgoelzer@docker.com

@govint

This comment has been minimized.

govint commented Apr 18, 2017

@crosbymichael

This comment has been minimized.

Member

crosbymichael commented May 1, 2017

I just wanted to comment here first before adding anything to the document to make sure I'm not going to stay anything stupid, so please stop me if i'm wrong.

The name is "container storage interface" but it really has nothing to do with "container storage". Its node storage, attaching/detaching volumes to nodes and other operations above a container level. It makes it very confusing because when I read the title, then read the contents, there is not much container specific functionality in there at all. I expected it to be a spec about defining container storage and apis for providing storage to containers but that does not seem to be true.

Am I understanding this correctly? Why isn't it called something else that is catchy like "cloud native storage interface"?

@GiorgioRegni

This comment has been minimized.

GiorgioRegni commented May 1, 2017

So this is redundant with the existing volume plugin API? Is it a proposal for a replacement? https://docs.docker.com/engine/extend/plugins_volume/

@cpuguy83

This comment has been minimized.

Contributor

cpuguy83 commented May 1, 2017

@crosbymichael Correct, it's more about getting storage to a node.

@GiorgioRegni Yes and yes. The existing API will most likely be enhanced to support the kind of things needed for cluster storage (specifically attach/detach similar to CSI), in addition to adding support for the spec (if we all agree that's what we want).

@Vanuan

This comment has been minimized.

Vanuan commented Jun 6, 2018

Meanwhile, v0.3 has been released
https://github.com/container-storage-interface/spec/releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment