Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out-of-tree CSI Volume Plugins #178

Closed
saad-ali opened this issue Jan 24, 2017 · 49 comments
Closed

Out-of-tree CSI Volume Plugins #178

saad-ali opened this issue Jan 24, 2017 · 49 comments

Comments

@saad-ali
Copy link
Member

@saad-ali saad-ali commented Jan 24, 2017

  • One-line feature description (can be used as a release note): Add support for Out-of-Tree CSI Volume Plugins in Kubernetes
  • Primary contact (assignee): @saad-ali
  • Responsible SIGs: storage
  • Design proposal link (community repo): kubernetes/community#1258
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
    @childsb
    @jsafrane
  • Approver (likely from SIG/area to which feature belongs):
    @thockin
  • Feature target (which target equals to which milestone):
    • Alpha release target (x.y): 1.9
    • Beta release target (x.y): 1.10
    • Stable release target (x.y): 1.13

Background

Kubernetes volume plugins are currently all "in-tree" meaning that their source code is included in the main Kubernetes repo. All volume plugins are compiled and ship along with kubernetes binaries.

The drawback to this approach is that it requires third-party storage vendors wanting to support Kubernetes to commit code to the Kubernetes repo, and thus be locked into Kubernetes release schedules. It requires them to make their source code public/open-source.

While the Flex volume plugin already provides a mechanism for Plugin developers to experiment with out-of-tree plugins, it provides no guarantees of backwards compatibility (since it is alpha), and is completely exec based (driver installation requires ability to deploy files to specific locations on node and master machines).

This feature aims to create a new (or adopt an existing) API for Volume Plugins in Kubernetes that:

  • Will allow volume plugins to be developed out-of-tree.
  • Not require building volume plugins (or their dependencies) into Kubernetes binaries.
  • Not requiring direct machine access to deploy new volume plugins (drivers).

CC @kubernetes/sig-storage-feature-requests @kubernetes/sig-storage-proposals

@saad-ali saad-ali added this to the next-milestone milestone Jan 24, 2017
@saad-ali
Copy link
Member Author

@saad-ali saad-ali commented Jan 24, 2017

We aim to have a design doc for this by 1.6.

@calebamiles calebamiles added this to the v1.6 milestone Feb 7, 2017
@calebamiles calebamiles removed this from the next-milestone milestone Feb 7, 2017
@idvoretskyi
Copy link
Member

@idvoretskyi idvoretskyi commented Mar 8, 2017

@saad-ali any update on this feature? Docs and release notes are required (please, provide them to the features spreadsheet.

@saad-ali saad-ali removed this from the v1.6 milestone Mar 13, 2017
@saad-ali
Copy link
Member Author

@saad-ali saad-ali commented Mar 13, 2017

This does not need to be tracked for 1.6. It's design only. No bits shipping as part of 1.6. Removing 1.6 label.

@idvoretskyi idvoretskyi added this to the next-milestone milestone Mar 14, 2017
@idvoretskyi
Copy link
Member

@idvoretskyi idvoretskyi commented Mar 14, 2017

@saad-ali thank you for clarifying.

@jeffvance
Copy link
Contributor

@jeffvance jeffvance commented Mar 21, 2017

@saad-ali is the idea behind the 3rd bullet Not requiring direct machine access to deploy new volume plugins (drivers) to be able to pull plugin dependencies (eg. client libraries) from a defined registry or repo and push them to each node?

@krmayankk
Copy link
Contributor

@krmayankk krmayankk commented Mar 26, 2017

@saad-ali also its not clear, how is this different from out -of tree dynamic provisioners ?

@saad-ali saad-ali changed the title Out-of-tree Volume Plugins (CVI) Out-of-tree Volume Plugins (CSI) May 18, 2017
@lpabon
Copy link

@lpabon lpabon commented Jun 14, 2017

@saad-ali How does this compare with your CSI work on the CNCF ?

@saad-ali
Copy link
Member Author

@saad-ali saad-ali commented Jun 14, 2017

@saad-ali is the idea behind the 3rd bullet Not requiring direct machine access to deploy new volume plugins (drivers) to be able to pull plugin dependencies (eg. client libraries) from a defined registry or repo and push them to each node?

Yep!

@saad-ali also its not clear, how is this different from out -of tree dynamic provisioners ?

This feature (CSI) will enable completely out-of-tree volume plugins. Out-of-tree dynamic provisioners exist today as an alpha mechinism to enable volume provisioning (one functionality of volume plugins) to 1) be implmented out-of-tree, and 2) be decoupled from the rest of the volume plugin (potentially enabling multiple provisioners per volume type). Once Kubernetes supports CSI volume plugins, the only remaining benefit of Out-of-tree dynamic provisioners will be 2 (decoupled from the rest of the volume plugin, potentially enabling multiple provisioners per volume type). We can decide at that point if it makes sense to continue to support them or deprecate them.

@saad-ali How does this compare with your CSI work on the CNCF ?

CSI is the proposed protocol spec. The intention (or at least one of the intentions) of this feature is to implement that interface in Kubernetes.

@luxas
Copy link
Member

@luxas luxas commented Sep 1, 2017

@saad-ali Are you targeting alpha for v1.9? Is that information up-to-date?
Thanks!

@saad-ali saad-ali removed this from the next-milestone milestone Oct 20, 2017
@saad-ali saad-ali added this to the 1.9 milestone Oct 20, 2017
@saad-ali saad-ali changed the title Out-of-tree Volume Plugins (CSI) Out-of-tree CSI Volume Plugins Oct 23, 2017
@saad-ali
Copy link
Member Author

@saad-ali saad-ali commented Oct 23, 2017

@saad-ali Are you targeting alpha for v1.9? Is that information up-to-date?

Yes! All info is up to date.

@saad-ali
Copy link
Member Author

@saad-ali saad-ali commented Oct 23, 2017

This feature (CSI) will enable completely out-of-tree volume plugins. Out-of-tree dynamic provisioners exist today as an alpha mechinism to enable volume provisioning (one functionality of volume plugins) to 1) be implmented out-of-tree, and 2) be decoupled from the rest of the volume plugin (potentially enabling multiple provisioners per volume type). Once Kubernetes supports CSI volume plugins, the only remaining benefit of Out-of-tree dynamic provisioners will be 2 (decoupled from the rest of the volume plugin, potentially enabling multiple provisioners per volume type). We can decide at that point if it makes sense to continue to support them or deprecate them.

An update on this: we've incorporated external-provisioners in to the CSI design!

@govint
Copy link

@govint govint commented Oct 24, 2017

@saad-ali can you point to the CSI design that has the external provisioners incorporated. Is it here kubernetes/community#1258?

@luxas
Copy link
Member

@luxas luxas commented Oct 24, 2017

@saad-ali Will this work be in v1.9? I assume as alpha as the initial step.

@kokhang
Copy link

@kokhang kokhang commented Nov 9, 2017

@luxas @saad-ali does "alpha" mean that csi would need to be enabled via a feature gate like all the other alpha features?

Also the CSI plugin depends on other features, which are currently in alpha, like mount-propagation and perhaps containerized-mount? Will these be promoted to beta? or does this mean that in order to consume the CSI plugin, these features have to be enabled via the feature gates?

@AishSundar
Copy link

@AishSundar AishSundar commented Oct 17, 2018

@saad-ali whats left of this work to go to GA in 1.13? Are kubernetes/kubernetes#69690 and kubernetes/kubernetes#69688 the only 2 pending PRs for this feature?

@jimangel
Copy link
Member

@jimangel jimangel commented Oct 17, 2018

@saad-ali when this goes GA will there need to be any doc changes (1.13)?

@jimangel
Copy link
Member

@jimangel jimangel commented Nov 5, 2018

@saad-ali will there be any updates to the docs necessary for 1.13? The deadline for placeholder PRs for the 1.13 release is November 8. So it's important to make a docs PR as soon as possible if needed. Thanks! cc @idvoretskyi @AishSundar @tfogo

@claurence
Copy link

@claurence claurence commented Nov 5, 2018

Hi @saad-ali I'm an enhancements shadow checking in on how this issue is tracking. Code slush is on 11/9 and code freeze is coming up on 11/15 do you have a status update on the likelihood that this will make the the code freeze date?

@msau42
Copy link
Member

@msau42 msau42 commented Nov 6, 2018

Tasks for 1.13:

We are still on track for 1.13, although timeline will be very tight. The longest pull item is moving the CSI spec to 1.0. All other tasks are close to being merged.

@AishSundar
Copy link

@AishSundar AishSundar commented Nov 12, 2018

@saad-ali @msau42 I understand you are working actively on getting this to GA. I see the 2 open PRs are close, but is there a tracking issue for the 1.0 spec?

Can either of you attend Release burndown meeting (Zoom: https://zoom.us/j/611312756) this Wedsnesday (11/14)? We would like the discuss the latest update Go/No-GO for 1.13. Thanks

@saad-ali
Copy link
Member Author

@saad-ali saad-ali commented Nov 14, 2018

CSI 1.0.0-rc1 Spec was cut on Monday (https://github.com/container-storage-interface/spec/releases/tag/v1.0.0-rc1). We are working to pick it up in Kubernetes (kubernetes/kubernetes#71020). @msau42 will attend the burn down.

@kacole2
Copy link
Member

@kacole2 kacole2 commented Nov 16, 2018

/reopen

@k8s-ci-robot
Copy link
Contributor

@k8s-ci-robot k8s-ci-robot commented Nov 16, 2018

@kacole2: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot reopened this Nov 16, 2018
@spiffxp
Copy link
Member

@spiffxp spiffxp commented Jan 10, 2019

@saad-ali @kacole2 What remains to be done here? I think this shipped as GA in 1.13 and can be closed

@msau42
Copy link
Member

@msau42 msau42 commented Jan 10, 2019

We have some post GA work, like updating the in-tree controllers to use the new v1 objects.

@kacole2
Copy link
Member

@kacole2 kacole2 commented Jan 10, 2019

@msau42 but is that "enhancement" related? or is that more updating?

@msau42
Copy link
Member

@msau42 msau42 commented Jan 10, 2019

It's cleanup/refactoring work that should not have any user-visible effects. So I guess this can be closed.

@claurence
Copy link

@claurence claurence commented Jan 15, 2019

@msau42 @saad-ali Hello - I'm the enhancements lead for 1.14 - it sounds like there is no work to be tracked for the 1.14 release so I can remove it from being tracked but let me know if that is an incorrect interpretation. Thanks

@msau42
Copy link
Member

@msau42 msau42 commented Jan 15, 2019

We can close this and open up issues for remaining cleanup work. There are already individual enhancement issues opened for csi alpha features
/close

@k8s-ci-robot
Copy link
Contributor

@k8s-ci-robot k8s-ci-robot commented Jan 15, 2019

@msau42: Closing this issue.

In response to this:

We can close this and open up issues for remaining cleanup work. There are already individual enhancement issues opened for csi alpha features
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ingvagabund pushed a commit to ingvagabund/enhancements that referenced this issue Apr 2, 2020
user-workload-monitoring: be more accurate about alerts & silences enpoints
tamalsaha pushed a commit to kmodules/airgap that referenced this issue Jun 24, 2021
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Introduce new `VolumeAttachment` API Object

**What this PR does / why we need it**:

Introduce a new `VolumeAttachment` API Object. This object will be used by the CSI volume plugin to enable external attachers (see design [here](kubernetes/community#1258). In the future, existing volume plugins can be refactored to use this object as well.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:  Part of issue kubernetes/enhancements#178

**Special notes for your reviewer**:
None

**Release note**:

```release-note
NONE
```

Kubernetes-commit: ebe8ea73fd1a961779242dfbb629befa153e96fc
tamalsaha pushed a commit to kmodules/airgap that referenced this issue Jun 24, 2021
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Introduce new `VolumeAttachment` API Object

**What this PR does / why we need it**:

Introduce a new `VolumeAttachment` API Object. This object will be used by the CSI volume plugin to enable external attachers (see design [here](kubernetes/community#1258). In the future, existing volume plugins can be refactored to use this object as well.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:  Part of issue kubernetes/enhancements#178

**Special notes for your reviewer**:
None

**Release note**:

```release-note
NONE
```

Kubernetes-commit: ebe8ea73fd1a961779242dfbb629befa153e96fc
tamalsaha pushed a commit to kmodules/airgap that referenced this issue Jun 24, 2021
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Introduce new `VolumeAttachment` API Object

**What this PR does / why we need it**:

Introduce a new `VolumeAttachment` API Object. This object will be used by the CSI volume plugin to enable external attachers (see design [here](kubernetes/community#1258). In the future, existing volume plugins can be refactored to use this object as well.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:  Part of issue kubernetes/enhancements#178

**Special notes for your reviewer**:
None

**Release note**:

```release-note
NONE
```

Kubernetes-commit: ebe8ea73fd1a961779242dfbb629befa153e96fc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment