Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot / Restore Volume Support for Kubernetes (CRD + External Controller) #177

Open
jingxu97 opened this Issue Jan 23, 2017 · 44 comments

Comments

@jingxu97
Copy link

jingxu97 commented Jan 23, 2017

Feature Description

  • One-line feature description (can be used as a release note):
Snapshot / restore functionality for Kubernetes and CSI. This provides standardized APIs design (CRDs) and adds PV snapshot/restore support for CSI volume drivers.

Old description:

Expose the ability in the Kubernetes API to create,list, delete, and restore snapshots from an arbitrary underlying storage systems that support it.

@jingxu97 jingxu97 added this to the next-milestone milestone Jan 23, 2017

@mdelio mdelio added the sig/storage label Jan 24, 2017

@davidopp

This comment has been minimized.

@timothysc

This comment has been minimized.

Copy link
Member

timothysc commented Apr 17, 2017

/cc @skriss

@mdelio

This comment has been minimized.

Copy link

mdelio commented May 4, 2017

@jingxu97 I think we're going to have something in alpha for 1.7, can we can we please set the milestone to 1.7?

@calebamiles calebamiles modified the milestones: v1.7, next-milestone May 4, 2017

@calebamiles

This comment has been minimized.

Copy link
Member

calebamiles commented May 4, 2017

@kubernetes/sig-storage-feature-requests could someone please update the issue description to the new template. Thanks!

@jingxu97

This comment has been minimized.

Copy link
Author

jingxu97 commented May 4, 2017

@alkar

This comment has been minimized.

@jingxu97

This comment has been minimized.

Copy link
Author

jingxu97 commented May 9, 2017

@alkar

This comment has been minimized.

Copy link

alkar commented May 10, 2017

@jingxu97 thanks! I can't access it but I just requested access.

@yanivlavi

This comment has been minimized.

Copy link

yanivlavi commented May 18, 2017

@jingxu97 I wanted to add an important note here regarding overall approach. I'm not this is the right place to put this, but I would be a happy to be guided to the right forum to bring this up.

Coming from a long background in oVirt and OSP I feel there is an important aspect that needs to be discuss which is ownership of the state of snapshots (and overall volume metadata) and discoverablility of the this metadata.

In cloud and on premise users (developers and admins) might prefer the native storage APIs over the Kube APIs. Kube is also not the one creating the snapshot, it is the cloud service or the storage itself, therefore it is not the owner of this metadata.

In OSP they have already made this mistake with Cinder that forces users that want to use snapshots to go through the Cinder API to have it be available in the OSP environment. Cinder doesn't know snapshot created on the storage and if you lose the Cinder you lose everything as the metadata that counts is in his stateful DB.

What I'm trying to say is that it is important that Kube doesn't try to be the owner of the volume metadata, which means things like periodically checking if a new snapshot was created directly via the storage service API or that the ID used for the snapshot is the storage service snapshot ID.

It is clear that Kube should expose snapshotting since it very needed to the container use case, but it is very important it doesn't become a storage abstraction service like Cinder. We do not want to chase the storage service features or limit user to use the Kube API, it should be a option. We also want to allow discovery of volumes no matter where they where created.

As we get to more complex features like QOS and oversubscription for example, we want to allow exposing and reusing the storage service capabilities, not replace them or block users from using them via the cloud service or storage management APIs.

@idvoretskyi

This comment has been minimized.

Copy link
Member

idvoretskyi commented May 18, 2017

@jingxu97 any progress on the feature description? @kubernetes/sig-storage-feature-requests

@idvoretskyi idvoretskyi added this to Action required in Kubernetes 1.7 features May 18, 2017

@idvoretskyi

This comment has been minimized.

Copy link
Member

idvoretskyi commented Jun 14, 2017

@mdelio @jingxu97 please, update the feature description with the new template - https://github.com/kubernetes/features/blob/master/ISSUE_TEMPLATE.md

@saad-ali saad-ali modified the milestones: next-milestone, v1.7 Jun 15, 2017

@saad-ali

This comment has been minimized.

Copy link
Member

saad-ali commented Jun 15, 2017

@idvoretskyi I updated the original comment with the new template. This feature is not actually shipping any bits in the Kubernetes core for v1.7, so therefore I moved it to the next-milestone. This means that it will not need documentation, etc. for 1.7 in the Kubernetes core. I will remove the feature from the 1.7 tracking board as well.

@idvoretskyi

This comment has been minimized.

Copy link
Member

idvoretskyi commented Sep 5, 2017

@saad-ali any updates for 1.8? Is this feature still on track for the release?

@rootfs

This comment has been minimized.

Copy link
Member

rootfs commented Sep 5, 2017

@childsb

This comment has been minimized.

Copy link
Member

childsb commented Sep 7, 2017

@jdumars

This comment has been minimized.

Copy link
Member

jdumars commented Sep 15, 2017

@jingxu97 @rootfs any update on missing docs for this? PR is due today.

@justaugustus

This comment has been minimized.

Copy link
Member

justaugustus commented Jul 28, 2018

/assign @xing-yang

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jul 28, 2018

@justaugustus: GitHub didn't allow me to assign the following users: xing-yang.

Note that only kubernetes members and repo collaborators can be assigned.
For more information please see the contributor guide

In response to this:

/assign @xing-yang

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@xing-yang

This comment has been minimized.

Copy link

xing-yang commented Jul 30, 2018

@justaugustus "Alpha release target 1.12..." is correct. The prototype snapshot support in "External repo" was added in 1.8.

@zparnold

This comment has been minimized.

Copy link
Member

zparnold commented Aug 20, 2018

Hey there! @jingxu97 I'm the wrangler for the Docs this release. Is there any chance I could have you open up a docs PR against the release-1.12 branch as a placeholder? That gives us more confidence in the feature shipping in this release and gives me something to work with when we start doing reviews/edits. Thanks! If this feature does not require docs, could you please update the features tracking spreadsheet to reflect it?

@xing-yang

This comment has been minimized.

Copy link

xing-yang commented Aug 21, 2018

@zparnold Added a placeholder doc PR kubernetes/website#9948

@zparnold

This comment has been minimized.

Copy link
Member

zparnold commented Aug 25, 2018

k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this issue Aug 29, 2018

Kubernetes Submit Queue
Merge pull request #67087 from xing-yang/datasource
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Add DataSource and TypedLocalObjectReference

**What this PR does / why we need it**:
This PR adds TypedLocalObjectReference in the core API and adds DataSource in PersistentVolumeClaimSpec.

It also enables feature gate for VolumeSnapshotDataSource.

This is part of the CSI snapshot design proposal to support restoring a volume from a snapshot: 
kubernetes/community#2495

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
kubernetes/enhancements#177

**Special notes for your reviewer**:

**Release note**:

```release-note
Added support to restore a volume from a volume snapshot data source. 
```

k8s-publishing-bot added a commit to kubernetes/api that referenced this issue Aug 29, 2018

Merge pull request #67087 from xing-yang/datasource
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Add DataSource and TypedLocalObjectReference

**What this PR does / why we need it**:
This PR adds TypedLocalObjectReference in the core API and adds DataSource in PersistentVolumeClaimSpec.

It also enables feature gate for VolumeSnapshotDataSource.

This is part of the CSI snapshot design proposal to support restoring a volume from a snapshot: 
kubernetes/community#2495

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
kubernetes/enhancements#177

**Special notes for your reviewer**:

**Release note**:

```release-note
Added support to restore a volume from a volume snapshot data source. 
```

Kubernetes-commit: d97ece0f36578b3af1c9ce275d184d711aa29baf
@justaugustus

This comment has been minimized.

Copy link
Member

justaugustus commented Sep 5, 2018

@jingxu97 @xing-yang --
Any update on docs status for this feature? Are we still planning to land it for 1.12?
At this point, code freeze is upon us, and docs are due on 9/7 (2 days).
If we don't here anything back regarding this feature ASAP, we'll need to remove it from the milestone.

cc: @zparnold @jimangel @tfogo

@xing-yang

This comment has been minimized.

Copy link

xing-yang commented Sep 5, 2018

@justaugustus Code was merged. Doc PR is ready for review: kubernetes/website#9948

@justaugustus

This comment has been minimized.

Copy link
Member

justaugustus commented Sep 6, 2018

Thanks for the update!

@claurence

This comment has been minimized.

Copy link

claurence commented Oct 5, 2018

Kubernetes 1.13 is going to be a 'stable' release since the cycle is only 10 weeks. We encourage no big alpha features and only consider adding this feature if you have a high level of confidence it will make code slush by 11/09. Are there plans for this enhancement to graduate to alpha/beta/stable within the 1.13 release cycle? If not, can you please remove it from the 1.12 milestone or add it to 1.13?

We are also now encouraging that every new enhancement aligns with a KEP. If a KEP has been created, please link to it in the original post. Please take the opportunity to develop a KEP

@xing-yang

This comment has been minimized.

Copy link

xing-yang commented Oct 5, 2018

Actually this feature is alpha in 1.12. We are not planning to go beta in 1.13. It should be changed as follows:

External repo (x.y): 1.8
Alpha release target (x.y): 1.12
Beta release target (x.y): 1.14
@kacole2

This comment has been minimized.

Copy link
Member

kacole2 commented Oct 5, 2018

Thanks @xing-yang
/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.12 milestone Oct 5, 2018

@kacole2 kacole2 added tracked/no and removed tracked/yes labels Oct 5, 2018

@claurence

This comment has been minimized.

Copy link

claurence commented Jan 16, 2019

@jingxu97 Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th and I want to remind that all enhancements must have a KEP

@xing-yang

This comment has been minimized.

Copy link

xing-yang commented Jan 16, 2019

Hi @claurence - we are working on "execution hook" and "PV/PVC taints and tolerations" for 1.14. Do we need KEP for those? If so, I'll submit a KEP for execution hook. Jing is working on a spec for taints and tolerations.

@claurence

This comment has been minimized.

Copy link

claurence commented Jan 18, 2019

@xing-yang Thanks - all issues should have a KEP associated with them so please add one for this issue. Also do you have links to the PRs for the two items referenced above?

@claurence

This comment has been minimized.

Copy link

claurence commented Jan 18, 2019

Also is alpha still the correct stage or is this feature graduating to beta?

@xing-yang

This comment has been minimized.

Copy link

xing-yang commented Jan 18, 2019

@claurence Yes, it is still Alpha.
We are still working on the KEP so it is not submitted yet. What is the deadline for submitting a KEP?

@xing-yang

This comment has been minimized.

Copy link

xing-yang commented Jan 20, 2019

@claurence here's the KEP on ExecutionHook: #705

@claurence

This comment has been minimized.

Copy link

claurence commented Jan 22, 2019

@xing-yang please submit the KEP before the enhancements freeze date for Jan 29th if there is expected work for 1.14 - if this is out of scope for 1.14 a KEP will be needed for work in 1.15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.