Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Durable (non-shared) local storage management #121

Closed
smarterclayton opened this issue Oct 9, 2016 · 65 comments

Comments

Projects
@smarterclayton
Copy link
Contributor

commented Oct 9, 2016

Feature Description

  • One-line feature description (can be used as a release note): Make locally attached (non-network attached) storage available as a persistent volume source.
  • Primary contact (assignee): @msau42 @vishh
  • Responsible SIGs: @kubernetes/sig-storage-feature-requests @kubernetes/sig-apps-feature-requests
  • Design proposal link (community repo): kubernetes/community#306
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
  • Approver (likely from SIG/area to which feature belongs):
  • Feature target (which target equals to which milestone):
    • Alpha release target (x.y): 1.7/1.8
    • Beta release target (x.y): 1.10
    • Stable release target (x.y): 1.14
@davidopp

This comment has been minimized.

Copy link
Member

commented Oct 9, 2016

I assume this is kubernetes/kubernetes#7562 ?

And that kubernetes/kubernetes#30044 is related?

@smarterclayton

This comment has been minimized.

Copy link
Contributor Author

commented Oct 11, 2016

Yes and yes. Consensus is to see if we can pull together a design at most in 1.5

@bprashanth

This comment has been minimized.

Copy link
Member

commented Oct 11, 2016

It would be great if we were ready to go at this in 1.6
@kubernetes/sig-storage @jingxu97

@thockin

This comment has been minimized.

Copy link
Member

commented Oct 11, 2016

I think we can commit to that.

On Mon, Oct 10, 2016 at 9:35 PM, Prashanth B notifications@github.com
wrote:

It would be great if we were ready to go at this in 1.6
@kubernetes/sig-storage
https://github.com/orgs/kubernetes/teams/sig-storage @jingxu97
https://github.com/jingxu97


You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
#121 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFVgVCoNyQWhL1_AAMD0eE1Uh79jaMWsks5qyxIggaJpZM4KR2_S
.

@idvoretskyi idvoretskyi modified the milestone: v1.5 Oct 11, 2016

@jingxu97

This comment has been minimized.

Copy link
Contributor

commented Oct 11, 2016

Yes, I will definitely spend more time on this after this short release
cycle for 1.5. Thanks!

Jing

On Mon, Oct 10, 2016 at 10:53 PM, Tim Hockin notifications@github.com
wrote:

I think we can commit to that.

On Mon, Oct 10, 2016 at 9:35 PM, Prashanth B notifications@github.com
wrote:

It would be great if we were ready to go at this in 1.6
@kubernetes/sig-storage
https://github.com/orgs/kubernetes/teams/sig-storage @jingxu97
https://github.com/jingxu97


You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/kubernetes/features/issues/
121#issuecomment-252810762>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AFVgVCoNyQWhL1_
AAMD0eE1Uh79jaMWsks5qyxIggaJpZM4KR2_S>
.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#121 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ASSNxQlkIyJ98tTTTqNRkJh2JB3OtI-jks5qyyRJgaJpZM4KR2_S
.

  • Jing

@idvoretskyi idvoretskyi modified the milestones: next-milestone, v1.5 Oct 13, 2016

@davidopp davidopp referenced this issue Nov 7, 2016

Closed

Data awareness and Data/Pod affinity #139

0 of 23 tasks complete

@vishh vishh assigned vishh and msau42 and unassigned smarterclayton Jan 18, 2017

@vishh

This comment has been minimized.

Copy link
Member

commented Jan 18, 2017

@msau42 and myself will be tackling this problem in v1.6. The plan is as follows:

  1. Design completion by v1.6
  2. Alpha support for local storage by v1.7
  3. Beta (or may be GA) by v1.8

@rkouj and @jingxu97 will also be helping with this effort.

@msau42 and @vishh will post a high level design proposal before end of Jan `17.

@vishh vishh changed the title Local storage for use with PetSets and bare-metal clusters Local storage Management (ephemeral & durable) Jan 18, 2017

@davidopp

This comment has been minimized.

Copy link
Member

commented Jan 24, 2017

Please loop me in on the scheduling aspects of this.

@idvoretskyi idvoretskyi modified the milestones: v1.6, next-milestone Jan 26, 2017

@idvoretskyi

This comment has been minimized.

Copy link
Member

commented Jan 26, 2017

Misread the @vishh comment, let's target 1.7 when this milestone will be open.

@wattsteve

This comment has been minimized.

Copy link

commented Jan 27, 2017

@vishh and others. Given that we're already talking about targeting releases, is there a design proposal for this? FYI @saad-ali @childsb

@vishh

This comment has been minimized.

Copy link
Member

commented Jan 27, 2017

@krmayankk

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2017

@vishh has this anything to do with sticky empty dir proposal or local storage for statefulset ? I am getting confused which feature is doing what ? There are many things needed for local storage , e.g. 1: local storage support in statefulset 2: local storage volumes which is this one 3: pod should be scheduled in the same node where their data is already available 4: may be moving data to where the pod gets scheduled . It will be good to have one local storage featur which covers all related issues and proposals

@vishh

This comment has been minimized.

Copy link
Member

commented Jan 29, 2017

@krmayankk the plan is to cover all supported use cases for local storage.

@davidopp

This comment has been minimized.

Copy link
Member

commented Jan 29, 2017

@vishh I assume you are not including (4) from @krmayankk's comment? (migrating data between nodes)

@msau42

This comment has been minimized.

Copy link
Member

commented Oct 8, 2018

Plan for 1.13 is to remain in beta. We are looking for more feedback on the feature before graduating to GA.

@Bessonov

This comment has been minimized.

Copy link

commented Dec 27, 2018

If you want your pod to be able to access the exact same disk/data if it restarts then yes.

@msau42 thanks. Does it mean that I can do following for a 5 nodes cluster:

  1. Mark 3 nodes with tag like "database"
  2. Set nodeAffinity to "database" tag for local-storage
  3. Create a Deployment with 3 replicas for CockroachDB with podAntiAffinity to run once on node
    ?

If one "database" node fail, then it should be ensured that new node has the tag "database". After "magic happens" everything is fine. It's some sort of DaemonSet.

Sorry if I get this wrong. Just started to learn about Kubernetes.

Is there any plan to do it a less strict? I mean get rid of tags and:

  1. "track" nodes which were used by Deployment (let's call it "dirty nodes")
  2. Prefer dirty nodes while schedule
  3. If no dirty node available (ex. down or count of dirty nodes is less than count of replicas), then use non-dirty node and mark it as dirty

? I think this can improve deployment of stateful apps with build-in replication. What do you think?

@Bessonov

This comment has been minimized.

Copy link

commented Dec 27, 2018

@ryanwalls thanks for posting your configuration for cassandra. How it's running on local-storage? Do you experience any problems with your setup or do you have new insights?

@claurence

This comment has been minimized.

Copy link

commented Jan 16, 2019

@msau42 Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th and I want to remind that all enhancements must have a KEP

@msau42

This comment has been minimized.

Copy link
Member

commented Jan 16, 2019

Plan for 1.14 is to move to GA. This feature already has a design proposal. Do we need to convert this to a KEP?

@claurence

This comment has been minimized.

Copy link

commented Jan 18, 2019

@msau42 yes we'd like the design proposal converted to a KEP! Thanks! Additionally if you have any open PRs left for this issue please let me know so we can add them to our tracking sheet.

@msau42

This comment has been minimized.

Copy link
Member

commented Jan 24, 2019

KEP is here: https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190124-local-persistent-volumes.md

For 1.14 work, the following needs to be done:

@msau42

This comment has been minimized.

Copy link
Member

commented Jan 30, 2019

/assign @cofyc

@saad-ali saad-ali added stage/stable and removed stage/beta labels Jan 30, 2019

@saad-ali saad-ali added this to the v1.14 milestone Jan 30, 2019

@spiffxp spiffxp added tracked/yes and removed tracked/no labels Feb 7, 2019

@spiffxp

This comment has been minimized.

Copy link
Member

commented Feb 7, 2019

Adding tracked label to reconcile against https://bit.ly/k8s114-enhancements
FYI @claurence @lachie83 @lledru @ameukam

@jimangel

This comment has been minimized.

Copy link
Member

commented Feb 13, 2019

Hey @cofyc @msau42 👋 I'm the v1.14 docs release lead. Just a friendly reminder we're looking for a PR against k/website (branch dev-1.14) due by Friday, March 1. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

@liggitt

This comment has been minimized.

Copy link
Member

commented Feb 15, 2019

For 1.14 work, the following needs to be done:

it doesn't sound like there are API changes involved in the 1.14 work. Is that accurate?

@thockin

This comment has been minimized.

Copy link
Member

commented Feb 15, 2019

I think API was done long ago.

@lledru

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

Hello, 1.14 enhancement shadow here. Code Freeze is March 7th and all PRs must be merged by then to your issue to make the 1.14 release. What open K/K PRs do you still have that need to merge? Thanks

@msau42

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

The items I listed above here tracks our 1.14 work.

@kacole2

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

Hello @msau42 @smarterclayton , I'm the Enhancement Lead for 1.15. Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet. If this was included as GA in 1.14 lets close this issue.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.14 milestone Apr 12, 2019

@kacole2 kacole2 added tracked/no and removed tracked/yes labels Apr 12, 2019

@msau42

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

1.14 was GA and any additional enhancements should be tracked as new features.
/close

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Apr 12, 2019

@msau42: Closing this issue.

In response to this:

1.14 was GA and any additional enhancements should be tracked as new features.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kacole2 kacole2 removed the tracked/no label Apr 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.