Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

maxUnavailable for StatefulSets #961

Open
krmayankk opened this issue Apr 14, 2019 · 106 comments
Open

maxUnavailable for StatefulSets #961

krmayankk opened this issue Apr 14, 2019 · 106 comments
Assignees
Labels
sig/apps Categorizes an issue or PR as relevant to SIG Apps. stage/beta Denotes an issue tracking an enhancement targeted for Beta status tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team

Comments

@krmayankk
Copy link
Contributor

krmayankk commented Apr 14, 2019

Implement maxUnavailable for StatefulSets

Please to keep this description up to date. This will help the Enhancement Team track efficiently the evolution of the enhancement

@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Apr 14, 2019
@krmayankk
Copy link
Contributor Author

krmayankk commented Apr 14, 2019

/sig auth

@k8s-ci-robot k8s-ci-robot added sig/auth Categorizes an issue or PR as relevant to SIG Auth. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Apr 14, 2019
@krmayankk
Copy link
Contributor Author

krmayankk commented Apr 14, 2019

/assign @krmayankk

@krmayankk
Copy link
Contributor Author

krmayankk commented Apr 14, 2019

/milestone v1.15

@k8s-ci-robot k8s-ci-robot added this to the v1.15 milestone Apr 14, 2019
@kacole2
Copy link
Member

kacole2 commented Apr 14, 2019

/stage alpha

@k8s-ci-robot k8s-ci-robot added the stage/alpha Denotes an issue tracking an enhancement targeted for Alpha status label Apr 14, 2019
@kacole2 kacole2 added the tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team label Apr 14, 2019
@mrbobbytables
Copy link
Member

mrbobbytables commented Apr 18, 2019

@krmayankk should this be sig auth? The KEP itself refers to sig-apps.

@krmayankk
Copy link
Contributor Author

krmayankk commented Apr 18, 2019

@mrbobbytables no this is specific to sig-apps

@mrbobbytables
Copy link
Member

mrbobbytables commented Apr 18, 2019

@krmayankk it has the responsible sig list as sig-auth 'in the post and is labeled with auth.

  • Responsible SIGs: sig-auth

Would you mind updating the issue?

Thanks!

/remove-sig auth
/sig apps

@k8s-ci-robot k8s-ci-robot added sig/apps Categorizes an issue or PR as relevant to SIG Apps. and removed sig/auth Categorizes an issue or PR as relevant to SIG Auth. labels Apr 18, 2019
@craiglpeters
Copy link
Contributor

craiglpeters commented Apr 29, 2019

@krmayankk, Kubernetes 1.15 Enhancement Freeze is 4/30/2019. To be included in the Kubernetes 1.15 milestone, KEPs are required to be in an "Implementable" state with proper test plans and graduation criteria. Please submit any PRs needed to make this KEP adhere to inclusion criteria. If this will slip from the 1.15 milestone, please let us know so we can make appropriate tracking changes.

@krmayankk
Copy link
Contributor Author

krmayankk commented Apr 29, 2019

@craiglpeters here is the PR #1010 I will try to see if i can get some decision on this , in the next 1-2 days. If yes, i can try an exception for this enhancement to be implementable in 1.15. If no decision is nearing, will ask you to remove it from tracking.

@craiglpeters
Copy link
Contributor

craiglpeters commented May 1, 2019

@krmayankk, Enhancement Freeze for Kubernetes 1.15 has passed and this did not meet the deadline. This is now being removed from the 1.15 milestone and the tracking sheet. If there is a need for this to be in 1.15, please file an Enhancement Exception. Thank you.

@craiglpeters craiglpeters added tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team and removed tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team labels May 1, 2019
@craiglpeters
Copy link
Contributor

craiglpeters commented May 1, 2019

/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.15 milestone May 1, 2019
@craiglpeters craiglpeters added tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team and removed tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team labels May 1, 2019
@craiglpeters
Copy link
Contributor

craiglpeters commented May 1, 2019

@krmayankk please work to get the KEP to the implementable state quickly

/milestone v1.15

@k8s-ci-robot k8s-ci-robot added this to the v1.15 milestone May 1, 2019
@draveness
Copy link
Member

draveness commented May 6, 2019

Is this enhancement will be included in milestone v1.15? I'd like to take this if it becomes implementable.

@kacole2
Copy link
Member

kacole2 commented May 6, 2019

This is still being tracked but I don't see an exception request submitted nor the KEP in an implementable state. @craiglpeters what's the status of this?

@craiglpeters
Copy link
Contributor

craiglpeters commented May 6, 2019

@kacole2 an exception request was filed on May 1st. However this still is not in the implementable state, so I recommend moving out of 1.15 tracking.

@soltysh
Copy link
Contributor

soltysh commented Feb 3, 2022

@gracenng this was updated in #3100

@mehabhalodiya
Copy link

mehabhalodiya commented Feb 12, 2022

Hi @krmayankk 👋🏻 1.24 Docs shadow here.

This enhancement is marked as 'Needs Docs' for the 1.24 release.

Please follow the steps detailed in the documentation to open a PR against the dev-1.24 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thu March 31, 11:59 PM PDT.

Also, if needed take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Thanks!

@gracenng
Copy link
Member

gracenng commented Mar 15, 2022

Hi @krmayankk 1.24 Enhancements Team here,
With Code Freeze approaching on 18:00 PDT Tuesday March 29th 2022, the enhancement status is at risk as there is no linked k/k PR. Kindly list them in this issue. Thanks!

(update)
Open Implementation k/k PR:

@gracenng
Copy link
Member

gracenng commented Mar 29, 2022

All good for 1.24 Code Freeze 🥳

@Priyankasaggu11929 Priyankasaggu11929 added tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team and removed tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team labels May 10, 2022
@Priyankasaggu11929 Priyankasaggu11929 removed this from the v1.24 milestone May 10, 2022
@krmayankk
Copy link
Contributor Author

krmayankk commented Jul 30, 2022

@Priyankasaggu11929 can you help marking this as tracked for 1.26 ?

@Priyankasaggu11929
Copy link
Member

Priyankasaggu11929 commented Jul 30, 2022

Hello @krmayankk the enhancement would require opting-in by the SIG leads, when the 1.26 Release cycle starts.

Once opted in, the then Enhancements team would track it for 1.26 cycle.

Right now, we're still in 1.25 Release cycle. 🙂

@soltysh
Copy link
Contributor

soltysh commented Aug 1, 2022

@krmayankk open a PR for KEP updates, and we can definitely push this over in 1.26, ping me on slack, when you do that :)

@krmayankk
Copy link
Contributor Author

krmayankk commented Sep 7, 2022

@soltysh can you mark this as tracked ? I am working on the KEP and test updates this week

@shellsuperhu
Copy link

shellsuperhu commented Sep 16, 2022

it is work in 1.24.4 ?

My k8s running in 1.24.4 , it seems not work

kube-apiserver:
running with --feature-gates=MaxUnavailableStatefulSet=true

[root@node10034 ~]# docker exec -it kube-apiserver /bin/bash
root@node10034:/# ps axfu|grep kube|grep Max
root         1 10.9  2.8 1316932 459372 ?      Ssl  03:07  16:45 kube-apiserver --etcd-keyfile=/etc/kubernetes/ssl/kube-node-key.pem --proxy-client-cert-file=/etc/kubernetes/ssl/kube-apiserver-proxy-client.pem --service-account-lookup=true --bind-address=0.0.0.0 --advertise-address=192.168.100.34 --requestheader-allowed-names=kube-apiserver-proxy-client --cloud-provider= --tls-private-key-file=/etc/kubernetes/ssl/kube-apiserver-key.pem --authentication-token-webhook-config-file=/etc/kubernetes/kube-api-authn-webhook.yaml --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction,Priority,TaintNodesByCondition,PersistentVolumeClaimResize --storage-backend=etcd3 --audit-log-maxage=30 --audit-log-format=json --etcd-certfile=/etc/kubernetes/ssl/kube-node.pem --etcd-prefix=/registry --authorization-mode=Node,RBAC --audit-log-maxsize=100 --service-cluster-ip-range=10.43.0.0/16 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --runtime-config=authorization.k8s.io/v1beta1=true --feature-gates=MaxUnavailableStatefulSet=true --api-audiences=unknown --service-account-signing-key-file=/etc/kubernetes/ssl/kube-service-account-token-key.pem --tls-cert-file=/etc/kubernetes/ssl/kube-apiserver.pem --anonymous-auth=false --requestheader-username-headers=X-Remote-User --profiling=false --etcd-cafile=/etc/kubernetes/ssl/kube-ca.pem --proxy-client-key-file=/etc/kubernetes/ssl/kube-apiserver-proxy-client-key.pem --requestheader-client-ca-file=/etc/kubernetes/ssl/kube-apiserver-requestheader-ca.pem --authentication-token-webhook-cache-ttl=5s --secure-port=6443 --audit-log-maxbackup=10 --etcd-servers=https://192.168.100.34:2379,https://192.168.100.37:2379,https://192.168.100.35:2379 --kubelet-client-certificate=/etc/kubernetes/ssl/kube-apiserver.pem --service-node-port-range=30000-32767 --requestheader-group-headers=X-Remote-Group --service-account-issuer=rke --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --client-ca-file=/etc/kubernetes/ssl/kube-ca.pem --kubelet-client-key=/etc/kubernetes/ssl/kube-apiserver-key.pem --requestheader-extra-headers-prefix=X-Remote-Extra- --audit-log-path=/var/log/kube-audit/audit-log.json --audit-policy-file=/etc/kubernetes/audit-policy.yaml --service-account-key-file=/etc/kubernetes/ssl/kube-service-account-token-key.pem --allow-privileged=true
root@node10034:/# kube-apiserver --version
Kubernetes v1.24.4

statefulset.yaml like:

---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: testx
  labels:
    app: testx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: testx
  serviceName: testx
 podManagementPolicy: "Parallel"
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 2

kubectl apply -f statefulset.yaml

kubectl get pods -l app=testx -w

testx-2   1/1     Terminating       0          3m44s
testx-2   1/1     Terminating       0          4m15s
testx-2   0/1     Terminating       0          4m16s
testx-2   0/1     Terminating       0          4m16s
testx-2   0/1     Terminating       0          4m16s
testx-2   0/1     Pending           0          0s
testx-2   0/1     Pending           0          0s
testx-2   0/1     Init:0/1          0          0s
testx-2   0/1     Init:0/1          0          2s
testx-2   0/1     PodInitializing   0          7s
testx-2   0/1     Running           0          8s
testx-2   0/1     Running           0          61s
testx-2   1/1     Running           0          61s
testx-1   1/1     Terminating       0          6m19s
testx-1   1/1     Terminating       0          6m49s
testx-1   0/1     Terminating       0          6m51s
testx-1   0/1     Terminating       0          6m51s
testx-1   0/1     Terminating       0          6m51s
testx-1   0/1     Pending           0          0s
testx-1   0/1     Pending           0          0s
testx-1   0/1     Init:0/1          0          0s
testx-1   0/1     Init:0/1          0          1s
testx-1   0/1     Init:0/1          0          5s
testx-1   0/1     PodInitializing   0          6s
testx-1   0/1     Running           0          7s
testx-1   0/1     Running           0          60s
testx-1   1/1     Running           0          60s
testx-0   1/1     Terminating       0          8m51s
testx-0   1/1     Terminating       0          9m22s
testx-0   0/1     Terminating       0          9m23s
testx-0   0/1     Terminating       0          9m23s
testx-0   0/1     Terminating       0          9m23s
testx-0   0/1     Pending           0          0s
testx-0   0/1     Pending           0          0s
testx-0   0/1     Init:0/1          0          0s
testx-0   0/1     Init:0/1          0          1s
testx-0   0/1     PodInitializing   0          6s
testx-0   0/1     Running           0          7s
testx-0   0/1     Running           0          60s
testx-0   1/1     Running           0          60s

it seems not work, like link below
https://kubernetes.io/blog/2022/05/27/maxunavailable-for-statefulset/

@krmayankk
Copy link
Contributor Author

krmayankk commented Sep 17, 2022

@shellsuperhu do you see the maxUnavailable field, wheen you do kubectl get statefulset -o yaml ? I think your feature gate is not enabled properly . You may need to enable it not only in the apiserver, but also in the kube-controller-manager . How are you running your k8s and how are you enabling the feature gate ?

@shellsuperhu
Copy link

shellsuperhu commented Sep 19, 2022

@krmayankk
Thanks. It's working fine.
kube-controller and kube-api is must enabled '--feature-gates=MaxUnavailableStatefulSet=true' at the same time.

@soltysh
Copy link
Contributor

soltysh commented Sep 21, 2022

/milestone v1.26
/stage beta
/label lead-opted-in

@k8s-ci-robot k8s-ci-robot added stage/beta Denotes an issue tracking an enhancement targeted for Beta status and removed stage/alpha Denotes an issue tracking an enhancement targeted for Alpha status labels Sep 21, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.26 milestone Sep 21, 2022
@k8s-ci-robot k8s-ci-robot added the lead-opted-in Denotes that an issue has been opted in to a release label Sep 21, 2022
@rhockenbury
Copy link

rhockenbury commented Sep 22, 2022

/label tracked/yes
/remove-label tracked/no

@k8s-ci-robot k8s-ci-robot added tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team and removed tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team labels Sep 22, 2022
@parul5sahoo
Copy link
Member

parul5sahoo commented Sep 25, 2022

Hello @krmayankk 👋, 1.26 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage beta for 1.26 (correct me, if otherwise)

Here's where this enhancement currently stands:

  • KEP readme using the latest template has been merged into the k/enhancements repo.
  • KEP status is marked as implementable for latest-milestone: 1.26
  • KEP readme has a updated detailed test plan section filled out
  • KEP readme has up to date graduation criteria
  • KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to update the following:

  • Edit the test plan section and add the release sign-off checklist to the KEP readme according to the latest template linked above.
  • Update the latest stage and milestone to beta for 1.26 in the kep.yaml file.
  • Complete and merge the PRR for beta stage.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Let us know if you have any queries. Thank you!

@parul5sahoo
Copy link
Member

parul5sahoo commented Oct 4, 2022

Hello @krmayankk just a quick remainder the enhancements freeze on 18:00 PDT on Thursday 6th October 2022.

@rhockenbury
Copy link

rhockenbury commented Oct 7, 2022

Hello 👋, 1.26 Enhancements Lead here.

Unfortunately, this enhancement did not meet requirements for enhancements freeze.

If you still wish to progress this enhancement in v1.26, please file an exception request. Thanks!

/milestone clear
/label tracked/no
/remove-label tracked/yes
/remove-label lead-opted-in

@k8s-ci-robot k8s-ci-robot added tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team and removed tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team labels Oct 7, 2022
@k8s-ci-robot k8s-ci-robot removed this from the v1.26 milestone Oct 7, 2022
@k8s-ci-robot k8s-ci-robot removed the lead-opted-in Denotes that an issue has been opted in to a release label Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/apps Categorizes an issue or PR as relevant to SIG Apps. stage/beta Denotes an issue tracking an enhancement targeted for Beta status tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team
Projects
Status: Graduating
Development

No branches or pull requests