Add Task: upgrading from PetSet to StatefulSet #1704

janetkuo · 2016-11-16T22:47:36Z

cc @bprashanth @erictune @foxish @kow3ns @enisoc @kubernetes/sig-apps

This change is

janetkuo · 2016-11-16T23:09:30Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

@@ -0,0 +1,92 @@
+---


@kubernetes/docs should I put assignees: here? It's not in the template, but I assume it works for all docs?

Added assignee

janetkuo · 2016-11-16T23:43:21Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+#### Create Stateful Sets
+
+Now, you want to recreate all Pet Sets, but turning them into Stateful Sets. To do this, for every Pet Set config file you have, 
+change `apiVersion` from `apps/v1alpha1` to `apps/v1beta1` and change `kind` from `PetSet` to `StatefulSet`. 


Should we ask the users to add the initialized=false annotation if not set in the original petset config file?

enisoc · 2016-11-16T23:33:29Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+
+* **Stateful Set is the new Pet Set**: Pet Set is no longer available in any Kubernetes release >= 1.5. It becomes beta Stateful Set.
+* **Stateful Set fencing**: Fencing guards is implemented in >= 1.5 to prevent split brain scenarios for Stateful Sets.
+* **Flipped debug hook behavior**: The behavior of debug hook for Pet Set, i.e. annotation `pod.alpha.kubernetes.io/initialized`, is now flipped for Stateful Sets. The absence of this annotation will pause the Pet Sets, but will NOT pause the Stateful Sets. 


Should we summarize this by recommending that in most cases you should now remove this annotation from all your StatefulSet manifests?

enisoc · 2016-11-16T23:35:07Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+
+### Differences between alpha Pet Sets and beta Stateful Sets
+
+Pet Set was introduced as an alpha resource in Kubernetes release 1.3, and was renamed to Stateful Set as a beta resource in 1.5. 


Would it make sense to link to the community vote thread for the rename? As a user, I would be confused at this point why it was renamed and why "Stateful Set" was chosen.

Added link to the issue

enisoc · 2016-11-16T23:37:04Z

_data/tasks.yml

-
+- title: Stateful Set
+  section:
+  - title: Upgrade from PetSet to Stateful Sets


I think it sounds better to keep everything singular (s/sets/set/ig) and consistent (e.g. the title says PetSet but the URL below is petsets).

Also is there any reason PetSet is one word but Stateful Set is two?

Changed it to two words. But keep is as plural to be consistent with other documents

enisoc · 2016-11-16T23:37:39Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+Here are some notable changes:
+
+* **Stateful Set is the new Pet Set**: Pet Set is no longer available in any Kubernetes release >= 1.5. It becomes beta Stateful Set.
+* **Stateful Set fencing**: Fencing guards is implemented in >= 1.5 to prevent split brain scenarios for Stateful Sets.


TODO: link to doc about fencing

enisoc · 2016-11-16T23:39:03Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+$ kubectl get petsets --all-namespaces -o yaml > all-petsets.yaml
+```
+
+#### Delete all Pet Sets, non-cascadingly 


I suggest "without cascading" instead of "non-cascadingly".

enisoc · 2016-11-16T23:40:19Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+#### Delete all Pet Sets, non-cascadingly 
+
+If you find existing Pet Sets instead, you need to delete all Pet Sets *non-cascadingly*. You can do this from `kubectl` with `--cascade=false`. 
+Note that if the flag wasn't set, cascading deletion will be performed by default. 


I think this Note should be big and loud, and say this will delete all your pods if you let it cascade. Do we have a template for "Warning" text?

Is bold loud enough?

enisoc · 2016-11-16T23:41:59Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+$ kubectl get petsets --all-namespaces
+```
+
+At this moment, you've deleted all Pet Sets in your cluster, but not their pods, persistent volumes, or persistent volume claims. 


Perhaps note the implications of this state, e.g. the pods will be vulnerable to node failures until you finish the upgrade and recreate StatefulSets.

enisoc · 2016-11-16T23:43:10Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+
+At this moment, you've deleted all Pet Sets in your cluster, but not their pods, persistent volumes, or persistent volume claims. 
+
+#### Upgrade your Kubernetes cluster to >= 1.5


Just to be clear, there's no way to keep the petsets around until after the upgrade? I thought I overheard someone saying that was possible, but I guess it makes sense that the controller code would be gone anyway so even if the API objects existed, it wouldn't do any good.

enisoc · 2016-11-16T23:45:36Z

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md

+#### Create Stateful Sets
+
+Now, you want to recreate all Pet Sets, but turning them into Stateful Sets. To do this, for every Pet Set config file you have, 
+change `apiVersion` from `apps/v1alpha1` to `apps/v1beta1` and change `kind` from `PetSet` to `StatefulSet`. 


On a practical level, it's probably best to prepare these updated files before actually deleting the petsets, to minimize downtime of the controller. I'm not sure what's the best place to mention that. Would it make sense to describe the file modifications as the first step up above, when you say to either gather or download the YAML?

Added this step and move to before deleting Pet Sets

kow3ns · 2016-11-17T18:29:09Z

Review status: 0 of 3 files reviewed at latest revision, 11 unresolved discussions.

docs/tasks/index.md, line 26 at r1 (raw file):

#### Stateful Set

* [Upgrade from PetSet to Stateful Sets](/docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md)

PetSet should probably be Pet Sets

docs/tasks/stateful-set/upgrade-from-petsets-to-stateful-sets.md, line 51 at r1 (raw file):

Previously, enisoc (Anthony Yeh) wrote…

I think this Note should be big and loud, and say this will delete all your pods if you let it cascade. Do we have a template for "Warning" text?

From what I see **scary text** seems to be the standard.

Comments from Reviewable

janetkuo · 2016-11-17T19:52:30Z

@enisoc @kow3ns comments addressed and some more updates, see the second commit for changes. PTAL

enisoc · 2016-11-17T19:50:02Z

_data/tasks.yml

@@ -23,4 +23,7 @@ toc:
  section:
  - title: Assigning Pods to Nodes
    path: /docs/tasks/administer-cluster/assign-pods-nodes/
-
+- title: Stateful Sets


I just noticed other section titles above have the form of "Doing something" rather than just a noun. Should we call our new section something like "Managing Stateful Applications"?

enisoc · 2016-11-17T19:51:21Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+Here are some notable changes:
+
+* **Stateful Set is the new Pet Set**: Pet Set is no longer available in any Kubernetes release >= 1.5. It becomes beta Stateful Set. To know why the name is changed, see this [discussion thread](https://github.com/kubernetes/kubernetes/issues/27430).
+* **Stateful Set fencing**: Fencing guards is implemented in >= 1.5 to prevent split brain scenarios for Stateful Sets. *TODO: Link to doc about fencing*


guards "are"?

enisoc · 2016-11-17T19:57:35Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+
+* **Stateful Set is the new Pet Set**: Pet Set is no longer available in any Kubernetes release >= 1.5. It becomes beta Stateful Set. To know why the name is changed, see this [discussion thread](https://github.com/kubernetes/kubernetes/issues/27430).
+* **Stateful Set fencing**: Fencing guards is implemented in >= 1.5 to prevent split brain scenarios for Stateful Sets. *TODO: Link to doc about fencing*
+* **Flipped debug hook behavior**: The behavior of debug hook for Pet Set, i.e. annotation `pod.alpha.kubernetes.io/initialized`, is now flipped for Stateful Sets. The absence of this annotation will pause the Pet Sets, but will NOT pause the Stateful Sets. In most cases, you no longer need this annotation in your Stateful Set manifests. 


Saying "The behavior [...] is now flipped" might sound like specifying "true" will now trigger the behavior that used to be triggered by "false" and vice versa. "debug hook" also needs a "the".

To be precise, I recommend something like:

The default value of the debug annotation (`pod.alpha.kubernetes.io/initialized`) is now `true`. The absence...

enisoc

LGTM

foxish · 2016-11-18T00:10:20Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+
+* **Stateful Set is the new Pet Set**: Pet Set is no longer available in any Kubernetes release >= 1.5. It becomes beta Stateful Set. To know why the name is changed, see this [discussion thread](https://github.com/kubernetes/kubernetes/issues/27430).
+* **Stateful Set fencing**: Fencing guards are implemented in >= 1.5 to prevent split brain scenarios for Stateful Sets. *TODO: Link to doc about fencing*
+* **Flipped debug hook behavior**: The default value of the debug annotation (`pod.alpha.kubernetes.io/initialized`) is now `true`. The absence of this annotation will pause the Pet Sets, but will NOT pause the Stateful Sets. In most cases, you no longer need this annotation in your Stateful Set manifests. 


s/debug hook/debug annotation/ ?
also, "The absence of this annotation will not pause Stateful Set operations" seems like a clearer thing to say.

foxish · 2016-11-18T00:13:09Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+Here are some notable changes:
+
+* **Stateful Set is the new Pet Set**: Pet Set is no longer available in any Kubernetes release >= 1.5. It becomes beta Stateful Set. To know why the name is changed, see this [discussion thread](https://github.com/kubernetes/kubernetes/issues/27430).
+* **Stateful Set fencing**: Fencing guards are implemented in >= 1.5 to prevent split brain scenarios for Stateful Sets. *TODO: Link to doc about fencing*


I think the word "fencing" shouldn't be in user-facing documentation.

Stateful Sets now guarantee at-most-one semantics to ensure exactly one instance of each pod running anywhere in a cluster, in order to guard against split brain scenarios with distributed applications.

foxish · 2016-11-18T00:14:44Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+1. Change `apiVersion` from `apps/v1alpha1` to `apps/v1beta1`
+2. Change `kind` from `PetSet` to `StatefulSet`
+3. If you have the debug hook annotation `pod.alpha.kubernetes.io/initialized` set to `true`, you may remove it since it's redundant. If you don't have this annotation, you should add one, with the value set to `false`.
+


I think we should explain here what it means to add that annotation with "false"?

foxish · 2016-11-18T00:16:49Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+
+#### Create Stateful Sets
+
+Now, you want to recreate all Pet Sets, but turning them into Stateful Sets. To do this, just create all Stateful Set manifests you generate in the previous step. 


Now, you want to recreate all Pet Sets, but turning them into Stateful Sets

This sentence seems a bit confusing. Maybe we can mention that we now create Stateful Sets objects which will adopt the pods belonging to the Pet Set objects formerly.

enisoc · 2016-11-18T19:25:48Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+Here are some notable changes:
+
+* **Stateful Set is the new Pet Set**: Pet Set is no longer available in any Kubernetes release >= 1.5. It becomes beta Stateful Set. To know why the name is changed, see this [discussion thread](https://github.com/kubernetes/kubernetes/issues/27430).
+* **Stateful Set guards against split brain**: Stateful Sets guarantee at-most-one semantics to ensure exactly one instance of each pod running anywhere in a cluster, in order to guard against split brain scenarios with distributed applications. *TODO: Link to doc about fencing*


The phrase "exactly one" contradicts "at-most-one". To avoid having to say "at most one" twice in the sentence, you could say, "Stateful Sets guarantee at most one pod for a given ordinal index can be running anywhere in the cluster".

enisoc · 2016-11-18T19:26:40Z

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md

+#### Create Stateful Sets
+
+You need to create Stateful Sets to adopt the pods belonging to the deleted Pet Sets. 
+To do this, just create all Stateful Set manifests you generate in the previous step. 


enisoc · 2016-11-24T00:02:59Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+At this moment, you've deleted all Pet Sets in your cluster, but not their pods, persistent volumes, or persistent volume claims. 
+However, since the pods are not managed by Pet Sets anymore, they will be vulnerable to node failures until you finish the upgrade and recreate Stateful Sets.
+
+#### Upgrade your Kubernetes cluster to >= 1.5


Should we say to upgrade kubectl as well? It seems necessary in order to run the commands below like kubectl get statefulsets.

Added this section

enisoc · 2016-11-24T00:04:11Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+### Upgrading from Pet Sets to Stateful Sets
+
+#### Find all Pet Sets and their manifests 


Should we mention that you need to keep the old kubectl version to run these commands?

Not sure what you man by "keep the old kubectl version". Would you clarify?

I was worried that if the user upgrades to kubectl from v1.4 to v1.5 first, then the following (kubectl get petsets) would not work. I tested it just now and it does seem to work, but there may be some unexpected results. For example, I did kubectl delete petset <name> with kubectl v1.5 and it did delete the petset, but it didn't scale it down first. I'm guessing that's because the scale-down is initiated by kubectl on the client side, so if you delete a type that the client doesn't know about, it will be the same as --cascade=false?

kubectl is backward compatible. So yes, you can do kubectl get petset with 1.5 kubectl + 1.4 cluster.

--cascade=true logic is baked into kubectl (implemented via Reaper in the code). We get the reaper of a resource to do cascading deletion. Since 1.5 kubectl doesn't have PetSetReaper anymore, it doesn't support cascading deletion for PetSet. This wouldn't make a difference in this case, since we're asking users to do --cascade=false. Do you think it's still necessary to make them not upgrade the kubectl yet? (I'm now leaning towards this, since it's safer, even though it may seem redundant.)

I think it makes sense to advise against upgrading to kubectl 1.5 until you're done acting on petsets. It's safer as a general rule - for example maybe you decide to delete some of your petsets WITH cascading since you don't care if they stay up throughout the upgrade process. If you had already upgraded to kubectl 1.5 it won't work the way you expected.

enisoc · 2016-11-24T01:06:31Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+#### Upgrade your Kubernetes cluster to >= 1.5
+
+Now, you may [upgrade your Kubernetes cluster](/docs/admin/cluster-management/#upgrading-a-cluster), or just your master, to >= 1.5.0.


Does upgrading nodes result in pod evictions? If so, then it's important to upgrade only the master, then create StatefulSets, then upgrade nodes.

You're right. They shouldn't upgrade nodes at this point, otherwise all pods will be gone.

enisoc · 2016-11-24T01:14:38Z

I manually tested that this flow works from v1.4.6 to current master. However, I was only able to test up to upgrading the master. I couldn't upgrade nodes because cluster/gce/upgrade.sh can't upgrade nodes with local binaries, and there isn't a new beta build yet with the fix for kubernetes/kubernetes#37257.

For the record, my playbook was:

# Install v1.4
git fetch --tags upstream
git checkout v1.4.6
build/release.sh
cluster/kube-up.sh

# Do petsets tutorial: http://kubernetes.io/docs/user-guide/petset/
# Prepare statefulset manifest.
# Delete petset with --cascade=false.
# Verify pods are still good.

# Upgrade master to v1.5
make clean
git checkout master
git pull upstream master
build-tools/release.sh
cluster/gce/upgrade.sh -M -l

# Switch kubectl from v1.4 to v1.5.
# Verify pods are still good.
# Create statefulset.
# Delete a pod and verify statefulset recreates it.

enisoc · 2016-11-28T19:59:59Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+```shell
+# Delete all existing Pet Sets without cascading 
+kubectl delete petsets -f <pet-set-file> --cascade=false


Take out the petsets argument when using -f. Also it may be worth warning against doing this with the YAML file you originally used to create the Pet Set, because that may contain other things like Services that you don't want to delete at this time.

Good catch. Thanks

enisoc · 2016-11-28T20:18:46Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+### Upgrading from Pet Sets to Stateful Sets
+
+#### Find all Pet Sets and their manifests 


I was worried that if the user upgrades to kubectl from v1.4 to v1.5 first, then the following (kubectl get petsets) would not work. I tested it just now and it does seem to work, but there may be some unexpected results. For example, I did kubectl delete petset <name> with kubectl v1.5 and it did delete the petset, but it didn't scale it down first. I'm guessing that's because the scale-down is initiated by kubectl on the client side, so if you delete a type that the client doesn't know about, it will be the same as --cascade=false?

enisoc · 2016-11-28T20:47:43Z

I repeated the test with v1.5.0-beta.2 and tried to continue to upgrade Nodes this time. After the upgrade.sh -N script reported success (all Nodes upgraded and Ready), the Pods managed by the StatefulSet never recovered:

NAME      READY     STATUS              RESTARTS   AGE
web-1     0/1       ContainerCreating   0          45m
web-3     0/1       ContainerCreating   0          45m
web-4     0/1       ContainerCreating   0          45m

There should be web-0 through web-4. I checked events on one of the Pods stuck in ContainerCreating and it seems to be a problem with reattaching the PV (see below). Is this a known issue for upgrades?

Also, is it expected that StatefulSet will not try to create web-0 until the existing Pods are Ready? I thought that StatefulSet would handle the case where Pod N+1 can't become Ready until Pod N is Ready, but if the behavior I'm seeing here (it's not trying to create web-0) is expected, then such dependencies must be disallowed to prevent deadlock.

$ kubectl describe pod web-1
Name:           web-1
Namespace:      default
Node:           kubernetes-minion-group-54hw/10.128.0.4
Start Time:     Mon, 28 Nov 2016 11:52:11 -0800
Labels:         app=nginx
Status:         Pending
IP:
Controllers:    PetSet/web
Containers:
  nginx:
    Container ID:
    Image:              gcr.io/google_containers/nginx-slim:0.8
    Image ID:   
    Port:               80/TCP
    Requests:
      cpu:              100m
    State:              Waiting
      Reason:           ContainerCreating
    Ready:              False
    Restart Count:      0
    Volume Mounts:
      /usr/share/nginx/html from www (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-xim26 (ro)
    Environment Variables:      <none>
Conditions:
  Type          Status
  Initialized   True 
  Ready         False 
  PodScheduled  True 
Volumes:
  www:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  www-web-1
    ReadOnly:   false
  default-token-xim26:
    Type:       Secret (a volume populated by a Secret)
    SecretName: default-token-xim26
QoS Class:      Burstable
Tolerations:    <none>
Events:
  FirstSeen     LastSeen        Count   From                                    SubObjectPath       Type             Reason          Message
  ---------     --------        -----   ----                                    -------------       -------- ------          -------
  40m           40m             1       {default-scheduler }                                        Normal           Scheduled       Successfully assigned web-1 to kubernetes-minion-group-54hw
  40m           40m             1       {kubelet kubernetes-minion-group-54hw}  spec.containers{nginx}       Normal          Pulling         pulling image "gcr.io/google_containers/nginx-slim:0.8"
  40m           40m             1       {kubelet kubernetes-minion-group-54hw}  spec.containers{nginx}       Normal          Pulled          Successfully pulled image "gcr.io/google_containers/nginx-slim:0.8"
  40m           40m             1       {kubelet kubernetes-minion-group-54hw}  spec.containers{nginx}       Normal          Created         Created container with docker id d2d15140a74a; Security:[seccomp=unconfined]
  40m           40m             1       {kubelet kubernetes-minion-group-54hw}  spec.containers{nginx}       Normal          Started         Started container with docker id d2d15140a74a
  16m           16m             4       {kubelet kubernetes-minion-group-54hw}                      Warning          FailedMount     MountVolume.SetUp failed for volume "kubernetes.io/secret/270e4f19-b5a4-11e6-8922-42010a800002-default-token-xim26" (spec.Name: "default-token-xim26") pod "270e4f19-b5a4-11e6-8922-42010a800002" (UID: "270e4f19-b5a4-11e6-8922-42010a800002") with: Get https://kubernetes-master/api/v1/namespaces/default/secrets/default-token-xim26: dial tcp: lookup kubernetes-master on 169.254.169.254:53: no such host
  16m           16m             1       {kubelet kubernetes-minion-group-54hw}                      Warning          FailedMount     MountVolume.SetUp failed for volume "kubernetes.io/secret/270e4f19-b5a4-11e6-8922-42010a800002-default-token-xim26" (spec.Name: "default-token-xim26") pod "270e4f19-b5a4-11e6-8922-42010a800002" (UID: "270e4f19-b5a4-11e6-8922-42010a800002") with: Get https://kubernetes-master/api/v1/namespaces/default/secrets/default-token-xim26: dial tcp 10.128.0.2:443: i/o timeout
  15m           15m             3       {kubelet kubernetes-minion-group-54hw}                      Warning          FailedMount     MountVolume.SetUp failed for volume "kubernetes.io/secret/270e4f19-b5a4-11e6-8922-42010a800002-default-token-xim26" (spec.Name: "default-token-xim26") pod "270e4f19-b5a4-11e6-8922-42010a800002" (UID: "270e4f19-b5a4-11e6-8922-42010a800002") with: Get https://kubernetes-master/api/v1/namespaces/default/secrets/default-token-xim26: dial tcp 10.128.0.2:443: getsockopt: connection refused
  9m            9m              1       {kubelet kubernetes-minion-group-54hw}                      Warning          FailedSync      Error syncing pod, skipping: network is not ready: [Kubenet does not have netConfig. This is most likely due to lack of PodCIDR]
  9m            9m              3       {kubelet kubernetes-minion-group-54hw}                      Warning          FailedSync      Error syncing pod, skipping: network is not ready: [Kubenet does not have netConfig. This is most likely due to lack of PodCIDR]
  6m            14s             4       {kubelet kubernetes-minion-group-54hw}                      Warning          FailedMount     Unable to mount volumes for pod "web-1_default(270e4f19-b5a4-11e6-8922-42010a800002)": timeout expired waiting for volumes to attach/mount for pod "web-1"/"default". list of unattached/unmounted volumes=[www]
  6m            14s             4       {kubelet kubernetes-minion-group-54hw}                      Warning          FailedSync      Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "web-1"/"default". list of unattached/unmounted volumes=[www]

steveperry-53 · 2016-11-28T22:03:19Z

I'll start a writing review today, and provide feedback tomorrow.

steveperry-53 · 2016-11-29T17:43:53Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+---
+
+{% capture overview %}
+This page shows how to upgrade from Pet Sets (Kubernetes cluster at version 1.3-1.4) to Stateful Sets (version >= 1.5).


Janet, We recently put together a style guide, and I realize now that I need to add some content to the style guide about API names. I've done that in PR 1807. The idea is that we want to refer to API objects by using the exact API object name. So for example, we would say PetSet, PetSets, StatefulSet, StatefulSets, etc.

steveperry-53 · 2016-11-29T17:45:46Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+{% capture prerequisites %}
+
+* If you don't have Pet Sets in your current cluster, or you don't plan to upgrade your cluster/master to >= 1.5 at this moment, you can skip this task now. 


Suggestion: upgrade your master to Kubernetes version 1.5 or later, you can skip this task.
Similar suggestion applies to all instances of >=1.5.

steveperry-53 · 2016-11-29T17:52:33Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+2. Change `kind` from `PetSet` to `StatefulSet`
+3. If you have the debug hook annotation `pod.alpha.kubernetes.io/initialized` set to `true`, you may remove it since it's redundant. If you don't have this annotation, you should add one, with the value set to `false`, to pause Stateful Sets operations.
+
+It's recommended that you keep both Pet Set manifests and Stateful Set manifests, so that you may safely rollback and recreate your Pet Sets, 


rollback -> roll back

steveperry-53 · 2016-11-29T17:55:04Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+Note that if the flag wasn't set, **cascading deletion will be performed by default**, and all pods managed by your Pet Sets will be gone. 
+
+You may delete those Pet Sets by specifying file names. This only works when in
+the files there are only Pet Sets, but not other resources (such as Services):


Suggestion: This only works when the files contain only PetSets, but not other resources such as Services.

steveperry-53 · 2016-11-29T17:59:54Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+1. Change `apiVersion` from `apps/v1alpha1` to `apps/v1beta1`
+2. Change `kind` from `PetSet` to `StatefulSet`
+3. If you have the debug hook annotation `pod.alpha.kubernetes.io/initialized` set to `true`, you may remove it since it's redundant. If you don't have this annotation, you should add one, with the value set to `false`, to pause Stateful Sets operations.


Suggestion: you can remove it because

Generally, we use these conventions:
may: permission
might: possibility
can: able to
because: why
since: from some time in the past

Similar suggestion applies to several spots in the document.

Fixed all usage by following the conventions.

Could the conventions be included in the style guide?

I'll look into adding these to the style guide.

steveperry-53 · 2016-11-29T18:03:54Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+Now, you may [upgrade your Kubernetes master](/docs/admin/cluster-management/#upgrading-a-cluster) to >= 1.5.0.
+Note that Kubernetes nodes should NOT be upgraded at this point, since the pods
+(that were once managed be Pet Sets) are now vulnerable to node failures. 


steveperry-53 · 2016-11-29T18:05:14Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+#### Create Stateful Sets
+
+Make sure you have both master and `kubectl` upgrade to a version >= 1.5 before


upgrade -> upgraded

steveperry-53 · 2016-11-29T18:06:19Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+#### Create Stateful Sets
+
+Make sure you have both master and `kubectl` upgrade to a version >= 1.5 before
+continuing. To verify this, run this command to make sure both client and server


Consider making the terminology in this sentence more consistent. On the one hand, we have (master, kubectl), and on the other hand we have (server, client).

This is because the command output only says "Client Version" and "Server Version". I updated this part to make it more clear.

janetkuo · 2016-11-29T23:17:07Z

Thanks @steveperry-53 for the review. All comments addressed. PTAL

enisoc · 2016-11-30T21:09:38Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+#### Upgrade master to Kubernetes version 1.5 or later
+
+Now, you can [upgrade your Kubernetes master](/docs/admin/cluster-management/#upgrading-a-cluster) to Kubernetes version 1.5 or later.
+Note that Kubernetes nodes should NOT be upgraded at this point, since the Pods


This is pretty important, so maybe bold it. Also active voice will help to clarify and emphasize this point:

Note that **you should not upgrade Nodes at this time**, because ...

enisoc · 2016-11-30T21:13:23Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+The output is similar to this:
+
+```shell
+Client Version: version.Info{Major:"1", Minor:"5+", GitVersion:"v1.5.0-beta.2.2+f64c9f2d999ceb-dirty", GitCommit:"f64c9f2d999ceb157d5672e9bba6639a4c456f6e", GitTreeState:"dirty", BuildDate:"2016-11-29T22:49:45Z", GoVersion:"go1.7.3", Compiler:"gc", Platform:"linux/amd64"}


Here's one with a cleaner GitVersion and non-dirty tree state:

Client Version: version.Info{Major:"1", Minor:"5+", GitVersion:"v1.5.0-beta.2", GitCommit:"0776eab45fe28f02bbeac0f05ae1a203051a21eb", GitTreeState:"clean", BuildDate:"2016-11-24T22:35:03Z", GoVersion:"go1.7.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"5+", GitVersion:"v1.5.0-beta.2", GitCommit:"0776eab45fe28f02bbeac0f05ae1a203051a21eb", GitTreeState:"clean", BuildDate:"2016-11-24T22:30:23Z", GoVersion:"go1.7.3", Compiler:"gc", Platform:"linux/amd64"}

You could also fake it by changing 5+ to 5 and removing the -beta.2 since these docs are supposed to target the final release. Users may be confused to see the beta stuff.

enisoc · 2016-11-30T21:14:10Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+#### Upgrade nodes to Kubernetes version 1.5 or later (optional)
+
+You can now [upgrading Kubernetes nodes](/docs/admin/cluster-management/#upgrading-a-cluster)


s/upgrading/upgrade/

enisoc · 2016-11-30T23:27:11Z

LGTM. Do you plan to fill in the TODOs before merging, or in a later PR?

janetkuo · 2016-11-30T23:36:24Z

Do you plan to fill in the TODOs before merging, or in a later PR?

I plan to do it later, unless those docs exist in the branch

steveperry-53 · 2016-11-30T23:50:33Z

_data/tasks.yml

-
+- title: Managing Stateful Applications
+  section:
+  - title: Upgrade from Pet Sets to Stateful Sets


For consistency with other titles, I suggest this title:
Upgrading from PetSets to StatefulSets

If you change the title here, also change it in index.md.

steveperry-53 · 2016-11-30T23:53:29Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+PetSet was introduced as an alpha resource in Kubernetes release 1.3, and was renamed to StatefulSet as a beta resource in 1.5. 
+Here are some notable changes:
+
+* **StatefulSet is the new PetSet**: PetSet is no longer available in Kubernetes release 1.5 or later. It becomes beta StatefulSet. To know why the name is changed, see this [discussion thread](https://github.com/kubernetes/kubernetes/issues/27430).


Suggestion:
"To know why the name is changed" -> "To understand why the name was changed"

steveperry-53 · 2016-11-30T23:55:06Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+### Upgrading from PetSets to StatefulSets
+
+Note that these steps need to be operated in the specified order. You **should


"need to be operated" -> "need to be done"

steveperry-53 · 2016-11-30T23:57:30Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+
+#### Prepare StatefulSet manifests 
+
+Now, for every PetSet manifest you have, prepare a corresponding StatefulSet manifest with the following steps: 


Suggestion: Strike "with the following steps."

... prepare a corresponding StatefulSet manifest:

steveperry-53 · 2016-11-30T23:59:00Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+Here are some notable changes:
+
+* **StatefulSet is the new PetSet**: PetSet is no longer available in Kubernetes release 1.5 or later. It becomes beta StatefulSet. To know why the name is changed, see this [discussion thread](https://github.com/kubernetes/kubernetes/issues/27430).
+* **StatefulSet guards against split brain**: StatefulSets guarantee at most one Pod for a given ordinal index can be running anywhere in a cluster, to guard against split brain scenarios with distributed applications. *TODO: Link to doc about fencing*


For the entire doc, put periods at the end of list items.

steveperry-53 · 2016-12-01T00:01:19Z

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md

+At this moment, you've deleted all PetSets in your cluster, but not their Pods, Persistent Volumes, or Persistent Volume Claims. 
+However, since the Pods are not managed by PetSets anymore, they will be vulnerable to node failures until you finish the master upgrade and recreate StatefulSets.
+
+#### Upgrade master to Kubernetes version 1.5 or later


"Upgrade master" -> "Upgrade your master"

steveperry-53 · 2016-12-01T00:03:34Z

I've added a few comments. Other than that, LGTM.

janetkuo · 2016-12-01T00:10:41Z

All comments addressed

steveperry-53 · 2016-12-01T17:57:58Z

Reviewed 1 of 4 files at r3, 1 of 3 files at r6, 3 of 3 files at r11.
Review status: all files reviewed at latest revision, 35 unresolved discussions.

Comments from Reviewable

steveperry-53 · 2016-12-01T18:00:04Z

@janetkuo Are you ready for me to merge this? Or would you like to wait until the 35 unresolved discussions in Reviewable are resolved?

janetkuo · 2016-12-01T23:08:16Z

Review status: all files reviewed at latest revision, 15 unresolved discussions.

_data/tasks.yml, line 28 at r10 (raw file):

Previously, steveperry-53 (Steve Perry) wrote…

For consistency with other titles, I suggest this title:
Upgrading from PetSets to StatefulSets

If you change the title here, also change it in index.md.

Done.

docs/tasks/index.md, line 26 at r1 (raw file):

Previously, kow3ns (Kenneth Owens) wrote…

PetSet should probably be Pet Sets

Changed back to PetSet according to new style guide

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md, line 37 at r7 (raw file):

Previously, enisoc (Anthony Yeh) wrote…

I think it makes sense to advise against upgrading to kubectl 1.5 until you're done acting on petsets. It's safer as a general rule - for example maybe you decide to delete some of your petsets WITH cascading since you don't care if they stay up throughout the upgrade process. If you had already upgraded to kubectl 1.5 it won't work the way you expected.

Done.

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md, line 31 at r10 (raw file):

Previously, steveperry-53 (Steve Perry) wrote…

Suggestion:
"To know why the name is changed" -> "To understand why the name was changed"

Done.

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md, line 32 at r10 (raw file):

Previously, steveperry-53 (Steve Perry) wrote…

For the entire doc, put periods at the end of list items.

Done.

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md, line 38 at r10 (raw file):

Previously, steveperry-53 (Steve Perry) wrote…

"need to be operated" -> "need to be done"

Done.

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md, line 65 at r10 (raw file):

Previously, steveperry-53 (Steve Perry) wrote…

Suggestion: Strike "with the following steps."

... prepare a corresponding StatefulSet manifest:

Done.

docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set.md, line 106 at r10 (raw file):

Previously, steveperry-53 (Steve Perry) wrote…

"Upgrade master" -> "Upgrade your master"

Done.

docs/tasks/stateful-sets/upgrade-pet-sets-to-stateful-sets.md, line 87 at r4 (raw file):

Previously, foxish (Anirudh Ramanathan) wrote…

Now, you want to recreate all Pet Sets, but turning them into Stateful Sets

This sentence seems a bit confusing. Maybe we can mention that we now create Stateful Sets objects which will adopt the pods belonging to the Pet Set objects formerly.

Done.

Comments from Reviewable

janetkuo · 2016-12-01T23:14:06Z

There are still 15 unresolved discussions left, even after I resolved them in Reviewable. All comments are addressed, but the discussions somehow won't go away. :/

@steveperry-53 we can just go ahead merging this.

Add Rodolfo Martinez to Spanish approver team

googlebot added the cla: yes label Nov 16, 2016

janetkuo added this to the 1.5 milestone Nov 16, 2016

k8s-github-robot assigned lavalamp Nov 16, 2016

janetkuo assigned erictune and unassigned lavalamp Nov 16, 2016

janetkuo mentioned this pull request Nov 16, 2016

Upgrade Path When PetSets are renamed kubernetes/kubernetes#35182

Closed

janetkuo commented Nov 16, 2016

View reviewed changes

enisoc reviewed Nov 16, 2016

View reviewed changes

janetkuo mentioned this pull request Nov 17, 2016

Rename PetSet to StatefulSet kubernetes/kubernetes#35534

Closed

11 tasks

janetkuo assigned enisoc Nov 17, 2016

enisoc reviewed Nov 17, 2016

View reviewed changes

erictune mentioned this pull request Nov 17, 2016

Pet Set in beta kubernetes/kubernetes#28718

Closed

enisoc approved these changes Nov 17, 2016

View reviewed changes

foxish reviewed Nov 18, 2016

View reviewed changes

enisoc reviewed Nov 18, 2016

View reviewed changes

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 19, 2016

idvoretskyi mentioned this pull request Nov 21, 2016

StatefulSets kubernetes/enhancements#137

Closed

23 tasks

devin-donnelly assigned steveperry-53 Nov 21, 2016

devin-donnelly added the Needs Docs Review label Nov 21, 2016

Add a task title for stateful sets

987c1c1

enisoc reviewed Nov 24, 2016

View reviewed changes

enisoc reviewed Nov 28, 2016

View reviewed changes

steveperry-53 reviewed Nov 29, 2016

View reviewed changes

enisoc reviewed Nov 30, 2016

View reviewed changes

steveperry-53 reviewed Nov 30, 2016

View reviewed changes

steveperry-53 reviewed Dec 1, 2016

View reviewed changes

steveperry-53 added Docs LGTM and removed Needs Docs Review labels Dec 1, 2016

Add Task: upgrading from PetSet to StatefulSet

1d9789c

janetkuo added the Tech Review LGTM label Dec 1, 2016

steveperry-53 merged commit c7bead1 into kubernetes:release-1.5 Dec 1, 2016

Okabe-Junya pushed a commit to Okabe-Junya/website that referenced this pull request Dec 4, 2023

Merge pull request kubernetes#1704 from electrocucaracha/add_ramrodo

71d092a

Add Rodolfo Martinez to Spanish approver team


		### Differences between alpha Pet Sets and beta Stateful Sets

		Pet Set was introduced as an alpha resource in Kubernetes release 1.3, and was renamed to Stateful Set as a beta resource in 1.5.


		At this moment, you've deleted all Pet Sets in your cluster, but not their pods, persistent volumes, or persistent volume claims.

		#### Upgrade your Kubernetes cluster to >= 1.5


		#### Create Stateful Sets

		Now, you want to recreate all Pet Sets, but turning them into Stateful Sets. To do this, just create all Stateful Set manifests you generate in the previous step.


		### Upgrading from Pet Sets to Stateful Sets

		#### Find all Pet Sets and their manifests


		#### Upgrade your Kubernetes cluster to >= 1.5

		Now, you may [upgrade your Kubernetes cluster](/docs/admin/cluster-management/#upgrading-a-cluster), or just your master, to >= 1.5.0.


		{% capture prerequisites %}

		* If you don't have Pet Sets in your current cluster, or you don't plan to upgrade your cluster/master to >= 1.5 at this moment, you can skip this task now.


		#### Create Stateful Sets

		Make sure you have both master and `kubectl` upgrade to a version >= 1.5 before


		#### Upgrade nodes to Kubernetes version 1.5 or later (optional)

		You can now [upgrading Kubernetes nodes](/docs/admin/cluster-management/#upgrading-a-cluster)


		### Upgrading from PetSets to StatefulSets

		Note that these steps need to be operated in the specified order. You **should


		#### Prepare StatefulSet manifests

		Now, for every PetSet manifest you have, prepare a corresponding StatefulSet manifest with the following steps:

Add Task: upgrading from PetSet to StatefulSet #1704

Add Task: upgrading from PetSet to StatefulSet #1704

Conversation

janetkuo commented Nov 16, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janetkuo Nov 17, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janetkuo Nov 17, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kow3ns commented Nov 17, 2016

janetkuo commented Nov 17, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enisoc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

foxish Nov 18, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enisoc commented Nov 24, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

enisoc commented Nov 28, 2016

steveperry-53 commented Nov 28, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

steveperry-53 Nov 29, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

steveperry-53 Nov 29, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janetkuo commented Nov 16, 2016 •

edited

janetkuo Nov 17, 2016 •

edited

janetkuo Nov 17, 2016 •

edited

foxish Nov 18, 2016 •

edited

steveperry-53 Nov 29, 2016 •

edited

steveperry-53 Nov 29, 2016 •

edited