Skip to content

Commit

Permalink
ref(docs): update uninstall guide
Browse files Browse the repository at this point in the history 
+ update uninstall guide
+ update uninstall troubleshooting guide
+ resolves openservicemesh#2880

Signed-off-by: Michelle Noorali <minooral@microsoft.com>
  • Loading branch information
Michelle Noorali committed Mar 26, 2021
1 parent aa87b2b commit 7defd42
Show file tree
Hide file tree
Showing 2 changed files with 126 additions and 27 deletions.
134 changes: 113 additions & 21 deletions docs/content/docs/install/uninstallation_guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,44 +8,136 @@ weight: 3

# Uninstallation Guide

This guide describes how to uninstall Open Service Mesh (OSM) from a Kubernetes cluster using the `osm` CLI.
This guide describes how to uninstall Open Service Mesh (OSM) from a Kubernetes cluster. This guide assumes there is a single OSM control plane (mesh) running. If there are multiple meshes in a cluster, repeat the process described for each control plane in the cluster before uninstalling any cluster wide resources at the end of the guide. Taking into consideration both the control plane and dataplane, this guide aims to walk through uninstalling all reminents of OSM with minimal downtime.

## Prerequisites

- Kubernetes cluster with OSM installed
- The osm CLI
- The `kubectl` CLI
- The `osm` CLI

## Uninstall OSM
## Remove Envoy Sidecars from Application Pods and Envoy Secrets

Use the `osm` CLI to uninstall the OSM control plane from a Kubernetes cluster.
The first step to uninstalling OSM is to remove the Envoy sidecar containers from application pods. The sidecar containers enforce traffic policies. Without them, traffic will flow to and from Pods according in accordance with default Kubernetes networking unless there are [Kubernetes Network Policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) applied.

Run `osm mesh uninstall`.
OSM Envoy sidecars and related secrets will be removed in the following steps:

1. [Disable automatic sidecar injection](#disable-sidecar-injection)
1. [Restart pods](#restart-pods)
1. [Update Ingress Resources](#update-ingress-resources)
1. [Delete Envoy bootsrap secrets](#delete-envoy-bootsrap-secrets)

### Disable Automatic Sidecar Injection

OSM Automatic Sidecar Injection is most commonly enabled by adding namespaces to the mesh via the `osm` CLI. Use the `osm` CLI to see which
namespaces have sidecar injection enabled. If there are multiple control planes installed, be sure to specify the `--mesh-name` flag.

View namespaces in a mesh:

```console
# Uninstall osm control plane components
$ osm mesh uninstall
Uninstall OSM [mesh name: osm] ? [y/n]: y
OSM [mesh name: osm] uninstalled
$ osm namespace list --mesh-name=<mesh-name>
NAMESPACE MESH SIDECAR-INJECTION
<namespace1> <mesh-name> enabled
<namespace2> <mesh-name> enabled
```

Run `osm mesh uninstall --help` for more options.
Remove each namespace from the mesh:

```console
$ osm namespace remove <namespace> --mesh-name=<mesh-name>
Namespace [<namespace>] successfully removed from mesh [<mesh-name>]
```

Alternatively, if sidecar injection is enabled via annotations on pods instead of per namespace, please modify the pod or deployment spec to remove the sidecar injection annotation.

### Restart Pods

Restart all pods running with a sidecar:

```console
# If pods are running as part of a Kuberenetes deployment
# Can use this strategy for daemonset as well
$ kubectl rollout restart deployment <deployment-name> -n <namespace>

# If pod is running standalone (not part of a deployment or replica set)
$ kubectl delete pod <pod-name> -n namespace
$ k apply -f <pod-spec> # if pod is not restarted as part of replicaset
```

Now, there should be no OSM Envoy sidecar containers running as part of the applications that were once part of the mesh. Traffic is no
longer managed by the OSM control plane with the `mesh-name` used above. During this process, your applications may experience some downtime
as all the Pods are restarting.

### Update Ingress Resources

There may be ingress resources in the cluster that are configured to allow traffic from outside the cluster to an application that was
once part of the mesh. Identify these resources by using the `kubectl get ingress -n <namespace>` command in each namespace that was
removed from the mesh earlier.

If there are ingress resources and they are configured to allow HTTPS traffic, SSL related annotations will need to be update appropriately.

If ingress resources need to be reconfigured, this may result in some down time of traffic from outside the cluster not being able to get to the applications.

### Delete Envoy Bootsrap Secrets

## Resource Management
Once the sidecar is removed, there is no need for the Envoy bootstrap config secrets OSM created. These are stored in the application namespace and can be deleted manually with `kubectl`.

The following sections detail which Kubernetes resources are cleaned up and which remain after uninstalling OSM.
## Uninstall OSM Control Plane and Remove User Provided Resources

### Removed during OSM uninstallation
The OSM control plane and related components will be uninstalled in the following steps:

1. [Uninstall the OSM control plane](#uninstall-the-osm-control-plane)
1. [Remove User Provided Resources](#remove-user-provided-resources)
1. [Delete OSM Namespace](#delete-osm-namespace)

### Uninstall the OSM control plane

Use the `osm` CLI to uninstall the OSM control plane from a Kubernetes cluster. The following step will remove:

1. OSM controller resources (deployment, service, config map, and RBAC)
1. Prometheus, Grafana, Jaeger, and Fluentbit resources installed by OSM
1. Mutating webhook and validating webhook

### Remaining after OSM uninstallation
Run `osm mesh uninstall`:

```console
# Uninstall osm control plane components
$ osm mesh uninstall --mesh-name=<mesh-name>
Uninstall OSM [mesh name: <mesh-name>] ? [y/n]: y
OSM [mesh name: <mesh-name>] uninstalled
```

Run `osm mesh uninstall --help` for more options.

### Remove User Provided Resources

If any resources were provided or created for OSM at install time, they can be deleted at this point.

For example, if [Hashicorp Vault](https://github.com/openservicemesh/osm/blob/e1c48b9b1d67be2f5b7ec90af4572fa6d5d42fd2/docs/content/docs/tasks_usage/certificates.md#installing-hashi-vault) was deployed for the sole purpose of managing certificates for OSM, all related resources can be deleted.

### Delete OSM Namespace

When installing a mesh, the `osm` CLI creates the namespace the control plane is installed into if it does not already exist. However, when uninstalling the same mesh, the namespace it lives in does not automatically get deleted by the `osm` CLI. This behavior occurs because
there may be resources a user created in the namespace that they may not want automatically deleted.

If the namespace was only used for OSM and there is nothing that needs to be kept around, the namespace can be deleted at this time with `kubectl`.

```console
$ kubectl delete namespace <namespace>
namespace "<namespace>" deleted
```

Repeat the steps above for each mesh installed in the cluster. After there are no OSM control planes remaining, move to following step.

## Remove OSM Cluster Wide resources

OSM ensures that there the Service Mesh Interface Custom Resource Definitions(CRDs) exist in the cluster at install time. If they are not already installed, the `osm` CLI will install them before installing the rest of the control plane components. This is the same behvior when using the Helm charts to install OSM as well. CRDs are cluster-wide resources and may be used by other instances of OSM in the same cluster
or other service meshes running in the same cluster. If there are no other instances of OSM or other service meshes running in the same
cluster, these CRDs and instances of the SMI custom resources can be removed from the cluster using `kubectl`. When the CRD is deleted, all
instances of that CRD will also be deleted.

Run the following `kubectl` commands:

1. Existing Envoy sidecar containers
- Redeploy application pods to delete sidecars
1. Envoy bootstrap config secrets (stored in the application namespace)
1. Namespace annotations, including but not limited to `openservicemesh.io/monitored-by`
1. Custom resource definitions ([CRDs](https://github.com/openservicemesh/osm/tree/main/charts/osm/crds))
1. Vault resources provided by the user
1. The namespace in which OSM was installed
kubectl delete -f [https://raw.githubusercontent.com/openservicemesh/osm/v0.8.0/charts/osm/crds/access.yaml](https://raw.githubusercontent.com/openservicemesh/osm/v0.8.0/charts/osm/crds/access.yaml)
kubectl delete -f [https://raw.githubusercontent.com/openservicemesh/osm/v0.8.0/charts/osm/crds/specs.yaml](https://raw.githubusercontent.com/openservicemesh/osm/v0.8.0/charts/osm/crds/specs.yaml)
kubectl delete -f [https://raw.githubusercontent.com/openservicemesh/osm/v0.8.0/charts/osm/crds/split.yaml](https://raw.githubusercontent.com/openservicemesh/osm/v0.8.0/charts/osm/crds/split.yaml)
19 changes: 13 additions & 6 deletions docs/content/docs/troubleshooting/uninstall/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,23 @@ type: docs
# OSM Uninstall Troubleshooting Guide

## Leaked Resources
If the [uninstallation guide](../../install/uninstallation_guide.md) was not followed, it is possible that resources could be leaked.

If the Helm release, OSM controller, or their respective namespaces are deleted, then the `osm` CLI won't be able to uninstall any remaining resources, particularly if they are cluster scoped.
During an improper or incomplete uninstallation, it is possible that OSM resources could be left behind in a Kubernetes cluster.

These leaked resources result in an error when trying to install a new mesh with the same name but different namespace.
For example, if the Helm release, OSM controller, or their respective namespaces are deleted, then the `osm` CLI won't be able to uninstall any remaining resources, particularly if they are cluster scoped.

```
Error: rendered manifests contain a resource that already exists. Unable to continue with install: ClusterRole "osm" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-namespace" must equal "osm-system2": current value is "osm-system"
As a result, one may see this error during a subsequent install of a new mesh with the same name but different namespace:

```console
Error: rendered manifests contain a resource that already exists. Unable to continue with install: ClusterRole "<mesh-name>" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-namespace" must equal "<new-namespace>": current value is "<old-namespace>"
```

In the `./scripts/cleanup` directory we have included a helper script to delete those leaked resources: `./scripts/cleanup/osm-cleanup.sh`
In the case of this error, use the [cleanup script](https://github.com/openservicemesh/osm/blob/release-v0.8/scripts/cleanup/osm-cleanup.sh) located in the osm repository to delete any remaining resources.

To run the script, create a `.env` environment variable file to set the values specified at the top of the script. These values should match the values used to deploy the mesh.

In the root directory of the osm repository locally, run:

```console
./scripts/cleanup/osm-cleanup.sh
```

0 comments on commit 7defd42

Please sign in to comment.