Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

label/taint UPI nodes for redhat-cop/declarative-openshift#32 #33

Merged
merged 1 commit into from
Dec 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ Result Success!
In some cases, a cluster administrator might have a need to apply a patch to a resource that already exists or is owned by some other process. Some use cases of this are:

- Labelling the `default`, `kube-system`, or other "out of the box" namespaces
- Labelling nodes not managed by an operator
- [Labelling or tainting nodes not managed by an operator (UPI)](/optional-bootstrap/0-nodes/README.md)

For these cases, we use the [Resource Locker Operator](https://github.com/redhat-cop/resource-locker-operator#resource-patch-locking) to provide a "declarative patch" that will be kept in place by the operator. Building this solution in a declarative way involves creating the following components:

Expand Down
29 changes: 29 additions & 0 deletions optional-bootstrap/0-nodes/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@

## Patching Nodes (UPI)

During a UPI (user provisioned infrastructure) install of OpenShift it could be appropriate to [label](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#step-one-attach-label-to-the-node) or [taint](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) a node according to expected workloads in a declarative manner.

Since this is not natively supported by OpenShift [yet](https://github.com/openshift/machine-config-operator/pull/845), we'll use the [Resource Locker Operator](https://github.com/redhat-cop/resource-locker-operator#resource-patch-locking) to provide a "declarative patch" that will be kept in place by the operator.

Example patches are provided below:

Prerequisite:
- A [manifest](rlo-node-rbac.yaml) defining the `ServiceAccount`, `ClusterRole`, and `RoleBinding` (or `ClusterRoleBinding`) with the appropriate permissions that will perform the patch

Patch Manifests:
- A [node label patch](rlo-node-label.yaml) defining the `ResourceLocker` manifest to enforce the label
- A [node label taint](rlo-node-taint.yaml) defining the `ResourceLocker` manifest to enforce the taint

After running this, we can see that our targeted node now has a label and taint on it.
```
metadata:
labels:
workload: production
...
spec:
...
taints:
- effect: NoSchedule
key: redhatcop.redhat.io/productionworkload
...
```
24 changes: 24 additions & 0 deletions optional-bootstrap/0-nodes/rlo-node-label.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: redhatcop.redhat.io/v1alpha1
kind: ResourceLocker
metadata:
name: node-patch-label
namespace: default
annotations:
config.example.com/managed-by: gitops
config.example.com/scm-url: git@github.com:redhat-cop/declarative-openshift.git
labels:
config.example.com/name: optional-bootstrap
spec:
serviceAccountRef:
name: node-patcher
patches:
- targetObjectRef:
apiVersion: v1
kind: Node
name: productionworkload.internal
patchTemplate: |
metadata:
labels:
workload: production
patchType: application/strategic-merge-patch+json
id: sa-label
54 changes: 54 additions & 0 deletions optional-bootstrap/0-nodes/rlo-node-rbac.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
config.example.com/managed-by: gitops
config.example.com/scm-url: git@github.com:redhat-cop/declarative-openshift.git
labels:
config.example.com/name: optional-bootstrap
config.example.com/component: rbac
name: node-patcher
rules:
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- get
- watch
- create
- update
- patch
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: node-patcher
namespace: default
annotations:
config.example.com/managed-by: gitops
config.example.com/scm-url: git@github.com:redhat-cop/declarative-openshift.git
labels:
config.example.com/name: optional-bootstrap
config.example.com/component: rbac
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: node-patcher
annotations:
config.example.com/managed-by: gitops
config.example.com/scm-url: git@github.com:redhat-cop/declarative-openshift.git
labels:
config.example.com/name: optional-bootstrap
config.example.com/component: rbac
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: node-patcher
subjects:
- kind: ServiceAccount
name: node-patcher
namespace: default
25 changes: 25 additions & 0 deletions optional-bootstrap/0-nodes/rlo-node-taint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
apiVersion: redhatcop.redhat.io/v1alpha1
kind: ResourceLocker
metadata:
name: node-patch-taint
namespace: default
annotations:
config.example.com/managed-by: gitops
config.example.com/scm-url: git@github.com:redhat-cop/declarative-openshift.git
labels:
config.example.com/name: optional-bootstrap
spec:
serviceAccountRef:
name: node-patcher
patches:
- targetObjectRef:
apiVersion: v1
kind: Node
name: productionworkload.internal
patchTemplate: |
spec:
taints:
- effect: NoSchedule
key: redhatcop.redhat.io/productionworkload
patchType: application/strategic-merge-patch+json
id: sa-nodetaint