Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update docs on the operator #726

Merged
merged 18 commits into from
Apr 8, 2024
Merged
Changes from 4 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 59 additions & 41 deletions docs/040_pepr-tutorials/030_create-pepr-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Go to the `capabilities` directory, create a new directory called `crd` with two
mkdir -p capabilities/crd/generated capabilities/crd/source
```

Generate a class based on the WebApp CRD using `kubernetes-fluent-client` and store it in the generated directory.
Generate a class based on the WebApp CRD using `kubernetes-fluent-client`, this way we can react to the fields of the CRD in a type-safe way.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

```bash
npx kubernetes-fluent-client crd https://gist.githubusercontent.com/cmwylie19/69b765af5ab25af62696f3337df13687/raw/72f53db7ddc06fc8891dc81136a7c190bc70f41b/WebApp.yaml .
Expand All @@ -130,7 +130,9 @@ export class WebApp extends a.GenericKind {
}
```

in the `source` folder, create a file called `webapp.crd.ts` and add the following:
Move the updated file to `capabilities/crd/generated/webapp-v1alpha1.ts`.

in the `capabilities/crd/source` folder, create a file called `webapp.crd.ts` and add the following, we want the controller to automatically create the CRD when it starts.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

```typescript
export const WebAppCRD = {
Expand Down Expand Up @@ -212,28 +214,7 @@ export const WebAppCRD = {
};
```

In the root of the crd folder, create an `index.ts` file and add the following:

```typescript
import { V1OwnerReference } from "@kubernetes/client-node";
export { WebApp, Phase, Status } from "./generated/webapp-v1alpha1";
import { WebApp } from "./generated/webapp-v1alpha1";

export function getOwnerRef(instance: WebApp): V1OwnerReference[] {
const { name, uid } = instance.metadata!;

return [
{
apiVersion: instance.apiVersion!,
kind: instance.kind!,
uid: uid!,
name: name!,
},
];
}
```

Add a `register.ts` file to the `crd` folder and add the following:
Add a `register.ts` file to the `capabilities/crd/` folder and add the following, this will auto register the CRD on startup.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

```typescript
import { K8s, Log, kind } from "pepr";
Expand All @@ -250,15 +231,14 @@ export const RegisterCRD = () => {
});
};
(() => RegisterCRD())();

```

Finally add a `validate.ts` file to the `crd` folder and add the following:
Finally add a `validate.ts` file to the `crd` folder and add the following, this will ensure that instances of the WebApp resource are in valid namespaces and have a maximum of 7 replicas.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

```typescript
import { PeprValidateRequest } from "pepr";

import { WebApp } from ".";
import { WebApp } from "./generated/webapp-v1alpha1";

const invalidNamespaces = [
"kube-system",
Expand All @@ -285,19 +265,20 @@ export async function validator(req: PeprValidateRequest<WebApp>) {
}
```

In this section we generated the CRD class for WebApp, created a function to add `ownerReferences` to the manifests that will be deployed by the Operator to handle deletion of Kubernetes objects, registered the CRD, and added a validator to validate that instances of WebApp are in valid namespaces.
In this section we generated the CRD class for WebApp and created a function to auto register the CRD, and added a validator to validate that instances of WebApp are in valid namespaces and have a maximum of 7 replicas.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

## Create Helpers

In this section we will create helper functions to help with the reconciliation process.
In this section we will create helper functions to help with the reconciliation process. The idea is that this operator will "remedy" any accidental deletions of the resources it creates. If any object deployed by the Operator is deleted for any reason, the Operator will abruptly redeploy the object.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

Create a `controller` folder in the `capabilities` folder and create a `generators.ts` file in the `capabilities` folder. This file will contain the functions that will generate the manifests that will be deployed by the Operator with the ownerReferences added to them.
Create a `controller` folder in the `capabilities` folder and create a `generators.ts` file. This file will contain the functions that will generate the manifests that will be deployed by the Operator with the ownerReferences added to them. Since these resources are owned by the WebApp resource, they will be deleted when the WebApp resource is deleted.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

```typescript
import { kind, K8s, Log } from "pepr";
import { getOwnerRef } from "../crd";
import { kind, K8s, Log, sdk } from "pepr";
import { WebApp } from "../crd/generated/webapp-v1alpha1";

const { getOwnerRefFrom } = sdk;

export default async function Deploy(instance: WebApp) {
try {
await Promise.all([
Expand All @@ -322,7 +303,7 @@ function deployment(instance: WebApp) {
apiVersion: "apps/v1",
kind: "Deployment",
metadata: {
ownerReferences: getOwnerRef(instance),
ownerReferences: getOwnerRefFrom(instance),
name,
namespace,
labels: {
Expand All @@ -338,7 +319,7 @@ function deployment(instance: WebApp) {
},
template: {
metadata: {
ownerReferences: getOwnerRef(instance),
ownerReferences: getOwnerRefFrom(instance),
annotations: {
buildTimestamp: `${Date.now()}`,
},
Expand Down Expand Up @@ -384,7 +365,7 @@ function service(instance: WebApp) {
apiVersion: "v1",
kind: "Service",
metadata: {
ownerReferences: getOwnerRef(instance),
ownerReferences: getOwnerRefFrom(instance),
name,
namespace,
labels: {
Expand Down Expand Up @@ -614,7 +595,7 @@ function configmap(instance: WebApp) {
apiVersion: "v1",
kind: "ConfigMap",
metadata: {
ownerReferences: getOwnerRef(instance),
ownerReferences: getOwnerRefFrom(instance),
name: `web-content-${name}`,
namespace,
labels: {
Expand All @@ -628,18 +609,23 @@ function configmap(instance: WebApp) {
}
```

Our job is to make the deployment of the WebApp simple. Instead of having to keep track of the versions and revisions to all of the Kubernetes manifest required for the WebApp, and rolling pods and updating configMaps, the deployer now only needs to focus on the `WebApp` instance. The controller will reconcile instances of the operand (WebApp) against the actual cluster state to reach the desired state.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

We decide which `ConfigMap` to deploy based on the language and theme specified in the WebApp resource and how many replicas to deploy based on the replicas specified in the WebApp resource.

## Create Reconciler

Now, we create the function that reacts to changes across instances of the WebApp instances. This function will be called and put into a Queue, guaranteeing ordered and synchronous processing of events, even when the system may be under heavy load.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved

In the base of the `capabilities` folder, create a `reconciler.ts` file and add the following:

```typescript
import { K8s, Log } from "pepr";

import { K8s, Log, sdk } from "pepr";
import Deploy from "./controller/generators";
import { Phase, Status, WebApp } from "./crd";

const { writeEvent } = sdk;

/**
* The reconciler is called from the queue and is responsible for reconciling the state of the instance
* with the cluster. This includes creating the namespace, network policies and virtual services.
Expand Down Expand Up @@ -691,6 +677,7 @@ export async function reconciler(instance: WebApp) {
* @param status The new status
*/
async function updateStatus(instance: WebApp, status: Status) {
await writeEvent(instance, {phase: status}, "Normal", "CreatedOrUpdate", instance.metadata.name, instance.metadata.name);
await K8s(WebApp).PatchStatus({
metadata: {
name: instance.metadata!.name,
Expand Down Expand Up @@ -734,10 +721,13 @@ When(WebApp)
}
});

// Remove the instance from the store BEFORE it is deleted so reconcile stops
// and a cascading deletion occurs for all owned resources.
// To make this work, we extended the timeout on the WebHook Configuration
When(WebApp)
.IsDeleted()
.Mutate(instance => {
Store.removeItemAndWait(instance.Raw.metadata.name);
.Mutate(async instance => {
await Store.removeItemAndWait(instance.Raw.metadata.name);
});

// Don't let the CRD get deleted
Expand Down Expand Up @@ -778,6 +768,9 @@ When(a.ConfigMap)
});

```
- When a WebApp is created or updated, validate it, store the name of the instance and enqueue it for processing.
- If an "owned" resources (ConfigMap, Service, or Deployment) is deleted, redeploy it, check and see if it was owned by a WebApp instance and redeploy it if it was.
cmwylie19 marked this conversation as resolved.
Show resolved Hide resolved
- Always redeploy the WebApp CRD if it was deleted as the controller depends on it

In this section we created a `reconciler.ts` file that contains the function that is responsible for reconciling the state of the instance with the cluster based on CustomResource and updating the status of the instance. The `index.ts` file that contains the WebAppController capability and the functions that are used to watch for changes to the WebApp resource and corresponding Kubernetes resources. The `Reconcile` action processes the callback in a queue guaranteeing ordered and synchronous processing of events

Expand All @@ -798,7 +791,7 @@ Make sure Pepr is update to date
npx pepr update
```

Build the Pepr manifests
Build the Pepr manifests (Already built with appropriate RBAC)
schaeferka marked this conversation as resolved.
Show resolved Hide resolved

```bash
npx pepr build
Expand Down Expand Up @@ -889,6 +882,31 @@ kubectl get wa webapp-light-en -n webapps -ojsonpath="{.status}" | jq
}
```

Describe the WebApp to look at events

```bash
kubectl describe wa webapp-light-en -n webapps
# output
Name: webapp-light-en
Namespace: webapps
API Version: pepr.io/v1alpha1
Kind: WebApp
Metadata: ...
Spec:
Language: en
Replicas: 1
Theme: light
Status:
Observed Generation: 1
Phase: Ready
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal InstanceCreatedOrUpdated 36s webapp-light-en Pending
Normal InstanceCreatedOrUpdated 36s webapp-light-en Ready

```

Port-forward and look at the WebApp in the browser

```bash
Expand Down
Loading