diff --git a/docs/addon/walkthrough/README.md b/docs/addon/walkthrough/README.md index 51bdf8d2..ec5c6efc 100644 --- a/docs/addon/walkthrough/README.md +++ b/docs/addon/walkthrough/README.md @@ -1,6 +1,6 @@ ## Walkthrough: Creating a new Operator -This walkthrough is for creating an operator to run the kubernetes dashboard. +This walkthrough is for creating an operator to run the [guestbook](https://github.com/kubernetes/examples/tree/master/guestbook) which is an example application for kubernetes. ### Basics @@ -16,8 +16,8 @@ Create a new directory and use kubebuilder to scafold the operator: ``` export GO111MODULE=on -mkdir -p dashboard-operator/ -cd dashboard-operator/ +mkdir -p guestbook-operator/ +cd guestbook-operator/ kubebuilder init --domain example.org --license apache2 --owner "TODO($USER): assign copyright" ``` @@ -31,7 +31,7 @@ go get sigs.k8s.io/kubebuilder-declarative-pattern ``` # generate the API/controllers -kubebuilder create api --controller=true --example=false --group=addons --kind=Dashboard --make=false --namespaced=true --resource=true --version=v1alpha1 +kubebuilder create api --controller=true --example=false --group=addons --kind=Guestbook --make=false --namespaced=true --resource=true --version=v1alpha1 # remove the test suites that are more checking that kubebuilder is working find . -name "*_test.go" -delete ``` @@ -43,7 +43,7 @@ controller under `controllers/` * Generate code: `make generate` * You should now be able to `go run main.go` (or `make run`), - though it will exit with an error from being unable to find the dashboard CRD. + though it will exit with an error from being unable to find the guestbook CRD. ### Adding a manifest @@ -58,7 +58,7 @@ changing namespaces, and tweaking flags) Some other advantages: -* Working with manifests lets us release a new dashboard version without needing +* Working with manifests lets us release a new guestbook version without needing a new operator version * The declarative manifest makes it easier for users to understand what is changing in each version @@ -69,8 +69,8 @@ For now, we embed the manifests into the image, but we'll be evolving this, for Create a manifest under `channels/packages///manifest.yaml` ```bash -mkdir -p channels/packages/dashboard/1.8.3/ -wget -O channels/packages/dashboard/1.8.3/manifest.yaml https://raw.githubusercontent.com/kubernetes/dashboard/v1.8.3/src/deploy/recommended/kubernetes-dashboard.yaml +mkdir -p channels/packages/guestbook/0.1.0/ +wget -O channels/packages/guestbook/0.1.0/manifest.yaml https://raw.githubusercontent.com/kubernetes/examples/master/guestbook/all-in-one/guestbook-all-in-one.yaml ``` We have a notion of "channels", which is a stream of updates. We'll have @@ -83,8 +83,8 @@ We need to define the default stable channel, so create `channels/stable`: ```bash cat > channels/stable </dashboard-operator:latest +IMG ?= gcr.io//guestbook-operator:latest ``` -1. Create a patch to modify the memory limit for the operator: +2. Create a patch to modify the memory limit for the operator: ```bash cat << EOF > config/default/manager_resource_patch.yaml @@ -358,7 +348,7 @@ spec: EOF ``` -1. Reference the patch by adding `manager_resource_patch.yaml` to the `patches` section of `config/default/kustomization.yaml`: +3. Reference the patch by adding `manager_resource_patch.yaml` to the `patches` section of `config/default/kustomization.yaml`: ```yaml patches: @@ -368,7 +358,7 @@ patches: This is requried to run kubectl in the container. -1. Modify the `Dockerfile` to pull in kubectl, the manifests (in `channels/`), +4. Modify the `Dockerfile` to pull in kubectl, the manifests (in `channels/`), and run in a slim container: ```Dockerfile @@ -379,10 +369,17 @@ RUN curl -fsSL https://dl.k8s.io/release/v1.13.4/bin/linux/amd64/kubectl > /usr/ RUN chmod a+rx /usr/bin/kubectl # Build the manager binary -FROM golang:1.10.3 as builder +FROM golang:1.13 as builder # Copy in the go src -WORKDIR /go/src/dashboard-operator +WORKDIR /go/src/guestbook-operator +# Copy the Go Modules manifests +COPY go.mod go.mod +COPY go.sum go.sum +# cache deps before building and copying source so that we don't need to re-download as much +# and so that source changes don't invalidate our downloaded layer +RUN go mod download + COPY vendor/ vendor/ COPY main.go main.go COPY api/ api/ @@ -394,13 +391,27 @@ RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o manager main.go # Copy the operator and dependencies into a thin image FROM gcr.io/distroless/static:latest WORKDIR / -COPY --from=builder /go/src/dashboard-operator/manager . +COPY --from=builder /go/src/guestbook-operator/manager . COPY --from=kubectl /usr/bin/kubectl /usr/bin/kubectl COPY channels/ channels/ ENTRYPOINT ["./manager"] ``` -1. Verify everything worked by building and pushing the image: +5. Modify the `Makefile` to run `go mod vendor` for building container image. + +```make +... +# Add vendor prerequisites before test. +docker-build: vendor test + docker build . -t ${IMG} + +# Add vendor target. +vendor: + go mod vendor +... +``` + +6. Verify everything worked by building and pushing the image: ```bash make docker-build docker-push @@ -413,20 +424,20 @@ tightly-scoped RBAC role. To do that we use kubebuilder's RBAC role generation based off of source annotations. In the future we may be able to generate RBAC rules from the manfiest. -The RBAC rules are included in the `dashboard_controller.go` snippet you pasted above. +The RBAC rules are included in the `guestbook_controller.go` snippet you pasted above. RBAC is the real pain-point here - we end up with a lot of permissions: * The operator needs RBAC rules to see the CRDs. * It needs permission to get / create / update the Deployments and other types that it is managing -* It needs permission to create the ClusterRoles / Roles that the dashboard +* It needs permission to create the ClusterRoles / Roles that the guestbook needs * Because of that, we also need permissions for all the permissions we are going to create. The last one in particular can result in a non-trivial RBAC policy. My approach: -* Start with minimal permissions (just watching addons.k8s.io dashboards), and +* Start with minimal permissions (just watching addons.k8s.io guestbooks), and then add permissions iteratively * If you're going to allow list, I tend to just allow get, list and watch - there's not a huge security reason to treat them separately as far as I can @@ -436,7 +447,7 @@ The last one in particular can result in a non-trivial RBAC policy. My approach tend to grant that one begrudgingly * The RBAC policy in the manifest may scope down the permissions even more (for example scoping to resourceNames), in which case we can - and should - copy - it. That's what we did here for dashboard. + it. That's what we did here for guestbook. ### Installing the operator in the cluster @@ -448,27 +459,27 @@ make deploy You can troubleshoot the operator by inspecting the controller: ```bash -kubectl -n dashboard-operator-system get deploy -kubectl -n dashboard-operator-system logs manager +kubectl -n guestbook-operator-system get deploy +kubectl -n guestbook-operator-system logs manager ``` -### Create a dashboard CR +### Create a guestbook CR ```bash -kubectl apply -n kube-system -f config/samples/addons_v1alpha1_dashboard.yaml +kubectl apply -n kube-system -f config/samples/addons_v1alpha1_guestbook.yaml ``` You can verify the CR is created successfully: ``` -kubectl get DashBoards -n kube-system +kubectl get Guestbooks -n kube-system ``` -You can verify that the operator has created the `kubernetes-dashboard` +You can verify that the operator has created the `kubernetes-guestbook` deployment: -e.g. `kubectl get pods -l k8s-app=kubernetes-dashboard -n kube-system` or -`kubectl get deploy kubernetes-dashboard -n kube-system`. +e.g. `kubectl get pods -n guestbook-operator-system` or +`kubectl get deploy -n guestbook-operator-system`. ## Manifest simplification: Automatic labels @@ -481,11 +492,11 @@ Instead, the Reconciler can add labels to every object in the manifest: ```go labels := map[string]string{ - "k8s-app": "kubernetes-dashboard", + "example-app": "guestbook", } - r := &ReconcileDashboard{} - r.Reconciler.Init(mgr, &api.Dashboard{}, "dashboard", + r := &ReconcileGuestbook{} + r.Reconciler.Init(mgr, &api.Guestbook{}, "guestbook", declarative.WithObjectTransform(declarative.AddLabels(labels)), ... ) @@ -520,19 +531,19 @@ status that can be surfaced in various user interfaces. curl https://raw.githubusercontent.com/kubernetes-sigs/application/master/config/crds/app_v1beta1_application.yaml -o config/crd/app_v1beta1_application.yaml ``` -1. Add an instance of the Application CR in your manifest: +2. Add an instance of the Application CR in your manifest: ```bash - cat <> channels/packages/dashboard/1.8.3/manifest.yaml + cat <> channels/packages/guestbook/0.1.0/manifest.yaml # ------------------- Application ------------------- # apiVersion: app.k8s.io/v1beta1 kind: Application metadata: - name: kubernetes-dashboard + name: guestbook spec: descriptor: - type: "kubernetes-dashboard" - description: "Kubernetes Dashboard is a general purpose, web-based UI for Kubernetes clusters. It allows users to manage applications running in the cluster and troubleshoot them, as well as manage the cluster itself." + type: "guestbook" + description: "Guestbook is a simple, multi-tier web application using Kubernetes. This application consists of the following components: A single-instance Redis master to store guestbook entries, Multiple replicated Redis instances to serve reads, Multiple web frontend instances." icons: - src: "https://github.com/kubernetes/kubernetes/raw/master/logo/logo.png" type: "image/png" @@ -541,17 +552,19 @@ status that can be surfaced in various user interfaces. email: maintainer@example.org keywords: - "addon" - - "dashboard" + - "guestbook" links: - - description: Project Homepage - url: "https://github.com/kubernetes/dashboard" + - description: Guide Document + url: "https://kubernetes.io/docs/tutorials/stateless-application/guestbook/" + - description: Source Code + url: "https://github.com/kubernetes/examples/tree/master/guestbook" EOF ``` -1. Add the two options for managing the Application to your controller: +3. Add the two options for managing the Application to your controller: ```go - r.Reconciler.Init(mgr, &api.Dashboard{}, "dashboard", + r.Reconciler.Init(mgr, &api.Guestbook{}, ... declarative.WithManagedApplication(r.watchLabels), declarative.WithObjectTransform(addon.TransformApplicationFromStatus), @@ -559,7 +572,7 @@ status that can be surfaced in various user interfaces. ) ``` -1. Rebuild the operator, reinstall the CRDs, and start the new operator. You can now see the Application: +4. Rebuild the operator, reinstall the CRDs, and start the new operator. You can now see the Application: ```bash kubectl -n kube-system get applications -oyaml diff --git a/examples/guestbook-operator/Dockerfile b/examples/guestbook-operator/Dockerfile index 032ea054..15cd21ce 100644 --- a/examples/guestbook-operator/Dockerfile +++ b/examples/guestbook-operator/Dockerfile @@ -1,7 +1,14 @@ +FROM ubuntu:latest as kubectl +RUN apt-get update +RUN apt-get install -y curl +RUN curl -fsSL https://dl.k8s.io/release/v1.13.4/bin/linux/amd64/kubectl > /usr/bin/kubectl +RUN chmod a+rx /usr/bin/kubectl + # Build the manager binary FROM golang:1.13 as builder -WORKDIR /workspace +# Copy in the go src +WORKDIR /go/src/guestbook-operator # Copy the Go Modules manifests COPY go.mod go.mod COPY go.sum go.sum @@ -9,20 +16,18 @@ COPY go.sum go.sum # and so that source changes don't invalidate our downloaded layer RUN go mod download -# Copy the go source +COPY vendor/ vendor/ COPY main.go main.go COPY api/ api/ COPY controllers/ controllers/ -COPY vendor/ vendor/ # Build -RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager main.go +RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o manager main.go -# Use distroless as minimal base image to package the manager binary -# Refer to https://github.com/GoogleContainerTools/distroless for more details -FROM gcr.io/distroless/static:nonroot +# Copy the operator and dependencies into a thin image +FROM gcr.io/distroless/static:latest WORKDIR / -COPY --from=builder /workspace/manager . -USER nonroot:nonroot - -ENTRYPOINT ["/manager"] +COPY --from=builder /go/src/guestbook-operator/manager . +COPY --from=kubectl /usr/bin/kubectl /usr/bin/kubectl +COPY channels/ channels/ +ENTRYPOINT ["./manager"] diff --git a/examples/guestbook-operator/Makefile b/examples/guestbook-operator/Makefile index 472b05e9..3ae6d46b 100644 --- a/examples/guestbook-operator/Makefile +++ b/examples/guestbook-operator/Makefile @@ -55,6 +55,7 @@ fmt: vet: go vet ./... +# Run go mod vendor vendor: go mod vendor diff --git a/examples/guestbook-operator/controllers/guestbook_controller.go b/examples/guestbook-operator/controllers/guestbook_controller.go index be0e5d25..4e3a273a 100644 --- a/examples/guestbook-operator/controllers/guestbook_controller.go +++ b/examples/guestbook-operator/controllers/guestbook_controller.go @@ -58,9 +58,6 @@ func (r *GuestbookReconciler) setupReconciler(mgr ctrl.Manager) error { ) } -// +kubebuilder:rbac:groups=addons.example.org,resources=guestbooks,verbs=get;list;watch;create;update;patch;delete -// +kubebuilder:rbac:groups=addons.example.org,resources=guestbooks/status,verbs=get;update;patch - func (r *GuestbookReconciler) SetupWithManager(mgr ctrl.Manager) error { if err := r.setupReconciler(mgr); err != nil { return err @@ -89,5 +86,7 @@ func (r *GuestbookReconciler) SetupWithManager(mgr ctrl.Manager) error { // for WithApplyPrune // +kubebuilder:rbac:groups=*,resources=*,verbs=list +// +kubebuilder:rbac:groups=addons.example.org,resources=guestbooks,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=addons.example.org,resources=guestbooks/status,verbs=get;update;patch // +kubebuilder:rbac:groups="",resources=services,verbs=get;list;watch;create;update;delete;patch // +kubebuilder:rbac:groups=apps;extensions,resources=deployments,verbs=get;list;watch;create;update;delete;patch