diff --git a/Makefile b/Makefile index 62d6a6552..6dfb499dc 100644 --- a/Makefile +++ b/Makefile @@ -147,6 +147,7 @@ copyright: -f ./hack/boilerplate.go.txt \ -ignore site/static/\*\* \ -ignore site/content/docs/\*/crds/\*.yaml \ + -ignore site/content/docs/\*/tutorials/files/\*/\*.yaml \ -ignore site/themes/\*\* \ -ignore experimental/live-editor/node_modules/\*\* \ . diff --git a/site/content/docs/development/img/tutorials/hello-world-nginx.png b/site/content/docs/development/img/tutorials/hello-world-nginx.png new file mode 100644 index 000000000..227d9b12c Binary files /dev/null and b/site/content/docs/development/img/tutorials/hello-world-nginx.png differ diff --git a/site/content/docs/development/tutorials/extending-a-supply-chain.md b/site/content/docs/development/tutorials/extending-a-supply-chain.md new file mode 100644 index 000000000..a9cf1fe4b --- /dev/null +++ b/site/content/docs/development/tutorials/extending-a-supply-chain.md @@ -0,0 +1,503 @@ +# Extending a Supply Chain (_or_ Multiple Templates in a Supply Chain) + +## Overview + +So far our supply chains have been a bit anemic, single step affairs. In this tutorial we’ll explore two topics: + +- Adding new templates to an existing supply chain +- Passing information from one object created by a template to the template about to create another object + +## Environment setup + +For this tutorial you will need a kubernetes cluster with Cartographer and +[kpack](https://buildpacks.io/docs/tools/kpack/) installed. You can find +[Cartographer's installation instructions here](https://github.com/vmware-tanzu/cartographer#installation) and +[kpack's installation instructions can be found here](https://github.com/pivotal/kpack/blob/main/docs/install.md). + +You will also need an image registry for which you have read and write permission. + +Alternatively, you may choose to use the +[./hack/setup.sh](https://github.com/vmware-tanzu/cartographer/blob/main/hack/setup.sh) script to install a kind cluster +with Cartographer, kpack and a local registry. _This script is meant for our end-to-end testing and while we rely on it +working in that role, no user guarantees are made about the script._ + +Command to run from the Cartographer directory: + +```shell +$ ./hack/setup.sh cluster cartographer-latest example-dependencies +``` + +If you later wish to tear down this generated cluster, run + +```shell +$ ./hack/setup.sh teardown +``` + +## Scenario + +### App Operator Steps + +#### Supply Chain + +We’ll start by considering the supply chain from our [“Build Your First Supply Chain"](first-supply-chain.md) tutorial: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterSupplyChain +metadata: + name: supply-chain +spec: + selector: + workload-type: pre-built + + resources: + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy +``` + +Let’s think through what we want to change here. We know that we’re going to create a new step, so we’ll need a new +resource. This step is going to build an image, so it will come before the existing step that creates a deployment of +the image. For readability we’ll list the new resource before the current `deploy` resource. We’ll give the resource a +reasonable name. So far our `.spec.resources` will look like this: + + +```yaml + resources: + - name: build-image + templateRef: + kind: ??? + Name: ??? + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy +``` + + +So far we’ve only seen the template kind ClusterTemplate. But now we want to template an object that will pass +information to further templates in the supply chain. Cartographer expects three different types of information to be +passed through a supply chain. There are subsequently three template types that expose information to later resources in +a supply chain: + +- ClusterSourceTemplates expose location of source code +- ClusterImageTemplates expose location of images +- ClusterConfigTemplates expose yaml specification of k8s objects + +(Documentation of these custom resources [can be found here](../reference/template/)) + +In our scenario, we know that we’re going to template some object that will take the location of source code from the +workload, build an image and then we’ll want to share the location of that image with the next object in the supply +chain. As we want to expose the location of an image to the supply chain, we’ll use the ClusterImageTemplate and we’ll +give it a reasonable name. Our supply chain .spec.resources now looks like this: + + +```yaml + resources: + - name: build-image + templateRef: + kind: ClusterImageTemplate + name: image-builder + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy +``` + + +There’s one more addition we must make to the resources. While the build-image step will make information available for +consumption, we need to explicitly indicate that the deploy step will consume that information. We’ll do so by adding an +images field to that step. We’ll refer to the resource providing an image and give that value a name by which the +app-deploy template can refer to that value: + + +```yaml + resources: + - name: build-image + templateRef: + kind: ClusterImageTemplate + name: image-builder + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy + images: + - resource: build-image + name: built-image +``` + + +Our resources section is looking good. Before we move on to writing the templates, let’s take a moment to think about +our app platform. We previously had just one supply chain that worked for all of our devs that provided prebuilt images. +We’re in the process of adding a supply chain that accepts apps defined in source code. This new supply chain doesn't +also support the prebuilt images; the final deploy step of this supply chain has a dependency on the build-image step. +We need to make three changes: + +1. Give the supply chain a new name. +2. Give the supply chain different selector(s). +3. Change the template reference for the deploy step. + +The name change is straightforward: + +```yaml +metadata: + name: source-code-supply-chain +``` + +The selector change in similarly straightforward: + + +```yaml + selector: + workload-type: source-code +``` + + +Before changing the template reference, let’s take a moment to think about why the deploy step needs a new reference. In +the general case, it is completely fine for 2 supply chains to refer to common templates; that reusability is a feature +of Cartographer. But in this case, we know that the deploy step of our two supply chains have different dependencies. In +our original supply chain the deploy step depended only on the workload values. In our new supply chain we’ve declared +that the deploy step depends on values from the build-image step. This indicates to us that the templates will need to +differ. So we’ll need to write a new deploy template. We’ll refer to it in the supply chain: + + +```yaml + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy-from-sc-image + ... +``` + + +Finally, we'll need a new service account for this supply chain, one that has permission to create the objects in both +templates. We'll specify a name for that service account now and create it below (after completing our templates). + + +```yaml + serviceAccountRef: + name: cartographer-from-source-sa + namespace: default +``` + + +We can see our final supply chain defined here: + +{{< tutorial supply-chain.yaml >}} + +#### Templates + +Now we’re ready to define our templates. Let’s begin with the template for the deploy step, as we’re familiar with it +already. There’s only one field that will change; previously the image location was defined by the workload. + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + name: app-deploy +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: $(workload.metadata.name)$-deployment + labels: + app: $(workload.metadata.name)$ + spec: + replicas: 3 + selector: + matchLabels: + app: $(workload.metadata.name)$ + template: + metadata: + labels: + app: $(workload.metadata.name)$ + spec: + serviceAccountName: $(params.image-pull-sa-name)$ + containers: + - name: $(workload.metadata.name)$ + image: $(workload.spec.image)$ # <=== No longer the proper source + params: + - name: image-pull-sa-name + default: expected-service-account +``` + +Now the template expects that value to come from a previous step in the supply chain. So we’ll simply replace + + +```yaml + image: $(workload.spec.image)$ +``` + + +with + + +```yaml + image: $(images.built-image.image)$ +``` + + +Let’s break down that syntax. In the supply chain we specified that we were providing an array of images to this +template. So we start with `images`. In the supply chain we further declared that the one image in the array of images +would have the name "built-image", so we continue `images.built-image`. Finally, images provide a single value, an image +location (if we were providing sources, each source would provide both a url and a revision). So we complete the +reference: `images.built-image.image`. + +Finally, we'll give this ClusterTemplate a new name (already specified in the supply chain): + +```yaml +metadata: + name: app-deploy-from-sc-image +``` + +The template for our deploy step is complete: + +{{< tutorial app-deploy-template.yaml >}} + +Now we’re ready to create our new template, `image-builder`. We’re going to rely on kpack, a kubernetes native container +build service. We’ll need to template out a kpack Image object. Much of this will be familiar from the +[Build Your First Supply Chain](../first-supply-chain) tutorial. We’ll have a value (the location of the source code) +that is only known by the application developer. We’ll also have values that must remain unique among templated objects +in the name space, for which we’ll use the name of the workload. And similar to the tutorial on Using Params, we’ll +leverage params to provide a default image registry but allow devs to specify another image registry if they so desire. +Let’s look at the .spec.template field of our ClusterImageTemplate: + + +```yaml + template: + apiVersion: kpack.io/v1alpha2 + kind: Image + metadata: + name: $(workload.metadata.name)$ + spec: + tag: $(params.image_prefix)$$(workload.metadata.name)$ + serviceAccountName: $(params.image-pull-sa-name)$ + builder: + kind: ClusterBuilder + name: my-builder + source: + git: + url: $(workload.spec.source.git.url)$ + revision: $(workload.spec.source.git.ref.branch)$ +``` + + +Though learners may not be as familiar with kpack Images as they are with the Deployment resource, there’s nothing novel +here from a Cartographer perspective. + +Since our template leverages 2 params, we’ll provide default values for them: + + +```yaml + params: + - name: image-pull-sa-name + default: expected-service-account + - name: image_prefix + default: 0.0.0.0:5000/example-basic-sc- +``` + + +_For those using using dockerhub, gcr or other registry, substitute the appropriate default value for image_prefix. For +those using the hack script to create a local registry for this tutorial, run the following command to get the ip +address to use in the place of 0.0.0.0:_ + +```shell +$ ./hack/ip.py +``` + +So far, creating our ClusterImageTemplate has been similar to what we’ve done in previous tutorials using the +ClusterTemplate. There’s only one novel step we must do. We must specify what value will be exposed from this template. +Again, we’re using a ClusterImageTemplate and this custom resource requires the specification of an imagePath. This is +the path on the templated object where we will find our desired value. +[The documentation for the kpack Image resource](https://github.com/pivotal/kpack/blob/main/docs/image.md) lets us know +“When an image resource has successfully built with its current configuration, its status will report the up to date +fully qualified built OCI image reference.” We can see this value is in the `.status.latestImage` field. So that is the +path that we put in the ClusterImageTemplate’s `.spec.imagePath`: + + +```yaml + imagePath: .status.latestImage +``` + + +We now have our full object: + +{{< tutorial image-builder-template.yaml >}} + +#### Service Account + +We now have our two templates. It's time to create the service account that will give Cartographer permission to create +the objects in the templates (a deployment and a kpack image). In the default namespace (because that's where we +declared it would be in the supply chain) we create: + +{{< tutorial cartographer-service-account.yaml >}} + +#### kpack Dependencies + +From Cartographer's perspective, we've completed our work as app operators. We've created our templates, our supply +chain and our service account. When app devs create workloads, Cartographer will happily begin creating objects. But +before we switch over to the app dev role, we have to consider the objects that we're creating and whether they have any +dependencies. We've already seen how the deployment will rely on a service account existing with imagePullCredentials. +Similarly, our kpack image relies on both a service account and on other kpack resources having been installed in the +cluster. + +This isn’t a tutorial on kpack, so we’ll quickly specify objects below. Learners interested in exploring kpack should +[read more here](https://github.com/pivotal/kpack/blob/main/docs/tutorial.md). + +{{< tutorial kpack-boilerplate.yaml >}} + +{{< tutorial registry-service-account.yaml >}} + +- Note that the ClusterBuilder object has a placeholder value in the `.spec.tag` field. Either a local registry ip + address or another image registry specification should be put here. + +- Note that the registry-credentials secret has placeholder values for the `.stringData` field. If learners are using + the local registry from the hack script, use the ./hack/ip.py ip address in place of the 0.0.0.0 address (the username + and password will then be correct). Otherwise learners should put the appropriate credentials for their image + registry. + +Now we've completed our work as app operators. Let’s step into our role as app devs. + +### App Dev Steps + +As is appropriate for an app platform, the complication undertaken by the app operators above is hidden from the app +devs. As devs, all we need to know is that our request has been answered: we can now submit a workload that specifies +the location of our source code and it will be built and deployed. Let’s do that! + +For our app we’ve used a copy of one of the many paketo buildpack sample apps. +[That copy resides here](https://github.com/waciumawanjohi/demo-hello-world) + +_Note: Copying a paketo sample app ensures that our app will be built. Troubleshooting kpack builds of an arbitrary +application is far outside the scope of this tutorial. The sample apps +[can be found here](https://github.com/paketo-buildpacks/samples). Note that for expediency we only installed kpack with +the ability to build golang and java applications. Users should feel free to install additional paketo buildpacks if +desired._ + +Let’s specify the location of our source code in the workload's spec: + +```yaml +spec: + source: + git: + ref: + branch: main + url: https://github.com/waciumawanjohi/demo-hello-world +``` + +We have a new type of app now; it is no longer pre-built. Let’s change our workload type label and match it to the new +supply chain's selector. + + +```yaml + workload-type: source-code +``` + + +And as app devs, we're done! Let’s look at the complete workload: + +{{< tutorial workload.yaml >}} + +## Observe + +### Workload + +Looking at the workload, we can see that it resolves to a healthy state: + +```shell +$ kubectl get -o yaml workload hello-again +``` + +```yaml +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + ... + name: hello-again +status: + conditions: + ... + - lastTransitionTime: ... + message: "" + reason: Ready + status: "True" + type: Ready +``` + +### Stamped Objects + +All the objects that we templated out exist. + +```shell +$ kubectl get -o yaml image.kpack.io hello-again +``` + +```yaml +apiVersion: kpack.io/v1alpha2 +kind: Image +metadata: + ... + name: hello-again +spec: ... +status: + buildCacheName: hello-again-cache + buildCounter: 1 + conditions: + - lastTransitionTime: ... + status: "True" + type: Ready + - lastTransitionTime: ... + status: "True" + type: BuilderReady + latestBuildImageGeneration: 1 + latestBuildReason: CONFIG + latestBuildRef: hello-again-build-1 + latestImage: 0.0.0.0:5000/example-basic-sc-hello-again@sha256:abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12 + latestStack: ... + observedGeneration: 1 +``` + +```shell +$ kubectl get -o yaml deployment hello-again-deployment +``` + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + ... + name: hello-again-deployment +spec: + ... + template: + ... + spec: + containers: + - image: 0.0.0.0:5000/example-basic-sc-hello-again@sha256:abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12 + ... + ... +status: + availableReplicas: 3 + conditions: + - status: "True" + type: Available + ... + - status: "True" + type: Progressing + ... + ... +``` + +The conditions of all of these objects are healthy. In addition, we can see where the kpack image's +`.status.latestImage` field has been used by the deployment's `spec.template.spec.containers[0].image` field. + +## Wrap Up + +Another tutorial under your belt; you’re well on your way to building a robust app platform for your organization! In +this tutorial you learned: + +- How to expose a value from a template to other steps in a supply chain +- How to consume an earlier exposed value in a template +- How to add to a supply chain +- How to create supply chains with different behavior/templates diff --git a/site/content/docs/development/tutorials/files/extending-a-supply-chain/app-deploy-template.yaml b/site/content/docs/development/tutorials/files/extending-a-supply-chain/app-deploy-template.yaml new file mode 100644 index 000000000..149d62b6e --- /dev/null +++ b/site/content/docs/development/tutorials/files/extending-a-supply-chain/app-deploy-template.yaml @@ -0,0 +1,30 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + name: app-deploy-from-sc-image +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: $(workload.metadata.name)$-deployment + labels: + app: $(workload.metadata.name)$ + spec: + replicas: 3 + selector: + matchLabels: + app: $(workload.metadata.name)$ + template: + metadata: + labels: + app: $(workload.metadata.name)$ + spec: + serviceAccountName: $(params.image-pull-sa-name)$ + containers: + - name: $(workload.metadata.name)$ + image: $(images.built-image.image)$ + params: + - name: image-pull-sa-name + default: expected-service-account diff --git a/site/content/docs/development/tutorials/files/extending-a-supply-chain/cartographer-service-account.yaml b/site/content/docs/development/tutorials/files/extending-a-supply-chain/cartographer-service-account.yaml new file mode 100644 index 000000000..0a93f6e17 --- /dev/null +++ b/site/content/docs/development/tutorials/files/extending-a-supply-chain/cartographer-service-account.yaml @@ -0,0 +1,69 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cartographer-from-source-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: deploy-image-role +rules: + - apiGroups: + - apps + resources: + - deployments + verbs: + - list + - create + - update + - delete + - patch + - watch + - get + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: cartographer-deploy-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: deploy-image-role +subjects: + - kind: ServiceAccount + name: cartographer-from-source-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: build-image-role +rules: + - apiGroups: + - kpack.io + resources: + - images + verbs: + - list + - create + - update + - delete + - patch + - watch + - get + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: cartographer-build-image-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: build-image-role +subjects: + - kind: ServiceAccount + name: cartographer-from-source-sa diff --git a/site/content/docs/development/tutorials/files/extending-a-supply-chain/image-builder-template.yaml b/site/content/docs/development/tutorials/files/extending-a-supply-chain/image-builder-template.yaml new file mode 100644 index 000000000..ca0ffa77d --- /dev/null +++ b/site/content/docs/development/tutorials/files/extending-a-supply-chain/image-builder-template.yaml @@ -0,0 +1,27 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterImageTemplate +metadata: + name: image-builder +spec: + template: + apiVersion: kpack.io/v1alpha2 + kind: Image + metadata: + name: $(workload.metadata.name)$ + spec: + tag: $(params.image_prefix)$$(workload.metadata.name)$ + serviceAccountName: $(params.image-pull-sa-name)$ + builder: + kind: ClusterBuilder + name: my-builder + source: + git: + url: $(workload.spec.source.git.url)$ + revision: $(workload.spec.source.git.ref.branch)$ + params: + - name: image-pull-sa-name + default: expected-service-account + - name: image_prefix + default: 0.0.0.0:5000/example-basic-sc- # <=== Change to proper image registry + imagePath: .status.latestImage diff --git a/site/content/docs/development/tutorials/files/extending-a-supply-chain/kpack-boilerplate.yaml b/site/content/docs/development/tutorials/files/extending-a-supply-chain/kpack-boilerplate.yaml new file mode 100644 index 000000000..09dbf91eb --- /dev/null +++ b/site/content/docs/development/tutorials/files/extending-a-supply-chain/kpack-boilerplate.yaml @@ -0,0 +1,43 @@ +--- +apiVersion: kpack.io/v1alpha1 +kind: ClusterStore +metadata: + name: default +spec: + sources: + - image: gcr.io/paketo-buildpacks/java + - image: gcr.io/paketo-buildpacks/go + +--- +apiVersion: kpack.io/v1alpha1 +kind: ClusterStack +metadata: + name: base +spec: + id: "io.buildpacks.stacks.bionic" + buildImage: + image: "paketobuildpacks/build:base-cnb" + runImage: + image: "paketobuildpacks/run:base-cnb" + +--- +apiVersion: kpack.io/v1alpha2 +kind: ClusterBuilder +metadata: + name: my-builder +spec: + serviceAccountRef: + name: expected-service-account + namespace: default + tag: "0.0.0.0:5000/go-java-builder" # <=== Change to proper image registry + stack: + name: base + kind: ClusterStack + store: + name: default + kind: ClusterStore + order: + - group: + - id: paketo-buildpacks/java + - group: + - id: paketo-buildpacks/go diff --git a/site/content/docs/development/tutorials/files/extending-a-supply-chain/registry-service-account.yaml b/site/content/docs/development/tutorials/files/extending-a-supply-chain/registry-service-account.yaml new file mode 100644 index 000000000..657747918 --- /dev/null +++ b/site/content/docs/development/tutorials/files/extending-a-supply-chain/registry-service-account.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: v1 +kind: Secret +metadata: + name: registry-credentials +type: kubernetes.io/dockerconfigjson +stringData: + .dockerconfigjson: '{"auths": {"0.0.0.0:5000": {"username": "admin", "password": "admin"}}}' # <=== Change to proper image registry + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: expected-service-account +secrets: + - name: registry-credentials +imagePullSecrets: + - name: registry-credentials diff --git a/site/content/docs/development/tutorials/files/extending-a-supply-chain/supply-chain.yaml b/site/content/docs/development/tutorials/files/extending-a-supply-chain/supply-chain.yaml new file mode 100644 index 000000000..352c3b886 --- /dev/null +++ b/site/content/docs/development/tutorials/files/extending-a-supply-chain/supply-chain.yaml @@ -0,0 +1,25 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterSupplyChain +metadata: + name: source-code-supply-chain +spec: + selector: + workload-type: source-code + + resources: + - name: build-image + templateRef: + kind: ClusterImageTemplate + name: image-builder + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy-from-sc-image + images: + - resource: build-image + name: built-image + + serviceAccountRef: + name: cartographer-from-source-sa + namespace: default diff --git a/site/content/docs/development/tutorials/files/extending-a-supply-chain/workload.yaml b/site/content/docs/development/tutorials/files/extending-a-supply-chain/workload.yaml new file mode 100644 index 000000000..f1e09410b --- /dev/null +++ b/site/content/docs/development/tutorials/files/extending-a-supply-chain/workload.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + name: hello-again + labels: + workload-type: source-code +spec: + source: + git: + ref: + branch: main + url: https://github.com/waciumawanjohi/demo-hello-world \ No newline at end of file diff --git a/site/content/docs/development/tutorials/files/first-supply-chain/app-deploy-template.yaml b/site/content/docs/development/tutorials/files/first-supply-chain/app-deploy-template.yaml new file mode 100644 index 000000000..fe93739fd --- /dev/null +++ b/site/content/docs/development/tutorials/files/first-supply-chain/app-deploy-template.yaml @@ -0,0 +1,26 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + name: app-deploy +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: $(workload.metadata.name)$-deployment + labels: + app: $(workload.metadata.name)$ + spec: + replicas: 3 + selector: + matchLabels: + app: $(workload.metadata.name)$ + template: + metadata: + labels: + app: $(workload.metadata.name)$ + spec: + containers: + - name: $(workload.metadata.name)$ + image: $(workload.spec.image)$ diff --git a/site/content/docs/development/tutorials/files/first-supply-chain/cartographer-service-account.yaml b/site/content/docs/development/tutorials/files/first-supply-chain/cartographer-service-account.yaml new file mode 100644 index 000000000..4d8de36c4 --- /dev/null +++ b/site/content/docs/development/tutorials/files/first-supply-chain/cartographer-service-account.yaml @@ -0,0 +1,6 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cartographer-pre-built-sa + namespace: default diff --git a/site/content/docs/development/tutorials/files/first-supply-chain/deployment-role.yaml b/site/content/docs/development/tutorials/files/first-supply-chain/deployment-role.yaml new file mode 100644 index 000000000..13793234f --- /dev/null +++ b/site/content/docs/development/tutorials/files/first-supply-chain/deployment-role.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: deploy-image-role +rules: + - apiGroups: + - apps + resources: + - deployments + verbs: + - list + - create + - update + - delete + - patch + - watch + - get diff --git a/site/content/docs/development/tutorials/files/first-supply-chain/role-binding-dep-carto.yaml b/site/content/docs/development/tutorials/files/first-supply-chain/role-binding-dep-carto.yaml new file mode 100644 index 000000000..7f5f10c7f --- /dev/null +++ b/site/content/docs/development/tutorials/files/first-supply-chain/role-binding-dep-carto.yaml @@ -0,0 +1,12 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: cartographer-prebuilt-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: deploy-image-role +subjects: + - kind: ServiceAccount + name: cartographer-pre-built-sa \ No newline at end of file diff --git a/site/content/docs/development/tutorials/files/first-supply-chain/supply-chain.yaml b/site/content/docs/development/tutorials/files/first-supply-chain/supply-chain.yaml new file mode 100644 index 000000000..a42e5f11b --- /dev/null +++ b/site/content/docs/development/tutorials/files/first-supply-chain/supply-chain.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterSupplyChain +metadata: + name: supply-chain +spec: + resources: + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy + + serviceAccountRef: + name: cartographer-pre-built-sa + namespace: default + + selector: + workload-type: pre-built diff --git a/site/content/docs/development/tutorials/files/first-supply-chain/workload-2.yaml b/site/content/docs/development/tutorials/files/first-supply-chain/workload-2.yaml new file mode 100644 index 000000000..a6e9cf006 --- /dev/null +++ b/site/content/docs/development/tutorials/files/first-supply-chain/workload-2.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + name: whale-hello-there + labels: + workload-type: pre-built +spec: + image: docker.io/crccheck/hello-world:latest diff --git a/site/content/docs/development/tutorials/files/first-supply-chain/workload.yaml b/site/content/docs/development/tutorials/files/first-supply-chain/workload.yaml new file mode 100644 index 000000000..13b8b0582 --- /dev/null +++ b/site/content/docs/development/tutorials/files/first-supply-chain/workload.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + name: hello + labels: + workload-type: pre-built +spec: + image: docker.io/nginxdemos/hello:latest diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/cartographer-service-account.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/cartographer-service-account.yaml new file mode 100644 index 000000000..f71951d8c --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/cartographer-service-account.yaml @@ -0,0 +1,101 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cartographer-from-source-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: deploy-image-role +rules: + - apiGroups: + - apps + resources: + - deployments + verbs: + - list + - create + - update + - delete + - patch + - watch + - get + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: cartographer-prebuilt-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: deploy-image-role +subjects: + - kind: ServiceAccount + name: cartographer-from-source-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: build-image-role +rules: + - apiGroups: + - kpack.io + resources: + - images + verbs: + - list + - create + - update + - delete + - patch + - watch + - get + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: cartographer-build-image-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: build-image-role +subjects: + - kind: ServiceAccount + name: cartographer-from-source-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: runnable-role +rules: + - apiGroups: + - carto.run + resources: + - runnables + verbs: + - list + - create + - update + - delete + - patch + - watch + - get + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: cartographer-runnable-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: runnable-role +subjects: + - kind: ServiceAccount + name: cartographer-from-source-sa diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/cluster-run-template.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/cluster-run-template.yaml new file mode 100644 index 000000000..e45a407d4 --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/cluster-run-template.yaml @@ -0,0 +1,31 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterRunTemplate +metadata: + name: md-linting-pipelinerun +spec: + template: + apiVersion: tekton.dev/v1beta1 + kind: PipelineRun + metadata: + generateName: $(runnable.metadata.name)$-pipeline-run- + spec: + pipelineRef: + name: linter-pipeline + params: + - name: repository + value: $(runnable.spec.inputs.repository)$ + - name: revision + value: $(runnable.spec.inputs.revision)$ + workspaces: + - name: shared-workspace + volumeClaimTemplate: + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 256Mi + outputs: + url: spec.params[?(@.name=="repository")].value + revision: spec.params[?(@.name=="revision")].value diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/kpack-boilerplate.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/kpack-boilerplate.yaml new file mode 100644 index 000000000..09dbf91eb --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/kpack-boilerplate.yaml @@ -0,0 +1,43 @@ +--- +apiVersion: kpack.io/v1alpha1 +kind: ClusterStore +metadata: + name: default +spec: + sources: + - image: gcr.io/paketo-buildpacks/java + - image: gcr.io/paketo-buildpacks/go + +--- +apiVersion: kpack.io/v1alpha1 +kind: ClusterStack +metadata: + name: base +spec: + id: "io.buildpacks.stacks.bionic" + buildImage: + image: "paketobuildpacks/build:base-cnb" + runImage: + image: "paketobuildpacks/run:base-cnb" + +--- +apiVersion: kpack.io/v1alpha2 +kind: ClusterBuilder +metadata: + name: my-builder +spec: + serviceAccountRef: + name: expected-service-account + namespace: default + tag: "0.0.0.0:5000/go-java-builder" # <=== Change to proper image registry + stack: + name: base + kind: ClusterStack + store: + name: default + kind: ClusterStore + order: + - group: + - id: paketo-buildpacks/java + - group: + - id: paketo-buildpacks/go diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/pipeline.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/pipeline.yaml new file mode 100644 index 000000000..f921a7e6f --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/pipeline.yaml @@ -0,0 +1,40 @@ +--- +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: linter-pipeline +spec: + params: + - name: repository + type: string + - name: revision + type: string + workspaces: + - name: shared-workspace + tasks: + - name: fetch-repository + taskRef: + name: git-clone + workspaces: + - name: output + workspace: shared-workspace + params: + - name: url + value: $(params.repository) + - name: revision + value: $(params.revision) + - name: subdirectory + value: "" + - name: deleteExisting + value: "true" + - name: md-lint-run #lint mardown + taskRef: + name: markdown-lint + runAfter: + - fetch-repository + workspaces: + - name: shared-workspace + workspace: shared-workspace + params: + - name: args + value: ["."] diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/registry-service-account.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/registry-service-account.yaml new file mode 100644 index 000000000..9680988e4 --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/registry-service-account.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: v1 +kind: Secret +metadata: + name: registry-credentials +type: kubernetes.io/dockerconfigjson +stringData: + .dockerconfigjson: '{"auths": {"0.0.0.0:5000": {"username": "admin", "password": "admin"}}}' # <=== Change to proper image registry + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: expected-service-account +secrets: + - name: registry-credentials +imagePullSecrets: + - name: registry-credentials diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/runnable-service-account.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/runnable-service-account.yaml new file mode 100644 index 000000000..3431f8224 --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/runnable-service-account.yaml @@ -0,0 +1,37 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: pipeline-run-management-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: testing-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: pipeline-run-management-role +subjects: + - kind: ServiceAccount + name: pipeline-run-management-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: pipeline-run-management-role +rules: + - apiGroups: + - tekton.dev + resources: + - pipelineruns + verbs: + - list + - create + - update + - delete + - patch + - watch + - get diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/source-linter-template.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/source-linter-template.yaml new file mode 100644 index 000000000..ab67b91bb --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/source-linter-template.yaml @@ -0,0 +1,20 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterSourceTemplate +metadata: + name: source-linter +spec: + template: + apiVersion: carto.run/v1alpha1 + kind: Runnable + metadata: + name: $(workload.metadata.name)$-linter + spec: + runTemplateRef: + name: md-linting-pipelinerun + inputs: + repository: $(workload.spec.source.git.url)$ + revision: $(workload.spec.source.git.ref.branch)$ + serviceAccountName: pipeline-run-management-sa + urlPath: .status.outputs.url + revisionPath: .status.outputs.revision diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/supply-chain.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/supply-chain.yaml new file mode 100644 index 000000000..a4d533396 --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/supply-chain.yaml @@ -0,0 +1,32 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterSupplyChain +metadata: + name: source-code-supply-chain +spec: + selector: + workload-type: source-code + + resources: + - name: lint-source + templateRef: + kind: ClusterSourceTemplate + name: source-linter + - name: build-image + templateRef: + kind: ClusterImageTemplate + name: image-builder-from-previous-step + sources: + - resource: lint-source + name: source + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy-from-sc-image + images: + - resource: build-image + name: built-image + + serviceAccountRef: + name: cartographer-from-source-sa + namespace: default diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/tasks.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/tasks.yaml new file mode 100644 index 000000000..0d40ef7aa --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/tasks.yaml @@ -0,0 +1,204 @@ +--- +apiVersion: tekton.dev/v1beta1 +kind: Task +metadata: + name: git-clone + labels: + app.kubernetes.io/version: "0.3" + annotations: + tekton.dev/pipelines.minVersion: "0.21.0" + tekton.dev/categories: Git + tekton.dev/tags: git + tekton.dev/displayName: "git clone" + tekton.dev/platforms: "linux/amd64" +spec: + description: >- + These Tasks are Git tasks to work with repositories used by other tasks + in your Pipeline. + + The git-clone Task will clone a repo from the provided url into the + output Workspace. By default the repo will be cloned into the root of + your Workspace. You can clone into a subdirectory by setting this Task's + subdirectory param. This Task also supports sparse checkouts. To perform + a sparse checkout, pass a list of comma separated directory patterns to + this Task's sparseCheckoutDirectories param. + + workspaces: + - name: output + description: The git repo will be cloned onto the volume backing this workspace + params: + - name: url + description: git url to clone + type: string + - name: revision + description: git revision to checkout (branch, tag, sha, ref…) + type: string + default: "" + - name: refspec + description: (optional) git refspec to fetch before checking out revision + default: "" + - name: submodules + description: defines if the resource should initialize and fetch the submodules + type: string + default: "true" + - name: depth + description: performs a shallow clone where only the most recent commit(s) will be fetched + type: string + default: "1" + - name: sslVerify + description: defines if http.sslVerify should be set to true or false in the global git config + type: string + default: "true" + - name: subdirectory + description: subdirectory inside the "output" workspace to clone the git repo into + type: string + default: "" + - name: sparseCheckoutDirectories + description: defines which directories patterns to match or exclude when performing a sparse checkout + type: string + default: "" + - name: deleteExisting + description: clean out the contents of the repo's destination directory (if it already exists) before trying to clone the repo there + type: string + default: "true" + - name: httpProxy + description: git HTTP proxy server for non-SSL requests + type: string + default: "" + - name: httpsProxy + description: git HTTPS proxy server for SSL requests + type: string + default: "" + - name: noProxy + description: git no proxy - opt out of proxying HTTP/HTTPS requests + type: string + default: "" + - name: verbose + description: log the commands used during execution + type: string + default: "true" + - name: gitInitImage + description: the image used where the git-init binary is + type: string + default: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.21.0" + results: + - name: commit + description: The precise commit SHA that was fetched by this Task + - name: url + description: The precise URL that was fetched by this Task + steps: + - name: clone + image: $(params.gitInitImage) + env: + - name: PARAM_URL + value: $(params.url) + - name: PARAM_REVISION + value: $(params.revision) + - name: PARAM_REFSPEC + value: $(params.refspec) + - name: PARAM_SUBMODULES + value: $(params.submodules) + - name: PARAM_DEPTH + value: $(params.depth) + - name: PARAM_SSL_VERIFY + value: $(params.sslVerify) + - name: PARAM_SUBDIRECTORY + value: $(params.subdirectory) + - name: PARAM_DELETE_EXISTING + value: $(params.deleteExisting) + - name: PARAM_HTTP_PROXY + value: $(params.httpProxy) + - name: PARAM_HTTPS_PROXY + value: $(params.httpsProxy) + - name: PARAM_NO_PROXY + value: $(params.noProxy) + - name: PARAM_VERBOSE + value: $(params.verbose) + - name: PARAM_SPARSE_CHECKOUT_DIRECTORIES + value: $(params.sparseCheckoutDirectories) + - name: WORKSPACE_OUTPUT_PATH + value: $(workspaces.output.path) + script: | + #!/bin/sh + set -eu -o pipefail + + if [[ "${PARAM_VERBOSE}" == "true" ]] ; then + set -x + fi + + CHECKOUT_DIR="${WORKSPACE_OUTPUT_PATH}/${PARAM_SUBDIRECTORY}" + + cleandir() { + # Delete any existing contents of the repo directory if it exists. + # + # We don't just "rm -rf $CHECKOUT_DIR" because $CHECKOUT_DIR might be "/" + # or the root of a mounted volume. + if [[ -d "$CHECKOUT_DIR" ]] ; then + # Delete non-hidden files and directories + rm -rf "$CHECKOUT_DIR"/* + # Delete files and directories starting with . but excluding .. + rm -rf "$CHECKOUT_DIR"/.[!.]* + # Delete files and directories starting with .. plus any other character + rm -rf "$CHECKOUT_DIR"/..?* + fi + } + + if [[ "${PARAM_DELETE_EXISTING}" == "true" ]] ; then + cleandir + fi + + test -z "${PARAM_HTTP_PROXY}" || export HTTP_PROXY="${PARAM_HTTP_PROXY}" + test -z "${PARAM_HTTPS_PROXY}" || export HTTPS_PROXY="${PARAM_HTTPS_PROXY}" + test -z "${PARAM_NO_PROXY}" || export NO_PROXY="${PARAM_NO_PROXY}" + + /ko-app/git-init \ + -url "${PARAM_URL}" \ + -revision "${PARAM_REVISION}" \ + -refspec "${PARAM_REFSPEC}" \ + -path "$CHECKOUT_DIR" \ + -sslVerify="${PARAM_SSL_VERIFY}" \ + -submodules="${PARAM_SUBMODULES}" \ + -depth "${PARAM_DEPTH}" \ + -sparseCheckoutDirectories "${PARAM_SPARSE_CHECKOUT_DIRECTORIES}" + cd "$CHECKOUT_DIR" + RESULT_SHA="$(git rev-parse HEAD)" + EXIT_CODE="$?" + if [ "$EXIT_CODE" != 0 ] ; then + exit $EXIT_CODE + fi + # ensure we don't add a trailing newline to the result + echo -n "$RESULT_SHA" > $(results.commit.path) + echo -n "${PARAM_URL}" > $(results.url.path) + +--- +apiVersion: tekton.dev/v1beta1 +kind: Task +metadata: + name: markdown-lint + labels: + app.kubernetes.io/version: "0.1" + annotations: + tekton.dev/pipelines.minVersion: "0.12.1" + tekton.dev/categories: Code Quality + tekton.dev/tags: linter + tekton.dev/displayName: "Markdown linter" + tekton.dev/platforms: "linux/amd64" +spec: + description: >- + This task can be used to perform lint check on Markdown files + workspaces: + - name: shared-workspace + description: A workspace that contains the fetched git repository. + params: + - name: args + type: array + description: extra args needs to append + default: ["--help"] + steps: + - name: lint-markdown-files + image: docker.io/markdownlint/markdownlint:0.11.0@sha256:399a199c92f89f42cf3a0a1159bd86ca5cdc293fcfd39f87c0669ddee9767724 #tag: 0.11.0 + workingDir: $(workspaces.shared-workspace.path) + command: + - mdl + args: + - $(params.args) \ No newline at end of file diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/templates.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/templates.yaml new file mode 100644 index 000000000..fb981b70a --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/templates.yaml @@ -0,0 +1,58 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterImageTemplate +metadata: + name: image-builder-from-previous-step +spec: + template: + apiVersion: kpack.io/v1alpha2 + kind: Image + metadata: + name: $(workload.metadata.name)$ + spec: + tag: $(params.image_prefix)$$(workload.metadata.name)$ + serviceAccountName: $(params.image-pull-sa-name)$ + builder: + kind: ClusterBuilder + name: my-builder + source: + git: + url: $(sources.source.url)$ + revision: $(sources.source.revision)$ + params: + - name: image-pull-sa-name + default: expected-service-account + - name: image_prefix + default: 0.0.0.0:5000/example-basic-sc- # <=== Change to proper image registry + imagePath: .status.latestImage + +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + name: app-deploy-from-sc-image +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: $(workload.metadata.name)$-deployment + labels: + app: $(workload.metadata.name)$ + spec: + replicas: 3 + selector: + matchLabels: + app: $(workload.metadata.name)$ + template: + metadata: + labels: + app: $(workload.metadata.name)$ + spec: + serviceAccountName: $(params.image-pull-sa-name)$ + containers: + - name: $(workload.metadata.name)$ + image: $(images.built-image.image)$ + params: + - name: image-pull-sa-name + default: expected-service-account diff --git a/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/workload.yaml b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/workload.yaml new file mode 100644 index 000000000..f1e09410b --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable-in-a-supply-chain/workload.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + name: hello-again + labels: + workload-type: source-code +spec: + source: + git: + ref: + branch: main + url: https://github.com/waciumawanjohi/demo-hello-world \ No newline at end of file diff --git a/site/content/docs/development/tutorials/files/runnable/cluster-run-template.yaml b/site/content/docs/development/tutorials/files/runnable/cluster-run-template.yaml new file mode 100644 index 000000000..6a12ef7ff --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable/cluster-run-template.yaml @@ -0,0 +1,30 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterRunTemplate +metadata: + name: md-linting-pipelinerun +spec: + template: + apiVersion: tekton.dev/v1beta1 + kind: PipelineRun + metadata: + generateName: linter-pipeline-run- + spec: + pipelineRef: + name: linter-pipeline + params: + - name: repository + value: $(runnable.spec.inputs.repository)$ + - name: revision + value: $(runnable.spec.inputs.revision)$ + workspaces: + - name: shared-workspace + volumeClaimTemplate: + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 256Mi + outputs: + lastTransitionTime: .status.conditions[0].lastTransitionTime diff --git a/site/content/docs/development/tutorials/files/runnable/pipeline.yaml b/site/content/docs/development/tutorials/files/runnable/pipeline.yaml new file mode 100644 index 000000000..e8ed9afe4 --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable/pipeline.yaml @@ -0,0 +1,40 @@ +--- +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: linter-pipeline +spec: + params: + - name: repository + type: string + - name: revision + type: string + workspaces: + - name: shared-workspace + tasks: + - name: fetch-repository + taskRef: + name: git-clone + workspaces: + - name: output + workspace: shared-workspace + params: + - name: url + value: $(params.repository) + - name: revision + value: $(params.revision) + - name: subdirectory + value: "" + - name: deleteExisting + value: "true" + - name: md-lint-run #lint markdown + taskRef: + name: markdown-lint + runAfter: + - fetch-repository + workspaces: + - name: shared-workspace + workspace: shared-workspace + params: + - name: args + value: ["."] diff --git a/site/content/docs/development/tutorials/files/runnable/runnable.yaml b/site/content/docs/development/tutorials/files/runnable/runnable.yaml new file mode 100644 index 000000000..9cdaae94e --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable/runnable.yaml @@ -0,0 +1,12 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: Runnable +metadata: + name: linter +spec: + runTemplateRef: + name: md-linting-pipelinerun + inputs: + repository: https://github.com/waciumawanjohi/demo-hello-world + revision: main + serviceAccountName: pipeline-run-management-sa diff --git a/site/content/docs/development/tutorials/files/runnable/service-account.yaml b/site/content/docs/development/tutorials/files/runnable/service-account.yaml new file mode 100644 index 000000000..3431f8224 --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable/service-account.yaml @@ -0,0 +1,37 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: pipeline-run-management-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: testing-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: pipeline-run-management-role +subjects: + - kind: ServiceAccount + name: pipeline-run-management-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: pipeline-run-management-role +rules: + - apiGroups: + - tekton.dev + resources: + - pipelineruns + verbs: + - list + - create + - update + - delete + - patch + - watch + - get diff --git a/site/content/docs/development/tutorials/files/runnable/tasks.yaml b/site/content/docs/development/tutorials/files/runnable/tasks.yaml new file mode 100644 index 000000000..0d40ef7aa --- /dev/null +++ b/site/content/docs/development/tutorials/files/runnable/tasks.yaml @@ -0,0 +1,204 @@ +--- +apiVersion: tekton.dev/v1beta1 +kind: Task +metadata: + name: git-clone + labels: + app.kubernetes.io/version: "0.3" + annotations: + tekton.dev/pipelines.minVersion: "0.21.0" + tekton.dev/categories: Git + tekton.dev/tags: git + tekton.dev/displayName: "git clone" + tekton.dev/platforms: "linux/amd64" +spec: + description: >- + These Tasks are Git tasks to work with repositories used by other tasks + in your Pipeline. + + The git-clone Task will clone a repo from the provided url into the + output Workspace. By default the repo will be cloned into the root of + your Workspace. You can clone into a subdirectory by setting this Task's + subdirectory param. This Task also supports sparse checkouts. To perform + a sparse checkout, pass a list of comma separated directory patterns to + this Task's sparseCheckoutDirectories param. + + workspaces: + - name: output + description: The git repo will be cloned onto the volume backing this workspace + params: + - name: url + description: git url to clone + type: string + - name: revision + description: git revision to checkout (branch, tag, sha, ref…) + type: string + default: "" + - name: refspec + description: (optional) git refspec to fetch before checking out revision + default: "" + - name: submodules + description: defines if the resource should initialize and fetch the submodules + type: string + default: "true" + - name: depth + description: performs a shallow clone where only the most recent commit(s) will be fetched + type: string + default: "1" + - name: sslVerify + description: defines if http.sslVerify should be set to true or false in the global git config + type: string + default: "true" + - name: subdirectory + description: subdirectory inside the "output" workspace to clone the git repo into + type: string + default: "" + - name: sparseCheckoutDirectories + description: defines which directories patterns to match or exclude when performing a sparse checkout + type: string + default: "" + - name: deleteExisting + description: clean out the contents of the repo's destination directory (if it already exists) before trying to clone the repo there + type: string + default: "true" + - name: httpProxy + description: git HTTP proxy server for non-SSL requests + type: string + default: "" + - name: httpsProxy + description: git HTTPS proxy server for SSL requests + type: string + default: "" + - name: noProxy + description: git no proxy - opt out of proxying HTTP/HTTPS requests + type: string + default: "" + - name: verbose + description: log the commands used during execution + type: string + default: "true" + - name: gitInitImage + description: the image used where the git-init binary is + type: string + default: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.21.0" + results: + - name: commit + description: The precise commit SHA that was fetched by this Task + - name: url + description: The precise URL that was fetched by this Task + steps: + - name: clone + image: $(params.gitInitImage) + env: + - name: PARAM_URL + value: $(params.url) + - name: PARAM_REVISION + value: $(params.revision) + - name: PARAM_REFSPEC + value: $(params.refspec) + - name: PARAM_SUBMODULES + value: $(params.submodules) + - name: PARAM_DEPTH + value: $(params.depth) + - name: PARAM_SSL_VERIFY + value: $(params.sslVerify) + - name: PARAM_SUBDIRECTORY + value: $(params.subdirectory) + - name: PARAM_DELETE_EXISTING + value: $(params.deleteExisting) + - name: PARAM_HTTP_PROXY + value: $(params.httpProxy) + - name: PARAM_HTTPS_PROXY + value: $(params.httpsProxy) + - name: PARAM_NO_PROXY + value: $(params.noProxy) + - name: PARAM_VERBOSE + value: $(params.verbose) + - name: PARAM_SPARSE_CHECKOUT_DIRECTORIES + value: $(params.sparseCheckoutDirectories) + - name: WORKSPACE_OUTPUT_PATH + value: $(workspaces.output.path) + script: | + #!/bin/sh + set -eu -o pipefail + + if [[ "${PARAM_VERBOSE}" == "true" ]] ; then + set -x + fi + + CHECKOUT_DIR="${WORKSPACE_OUTPUT_PATH}/${PARAM_SUBDIRECTORY}" + + cleandir() { + # Delete any existing contents of the repo directory if it exists. + # + # We don't just "rm -rf $CHECKOUT_DIR" because $CHECKOUT_DIR might be "/" + # or the root of a mounted volume. + if [[ -d "$CHECKOUT_DIR" ]] ; then + # Delete non-hidden files and directories + rm -rf "$CHECKOUT_DIR"/* + # Delete files and directories starting with . but excluding .. + rm -rf "$CHECKOUT_DIR"/.[!.]* + # Delete files and directories starting with .. plus any other character + rm -rf "$CHECKOUT_DIR"/..?* + fi + } + + if [[ "${PARAM_DELETE_EXISTING}" == "true" ]] ; then + cleandir + fi + + test -z "${PARAM_HTTP_PROXY}" || export HTTP_PROXY="${PARAM_HTTP_PROXY}" + test -z "${PARAM_HTTPS_PROXY}" || export HTTPS_PROXY="${PARAM_HTTPS_PROXY}" + test -z "${PARAM_NO_PROXY}" || export NO_PROXY="${PARAM_NO_PROXY}" + + /ko-app/git-init \ + -url "${PARAM_URL}" \ + -revision "${PARAM_REVISION}" \ + -refspec "${PARAM_REFSPEC}" \ + -path "$CHECKOUT_DIR" \ + -sslVerify="${PARAM_SSL_VERIFY}" \ + -submodules="${PARAM_SUBMODULES}" \ + -depth "${PARAM_DEPTH}" \ + -sparseCheckoutDirectories "${PARAM_SPARSE_CHECKOUT_DIRECTORIES}" + cd "$CHECKOUT_DIR" + RESULT_SHA="$(git rev-parse HEAD)" + EXIT_CODE="$?" + if [ "$EXIT_CODE" != 0 ] ; then + exit $EXIT_CODE + fi + # ensure we don't add a trailing newline to the result + echo -n "$RESULT_SHA" > $(results.commit.path) + echo -n "${PARAM_URL}" > $(results.url.path) + +--- +apiVersion: tekton.dev/v1beta1 +kind: Task +metadata: + name: markdown-lint + labels: + app.kubernetes.io/version: "0.1" + annotations: + tekton.dev/pipelines.minVersion: "0.12.1" + tekton.dev/categories: Code Quality + tekton.dev/tags: linter + tekton.dev/displayName: "Markdown linter" + tekton.dev/platforms: "linux/amd64" +spec: + description: >- + This task can be used to perform lint check on Markdown files + workspaces: + - name: shared-workspace + description: A workspace that contains the fetched git repository. + params: + - name: args + type: array + description: extra args needs to append + default: ["--help"] + steps: + - name: lint-markdown-files + image: docker.io/markdownlint/markdownlint:0.11.0@sha256:399a199c92f89f42cf3a0a1159bd86ca5cdc293fcfd39f87c0669ddee9767724 #tag: 0.11.0 + workingDir: $(workspaces.shared-workspace.path) + command: + - mdl + args: + - $(params.args) \ No newline at end of file diff --git a/site/content/docs/development/tutorials/files/using-params/app-deploy-template.yaml b/site/content/docs/development/tutorials/files/using-params/app-deploy-template.yaml new file mode 100644 index 000000000..5d6ef0744 --- /dev/null +++ b/site/content/docs/development/tutorials/files/using-params/app-deploy-template.yaml @@ -0,0 +1,30 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + name: app-deploy +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: $(workload.metadata.name)$-deployment + labels: + app: $(workload.metadata.name)$ + spec: + replicas: 3 + selector: + matchLabels: + app: $(workload.metadata.name)$ + template: + metadata: + labels: + app: $(workload.metadata.name)$ + spec: + serviceAccountName: $(params.image-pull-sa-name)$ + containers: + - name: $(workload.metadata.name)$ + image: $(workload.spec.image)$ + params: + - name: image-pull-sa-name + default: expected-service-account diff --git a/site/content/docs/development/tutorials/files/using-params/cartographer-service-account.yaml b/site/content/docs/development/tutorials/files/using-params/cartographer-service-account.yaml new file mode 100644 index 000000000..8dc26ceee --- /dev/null +++ b/site/content/docs/development/tutorials/files/using-params/cartographer-service-account.yaml @@ -0,0 +1,37 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cartographer-pre-built-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: cartographer-prebuilt-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: deploy-image-role +subjects: + - kind: ServiceAccount + name: cartographer-pre-built-sa + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: deploy-image-role +rules: + - apiGroups: + - apps + resources: + - deployments + verbs: + - list + - create + - update + - delete + - patch + - watch + - get diff --git a/site/content/docs/development/tutorials/files/using-params/registry-service-account.yaml b/site/content/docs/development/tutorials/files/using-params/registry-service-account.yaml new file mode 100644 index 000000000..c14e50908 --- /dev/null +++ b/site/content/docs/development/tutorials/files/using-params/registry-service-account.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: v1 +kind: Secret +metadata: + name: registry-credentials +type: kubernetes.io/dockerconfigjson +stringData: + .dockerconfigjson: '{"auths": {"0.0.0.0:5000": {"username": "admin", "password": "admin"}}}' # <=== Change to proper image registry credentials + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: unconventionally-named-service-account +secrets: + - name: registry-credentials +imagePullSecrets: + - name: registry-credentials diff --git a/site/content/docs/development/tutorials/files/using-params/supply-chain.yaml b/site/content/docs/development/tutorials/files/using-params/supply-chain.yaml new file mode 100644 index 000000000..a42e5f11b --- /dev/null +++ b/site/content/docs/development/tutorials/files/using-params/supply-chain.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterSupplyChain +metadata: + name: supply-chain +spec: + resources: + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy + + serviceAccountRef: + name: cartographer-pre-built-sa + namespace: default + + selector: + workload-type: pre-built diff --git a/site/content/docs/development/tutorials/files/using-params/workload.yaml b/site/content/docs/development/tutorials/files/using-params/workload.yaml new file mode 100644 index 000000000..8bfde1945 --- /dev/null +++ b/site/content/docs/development/tutorials/files/using-params/workload.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + name: hello-again + labels: + workload-type: pre-built +spec: + image: 0.0.0.0:5000/hello-world # <=== Change this to some proper registry and image + serviceAccountName: cartographer-pre-built-sa + params: + - name: image-pull-sa-name + value: unconventionally-named-service-account diff --git a/site/content/docs/development/tutorials/first-supply-chain.md b/site/content/docs/development/tutorials/first-supply-chain.md new file mode 100644 index 000000000..98ee4c08b --- /dev/null +++ b/site/content/docs/development/tutorials/first-supply-chain.md @@ -0,0 +1,500 @@ +# Build Your First Supply Chain + +## Overview + +In this example, we’ll explore the two fundamental resources that an operator deploys, templates and supply-chains, and +how these interact with the resource a dev deploys, the workload. We'll also see how we grant Cartographer RBAC +permission to create our specified objects with a service account. We’ll do this with an incredibly simple supply chain, +one that has a single step: creating a deployment from an image. + +## Environment setup + +For this tutorial you will need a kubernetes cluster with Cartographer installed. You may follow +[the installation instructions here](https://github.com/vmware-tanzu/cartographer#installation). + +Alternatively, you may choose to use the +[./hack/setup.sh](https://github.com/vmware-tanzu/cartographer/blob/main/hack/setup.sh) script to install a kind cluster +with Cartographer. _This script is meant for our end-to-end testing and while we rely on it working in that role, no +user guarantees are made about the script._ + +Command to run from the Cartographer directory: + +```shell +$ ./hack/setup.sh cluster cartographer-latest +``` + +If you later wish to tear down this generated cluster, run + +```shell +$ ./hack/setup.sh teardown +``` + +## Scenario + +We will work as the devs and the app operators in a company creating hello world web applications. As app devs we will +already have created pre-built images of these distros. Our supply chain will create deployments with these pre-built +images on them. We will work first as the app operators to create the appropriate template, supply chain, and service +account. Then we will work as the app devs to create our workload object. + +## Steps + +### App Operator Steps + +#### Templates + +For any template there are fields that the operator can hardcode for all apps and there are other fields that will need +to vary. There are two concerns that can necessitate a field varying: + +- Some fields must be unique across the fleet of apps from all devs. +- Some fields must have specific values specific to the particular application (generally known only by the application + developer). + +For our scenario, we will create a +[Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) for each application. Let +us wrap a deployment in a [Cartographer ClusterTemplate](../reference/template/#clustertemplate), notating fields that +will need to vary and fields that can be hardcoded: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + name: app-deploy +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: # MUST BE UNIQUE + labels: + app: # MUST BE UNIQUE + spec: + replicas: # CAN BE HARDCODED + selector: + matchLabels: + app: # MUST BE UNIQUE + template: + metadata: + labels: + app: # MUST BE UNIQUE + spec: + containers: + - name: # MUST BE UNIQUE + image: # KNOWN ONLY TO THE DEV +``` + +We can see an example of a field that may be hardcoded. The number of replicas is a concern of the app operator and can +be set for all apps as policy. We can see the template evolve: + +```yaml +... +spec: + template: + apiVersion: apps/v1 + kind: Deployment + spec: + replicas: 3 + ... + ... +``` + +Next, we will fill the fields that must be unique. Every application will be specified in a workload object specified by +the developer. We know that these objects (as all kubernetes objects) have unique names in their namespaces. As such, we +can leverage that name for all fields that must be unique. That name can be found on the workload object at +`.metadata.name`. We can see that in use below: + +```yaml +... +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: $(workload.metadata.name)$-deployment + labels: + app: $(workload.metadata.name)$ + spec: + ... + selector: + matchLabels: + app: $(workload.metadata.name)$ + template: + metadata: + labels: + app: $(workload.metadata.name)$ + spec: + containers: + - name: $(workload.metadata.name)$ + ... +``` + +Finally, there are the fields that are known only to the dev. There are a few classes of information so fundamental to +the process of building and deploying applications that the workload has fields for their specification. We can see +these in [the reference for the Workload custom resource](../reference/workload/#workload). They include: + +- Location of source code +- Environment variables to use in building images +- Location of pre-built images +- Resource constraints +- And more… + +For our example, the workload field for specifying the location of pre-built images is exactly what is necessary. We +will expect the appropriate image address to be specified on the workload in the ".spec.image" field. + +```yaml +... +spec: + template: + ... + spec: + template: + spec: + containers: + - image: $(workload.spec.image)$ + ... + ... +``` + +Let’s look at the completed template: + +{{< tutorial "app-deploy-template.yaml" >}} + +Wonderful! As app operators, we have created the template desired for our supply chain. + +#### Service Account + +Next, we turn to permissions. For Cartographer to create objects it needs RBAC permission to do so. For that we’ll +create a service account that will specify the requisite permissions to create, update, and otherwise manage the objects +referred to in our templates. + +First let’s create a service account. For ease of use in this tutorial, we’ll create it in the default namespace. + +{{< tutorial cartographer-service-account.yaml >}} + +Now we’ll create the roles we want this service account to have. In our case, we need a role allowing us to manage +deployments. + +{{< tutorial deployment-role.yaml >}} + +Now we bind the role to our service account: + +{{< tutorial role-binding-dep-carto.yaml >}} + +Great, we’ve created all of the objects to which the supply chain will refer: templates and a service account. Let’s +create the supply chain! + +#### Supply Chain + +The supply chain has three top level fields in its spec, the resources, a service account reference and a selector for +workloads. + +We’ll start with the resources field, which is a list. Each item in the resource list represents an object that will be +stamped out in the supply chain of each workload. We give each a name and a reference to a template. In the example +below we can see the templateRef has the name and kind of the template: + + +```yaml + resources: + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy +``` + + +Next we’ll add the reference to the service account: + + +```yaml + serviceAccountRef: + name: cartographer-pre-built-sa + namespace: default +``` + + +Our last step with the supply chain is to specify the selector. This is the set of labels on a workload that will +indicate that the workload matches with this supply-chain, rather than some other supply chain. We specify the types of +workloads that are appropriate for this supply chain. For our example, we only want workloads that have pre-built images +to match with this supply chain. Let’s enforce that by expecting workloads that match to have the label +`workload-type: pre-built`: + + +```yaml + selector: + workload-type: pre-built +``` + + +We can bring these all together for our complete supply chain: + +{{< tutorial supply-chain.yaml >}} + +We’re now ready to submit these objects to our cluster. The next responsibility is for our app developers to submit +workloads. Let’s step into that role now. + +### App Dev Steps + +Our app operators have created a contract that needs to be fulfilled by our workload. First, we’ll need to ensure that +our workload has the necessary labels to match with the selector. We can see that on the workload below. + +```yaml +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + name: hello + labels: + workload-type: pre-built # <=== label matches selector of our supply chain +spec: ... +``` + +Next, our workload must provide all of the values referenced in the templates of the supply chain. We’ll remember that +for this supply chain that is the location of a pre-built image. Let’s point to +[our very sophisticated hello-world app](https://hub.docker.com/r/nginxdemos/hello/): +`docker.io/nginxdemos/hello:latest` + +```yaml +spec: + image: docker.io/nginxdemos/hello:latest +``` + +We bring it all together: + +{{< tutorial workload.yaml >}} + +And we're done! The app dev always has less work to do than the app operator. + +Let's submit all of these items to the cluster! + +## Observe + +### Workload + +The workload quickly resolves. We can examine the workload object: + +```shell +$ kubectl get -o yaml workload hello +``` + +```yaml +apiVersion: carto.run/v1alpha1 +kind: Workload +metadata: + generation: 1 + labels: + workload-type: pre-built + name: hello + namespace: default + ... +spec: + image: docker.io/nginxdemos/hello:latest + serviceAccountName: cartographer-pre-built-sa +status: + conditions: ... + observedGeneration: 1 + resources: ... + supplyChainRef: + kind: ClusterSupplyChain + name: supply-chain +``` + +Let's look more closely at the `status.conditions`: + +```yaml +status: + conditions: + - lastTransitionTime: ... + message: "" + reason: Ready + status: "True" + type: SupplyChainReady + - lastTransitionTime: ... + message: "" + reason: ResourceSubmissionComplete + status: "True" + type: ResourcesSubmitted + - lastTransitionTime: ... + message: "" + reason: Ready + status: "True" + type: Ready + ... +``` + +The `SupplyChainReady` condition merely tells us that the SupplyChain is in a healthy condition. e.g. if we do a +`kubectl get -o yaml clustersupplychain supply-chain`, we'll see that it's top level `Ready` condition is true. + +The `ResourcesSubmitted` condition is more important. It tells us that all of the objects that were specified in the +supply chain have been created. It is important to note, _this does not indicate that every submitted resource has +finished reconciling_. + +The `Ready` condition is the top level condition. For this condition to be true, all other conditions must be as well. A +quick scan of this value can let you know if the workload is in a ready state. + +You may want to more closely examine the resources deployed. Maybe a step in the supply chain is reporting a problem. Or +maybe you want to check if a successfully deployed object has completed reconciling. For such purposes we can use the +`status.resources` field. + +```yaml +status: + resources: + - name: deploy + stampedRef: + apiVersion: apps/v1 + kind: Deployment + name: hello-deployment + namespace: default + templateRef: + apiVersion: carto.run/v1alpha1 + kind: ClusterTemplate + name: app-deploy + ... +``` + +`status.resources` contains a reference to each of the objects created by the supply chain. For our single step supply +chain, we can see the name of that step/resource, "deploy". There is a reference to the template object (just as we +specified in the supply chain object) and a reference to the object that was stamped out. Let’s examine that object: + +### Stamped Object + +```shell +$ kubectl get -o yaml deployment hello-deployment +``` + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: ... +spec: ... +status: + availableReplicas: 3 + conditions: + - lastTransitionTime: ... + lastUpdateTime: ... + message: Deployment has minimum availability. + reason: MinimumReplicasAvailable + status: "True" + type: Available + - lastTransitionTime: ... + lastUpdateTime: ... + message: ReplicaSet "hello-deployment-5dddb657c" has successfully progressed. + reason: NewReplicaSetAvailable + status: "True" + type: Progressing + observedGeneration: 1 + readyReplicas: 3 + replicas: 3 + updatedReplicas: 3 +``` + +Success! We can tell that the deployment is ready on the cluster because its condition `Available` is true. + +We should also note some of the metadata of our created object: + +```yaml +metadata: + ownerReferences: + - apiVersion: carto.run/v1alpha1 + blockOwnerDeletion: true + controller: true + kind: Workload + name: hello + uid: d86b7dbf-8fcf-466e-abab-ebbc25404a06 + ... +``` + +First, we see an owner reference to our workload. This is helpful in a number of ways. It allows easy tracking of +relationships. If you use a tool like [kubectl tree](https://github.com/ahmetb/kubectl-tree) or +[kube-lineage](https://github.com/tohjustin/kube-lineage), examining the workload will display the child objects created +and the children of those objects. + +```shell +$ kubectl tree workload hello +``` + +```console +NAMESPACE NAME READY REASON AGE +default Workload/hello True Ready ... +default └─Deployment/hello-deployment - ... +default └─ReplicaSet/hello-deployment-abc123 - ... +default ├─Pod/hello-deployment-abc123-def45 True ... +default ├─Pod/hello-deployment-abc123-ghi67 True ... +default └─Pod/hello-deployment-abc123-jkl89 True ... +``` + +The other advantage of this owner relation is in cleanup. If the app dev deletes the workload, kubernetes will handle +deletion of the child objects. + +The other part of the metadata for us to observe are the labels: + +```yaml +metadata: ... + labels: + carto.run/cluster-template-name: app-deploy + carto.run/resource-name: deploy + carto.run/supply-chain-name: supply-chain + carto.run/template-kind: ClusterTemplate + carto.run/workload-name: hello + carto.run/workload-namespace: default +``` + +Cartographer adds helpful labels to indicate all of the Cartographer objects involved in the creation of a stamped +object (template, supply chain, workload). + +### Interacting with the app + +As a final step let’s create a quick port-forward and see our deployment serve traffic: + +```shell +$ kubectl port-forward deployment/hello-deployment 3000:80 +``` + +Now we visit the site! [http://localhost:3000/](http://localhost:3000/) + +We should see our app serving: + +{{< figure src="../../img/tutorials/hello-world-nginx.png" alt="Hello World" +width="400px" >}} + +Wonderful. Our company's first application has been delivered. + +## Steps for a second dev + +We use Cartographer to create an application platform for our developers. Let’s act now as a second dev, bringing a new +application to the platform. + +We assume that all the app operator setup from above remains. All we need to do is create a new workload with a +reference to a different pre-built image: + +{{< tutorial workload-2.yaml >}} + +We can follow the same steps to observe the workload and the created objects. And we can do a similar port-forward (note +that this app serves traffic on its 8000 port): + +```shell +$ kubectl port-forward deployment/whale-hello-there-deployment 3000:8000 +$ curl localhost:3000 +``` + +```html +
+Hello World
+
+
+                                       ##         .
+                                 ## ## ##        ==
+                              ## ## ## ## ##    ===
+                           /""""""""""""""""\___/ ===
+                      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
+                           \______ o          _,/
+                            \      \       _,'
+                             `'--.._\..--''
+
+``` + +## Wrap Up + +Congratulations, you’ve built your first supply chain! Here are some of the things you’ve learned: + +- The relationship between supply chains and templates +- The syntax for templates to refer to workload values +- The relationship between the selector on a supply chain and the labels on a workload +- The creation and referencing of a service account for Cartographer to manage the objects specified by the supply chain diff --git a/site/content/docs/development/tutorials/runnable-in-a-supply-chain.md b/site/content/docs/development/tutorials/runnable-in-a-supply-chain.md new file mode 100644 index 000000000..d1b30d2cb --- /dev/null +++ b/site/content/docs/development/tutorials/runnable-in-a-supply-chain.md @@ -0,0 +1,415 @@ +# Using Runnable in a Supply Chain + +## Overview + +In the previous tutorial we saw how Runnable brings updateable behavior to immutable kubernetes objects. In this +tutorial, we’ll see how we can use Runnable in our supply chains for common behavior like linting, scanning, and +testing. + +## Environment setup + +For this tutorial you will need a kubernetes cluster with Cartographer, kpack and Tekton installed. You can find +[Cartographer's installation instructions here](https://github.com/vmware-tanzu/cartographer#installation), kpack's +[here](https://github.com/pivotal/kpack/blob/main/docs/install.md) and Tekton's +[here](https://github.com/pivotal/kpack/blob/main/docs/install.md). + +You will also need an image registry for which you have read and write permission. + +You may choose to use the [./hack/setup.sh](https://github.com/vmware-tanzu/cartographer/blob/main/hack/setup.sh) script +to install a kind cluster with Cartographer, Tekton, kpack and a local registry. _This script is meant for our +end-to-end testing and while we rely on it working in that role, no user guarantees are made about the script._ + +Command to run from the Cartographer directory: + +```shell +$ ./hack/setup.sh cluster cartographer-latest example-dependencies +``` + +If you later wish to tear down this generated cluster, run + +```shell +$ ./hack/setup.sh teardown +``` + +## Scenario + +Continuing the scenario from [the previous tutorial](runnable.md), we remember that our CTO is interested in putting +quality controls in place; only code that passes certain checks should be built and deployed. They want to start small, +and have decided all source code repositories that are built must pass markdown linting. In order to do this we’re going +to leverage +[the markdown linting pipeline in the TektonCD catalog](https://github.com/tektoncd/catalog/tree/main/task/markdown-lint/0.1). + +Last tutorial we saw how to use Cartographer’s Runnable to give us easy updating behavior of Tekton (no need for Tekton +Triggers and Github Webhooks). In this following tutorial we’ll complete the scenario by using Runnable in a supply +chain. + +## Steps + +### App Operator Steps + +Much of our work from the previous tutorial remains the same. We deployed a Tekton pipeline and two Tekton Tasks in the +cluster. These objects will be applied in this tutorial with no change. + +We deployed a ClusterRunTemplate that templated a Tekton PipelineRun. This will stay largely the same, with only a +change to the outputs (We're going to define outputs that are slightly more useful than the ones we chose last +tutorial). We will still deploy this object directly. + +The object that we’ll deploy differently is the Runnable. To use Runnable in a supply chain we’ll wrap it in a +Cartographer template. This template will be referenced in our supply chain. This work should feel very familiar to the +steps we took in the [Build Your First Supply Chain](first-supply-chain.md) tutorial! + +#### Supply Chain + +Let’s begin by thinking through what template we need and where it will go in our supply chain. Our goal is to ensure +that the only repos that are built and deployed are those that pass linting. So we’ll need our new step to be the first +step in a supply chain. This step will receive the location of a source code and if the source code passes linting it +will pass that location inforation to the next step in the supply chain. Do you remember what template is meant to +expose information about the location of source code? That’s right, the ClusterSourceTemplate. + +Let’s define our supply chain now. We’ll start with the supply chain we created in the Extending a Supply Chain +tutorial. The resources then looked like this: + + +```yaml + resources: + - name: build-image + templateRef: + kind: ClusterImageTemplate + name: image-builder + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy-from-sc-image + images: + - resource: build-image + name: built-image +``` + + +We’ll add a new first step, lint source code. As we determined before, this will refer to a ClusterSourceTemplate. Our +second step will remain a ClusterImageTemplate, but it will have to be a new template. This is because it will consume +the source code from the previous step rather than directly from the workload. The rest of the resources will remain the +same. + + +```yaml + resources: + - name: lint-source + templateRef: + kind: ClusterSourceTemplate + Name: source-linter + - name: build-image + templateRef: + kind: ClusterImageTemplate + name: image-builder-from-previous-step + sources: + - resource: lint-source + name: source + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy-from-sc-image + images: + - resource: build-image + name: built-image +``` + + +Our final step with the supply chain will be referring to a service-account. Let's think through what permissions we +need: + +- the `source-linter` template will create a runnable +- the `image-builder-from-previous-step` template will create a kpack image (just as the supply chain from the + [Extending a Supply Chain](extending-a-supply-chain.md) tutorial) +- the `app-deploy-from-sc-image` template will create a deployment (just as the supply chain from the + [Extending a Supply Chain](extending-a-supply-chain.md) tutorial) + +The only new object created here is a runnable, which is a Cartographer resource. The Cartographer controller already +has permission to manipulate Cartographer resources. So we do not need to do any alterations, we can simply reuse the +service account (and roles and role bindings) from the [Extending a Supply Chain](extending-a-supply-chain.md) tutorial. + +_Note that while the supply chain refers to a service account, the Runnable itself also refers to a service account. +More on that in a moment._ + +Here is our complete supply chain. + +{{< tutorial supply-chain.yaml >}} + +#### Templates + +There are two new templates that need to be written, `image-builder-from-previous-step` and `source-linter`. Creating +the `image-builder-from-previous-step` will be left as an exercise for the reader. Refer to the Extending a Supply Chain +tutorial for help. Let's turn to creating the `source-linter` template. + +We know we’ll wrap our Runnable in a ClusterSourceTemplate. We’ll begin as always, taking our previously created +Runnable and simply pasting it into a ClusterSourceTemplate: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterSourceTemplate +metadata: + name: source-linter +spec: + template: + apiVersion: carto.run/v1alpha1 + kind: Runnable + metadata: + name: linter + spec: + runTemplateRef: + name: md-linting-pipelinerun + inputs: + repository: https://github.com/waciumawanjohi/demo-hello-world + revision: main + serviceAccountName: pipeline-run-management-sa + urlPath: ??? + revisionPath: ??? +``` + +We can quickly see that there are hardcoded values that we’ll want to replace with references to the workload (so that +our supply chain can work with many different workloads). These are the `inputs` values. Remember, these fields are +inputs to the ClusterRunTemplate. If a value could be hardcoded, it should not be a field in the Runnable’s inputs at +all, it should simply be hardcoded in the ClusterRunTemplate (e.g. every field in the runnable `spec.inputs` should +become a Cartographer variable). Let’s look at how we’ll change `inputs`: + + +```yaml + inputs: + repository: $(workload.spec.source.git.url)$ + revision: $(workload.spec.source.git.revision)$ +``` + + +Quite straightforward and familiar. Simply pull the requisite values from the workload. The next step will be +straightforward as well; we need to make sure that multiple apps don’t overwrite one Runnable object. We need to +template the Runnable’s name: + + +```yaml + metadata: + name: $(workload.metadata.name)$-linter +``` + + +Finally, we need to fill the `urlPath` and `revisionPath` to tell the ClusterSourceTemplate what field of the Runnable +to expose for the `url` and `revision`. Remember, that’s the contract of a ClusterSourceTemplate, it exposes those two +values to the rest of the supply chain. Runnables have a `.status` field, which we'll use. The contents of that field +are determined by fields on the ClusterRunTemplate. In the [previous tutorial](runnable.md) the ClusterRunTemplate +declared that the output would be called `lastTransitionTime`. Let's declare our intention now to change the +ClusterRunTemplate. In a moment we'll alter it to set new outputs named `url` and `revision`. That will allow us to +finish the ClusterSourceTemplate wrapping our runnable. + +```yaml +spec: + template: ... + urlPath: .status.outputs.url + revisionPath: .status.outputs.revision +``` + +Wonderful. Our ClusterSourceTemplate is complete: + +{{< tutorial source-linter-template.yaml >}} + +#### ClusterRunTemplate + +Our supply-chain will now stamp out a Runnable. But we still have to change the ClusterRunTemplate to ensure that the +status of the Runnable has the fields we want. Just a moment ago we decided that these fields should be `url` and +`revision`. Before we alter the previous ClusterRunTemplate from the previous tutorial, let's look at it: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterRunTemplate +metadata: + name: md-linting-pipelinerun +spec: + template: + apiVersion: tekton.dev/v1beta1 + kind: PipelineRun + metadata: + generateName: linter-pipeline-run- + spec: + pipelineRef: + name: linter-pipeline + params: + - name: repository + value: $(runnable.spec.inputs.repository)$ + - name: revision + value: $(runnable.spec.inputs.revision)$ + workspaces: + - name: shared-workspace + volumeClaimTemplate: + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 256Mi + outputs: + lastTransitionTime: .status.conditions[0].lastTransitionTime +``` + +We can see that the `spec.outputs` field was used to prescribe a `lastTransitionTime` field. We’ll change that to a +`url` and `revision` field: + + +```yaml + outputs: + url: ??? + revision: ??? +``` + + +Great. Now let’s think; where on the object that’s created will we find these values. Actually, why think about it; +let’s take a look at the pipelinerun that was created in the last tutorial! + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: linter-pipeline-run-123az + generateName: linter-pipeline-run- + labels: + carto.run/run-template-name: md-linting-pipelinerun + carto.run/runnable-name: linter + tekton.dev/pipeline: linter-pipeline + ownerReferences: + - apiVersion: carto.run/v1alpha1 + blockOwnerDeletion: true + controller: true + kind: Runnable + name: linter + uid: ... + ... +spec: + params: + - name: repository + value: https://github.com/waciumawanjohi/demo-hello-world + - name: revision + value: main + pipelineRef: + name: linter-pipeline + serviceAccountName: default + timeout: 1h0m0s + workspaces: + - name: shared-workspace + volumeClaimTemplate: + metadata: + creationTimestamp: null + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 256Mi + status: {} +status: + completionTime: ... + conditions: ... + pipelineSpec: ... + startTime: ... + taskRuns: ... +``` + +While it feels most natural to read outputs from the `.status` of objects, in this case the value we want is in the +`.spec`. We can see that the `.spec.params` of this object has the url and revision of the source code. That's the value +we wish to pass on if the pipeline-run is successful. We'll use jsonpath in our outputs to grab these values: + + +```yaml + outputs: + url: spec.params[?(@.name=="repository")].value + revision: spec.params[?(@.name=="revision")].value +``` + + +Great, our outputs are complete. + +There’s one more thing that we can do to make things easy on ourselves, differentiate the name of pipeline-runs of one +workload from those of another. Technically, we do not have to do this. Cartographer is smart enough to stamp out these +objects and differentiate objects created from the same prefix from Runnable 1 and Runnable 2. But we’re human and it’ll +be nice for us to quickly be able to distinguish. + +```yaml +spec: + template: + metadata: + generateName: $(runnable.metadata.name)$-pipeline-run- +``` + +Let’s pull it all together! Our ClusterRunTemplate now reads: + +{{< tutorial cluster-run-template.yaml >}} + +#### Runnable's Service Account + +The last object to mention is the serviceAccount object referred to in the ClusterSourceTemplate wrapped Runnable. We +could refer to the same service account referred to in the supply chain. Then we would simply bind an additional role to +the account, one that allows creation of a Tekton PipelineRun. Or we can refer to a different service account with these +permissions. We created such a service account in the [previous tutorial](runnable.md) and our runnable still refers to +that name. We will simply deploy that service account. + +Now we're ready. Let’s submit all these objects and step into our role as App Devs. + +### App Dev Steps + +As devs, our work is easy! We submit a workload. We’re being asked for the same information as ever from the templates, +a url and a revision for the location of the source code. We can submit the same workload from the +[Extending a Supply Chain tutorial](extending-a-supply-chain.md): + +{{< tutorial workload.yaml >}} + +## Observe + +Using [kubectl tree](https://github.com/ahmetb/kubectl-tree) we can see our workload is parent to a runnable which in +turn is parent to a pipeline-run. + +```shell +$ kubectl tree workload hello-again +``` + +```console +NAMESPACE NAME READY REASON AGE +default Workload/hello-again True Ready +default ├─Deployment/hello-again-deployment - +default │ └─ReplicaSet/hello-again-deployment-67b7dc6d5 - +default │ ├─Pod/hello-again-deployment-67b7dc6d5-djtph True +default │ ├─Pod/hello-again-deployment-67b7dc6d5-f2nkv True +default │ └─Pod/hello-again-deployment-67b7dc6d5-p4m9c True +default ├─Image/hello-again True +default │ ├─Build/hello-again-build-1 - +default │ │ └─Pod/hello-again-build-1-build-pod False PodCompleted +default │ ├─PersistentVolumeClaim/hello-again-cache - +default │ └─SourceResolver/hello-again-source True +default └─Runnable/hello-again-linter True Ready +default └─PipelineRun/hello-again-linter-pipeline-run-x123x - +default ├─PersistentVolumeClaim/pvc-1a89a7e201 - +default ├─TaskRun/hello-again-linter-pipeline-run-x123x-fetch-repository - +default │ └─Pod/hello-again-linter-pipeline-run-x123x-fetch-repository-pod False PodCompleted +default └─TaskRun/hello-again-linter-pipeline-run-x123x-md-lint-run - +default └─Pod/hello-again-linter-pipeline-run-x123x-md-lint-run-pod False PodCompleted +``` + +We also see that the workload is in a ready state, as are all of the pods of our deployment. + +## Wrap Up + +You’ve now built a supply chain leveraging runnables. Your app platform can now provide testing, scanning, linting and +more to all the applications brought by your dev teams. Let’s look at what you learned in this tutorial: + +- How to wrap a Runnable in a Cartographer template +- How to align the outputs from a ClusterRunTemplate with the values exposed by the template wrapping the Runnable + +Before we leave this tutorial, we should note that the supply chain that we’ve created deals very well with new apps +that are brought to the platform. That is, when a workload is submitted, the app will be linted and upon success will +proceed to be built. But this supply chain is not resilient to changes made to the source code of said app. What will +happen if the code was good, but is changed to a bad state? The linting step won’t rerun, as from the tekton +perspective, no value has changed; it still has the same url and revision. + +In order to address this problem, production supply chains should leverage a resource like +[fluxCD’s source controller](https://fluxcd.io/docs/components/source/). This kubernetes resource will translate a +revision like `main` into the revision of the current commit on main (e.g. commit `abc123`). When main is updated, +source controller will ensure that it outputs the new commit that is the head of main. Leveraging this resource, we can +avoid the dilemma presented above. + +Users can see supply chains that use fluxCD’s source controller in +[Cartographer’s examples](https://github.com/vmware-tanzu/cartographer/tree/main/examples). diff --git a/site/content/docs/development/tutorials/runnable.md b/site/content/docs/development/tutorials/runnable.md new file mode 100644 index 000000000..81f2dcc93 --- /dev/null +++ b/site/content/docs/development/tutorials/runnable.md @@ -0,0 +1,386 @@ +# Runnable: Templating Objects That Cannot Update + +## Overview + +In this tutorial we’ll explore a new Cartographer resource: runnable. Runnables will enable us to choreograph resources +that do not support standard update behavior. We’ll see how we can wrap Runnables around tools useful for testing, like +Tekton, and how that will provide us easy updating behavior of our testing objects. + +## Environment setup + +For this tutorial you will need a kubernetes cluster with Cartographer and Tekton installed. You can find +[Cartographer's installation instructions here](https://github.com/vmware-tanzu/cartographer#installation) and +[Tekton's installation instructions are here](https://github.com/pivotal/kpack/blob/main/docs/install.md). + +Alternatively, you may choose to use the +[./hack/setup.sh](https://github.com/vmware-tanzu/cartographer/blob/main/hack/setup.sh) script to install a kind cluster +with Cartographer and Tekton. _This script is meant for our end-to-end testing and while we rely on it working in that +role, no user guarantees are made about the script._ + +Command to run from the Cartographer directory: + +```shell +$ ./hack/setup.sh cluster cartographer-latest example-dependencies +``` + +If you later wish to tear down this generated cluster, run + +```shell +$ ./hack/setup.sh teardown +``` + +## Scenario + +Our CTO is interested in putting quality controls in place; only code that passes certain checks should be built and +deployed. They want to start small: all source code repositories that are built must pass markdown linting. In order to +do this we’re going to leverage +[the markdown linting pipeline in the TektonCD catalog](https://github.com/tektoncd/catalog/tree/main/task/markdown-lint/0.1). + +In this tutorial we’ll see how to use Cartographer’s Runnable to give us easy updating behavior of Tekton (no need for +Tekton Triggers and Github Webhooks). In [the next tutorial](runnable-in-a-supply-chain.md) we’ll complete the scenario +by using Runnable in a supply chain. + +## Steps + +### Tekton Basics + +Before using Cartographer, let’s think about how we would use Tekton on its own to lint a repo. First we would define a +pipeline: + +{{< tutorial pipeline.yaml >}} + +We would apply this pipeline to the cluster, along with the tasks. Those tasks are in the TektonCD Catalog: + +```shell +$ kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/git-clone/0.3/git-clone.yaml +$ kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/markdown-lint/0.1/markdown-lint.yaml +``` + +Finally, we need to create a pipeline-run object. This object provides the param and workspace values defined at the top +level `.spec` field of the pipeline. + +```yaml +--- +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: linter-pipeline-run +spec: + pipelineRef: + name: linter-pipeline + params: + - name: repository + value: https://github.com/waciumawanjohi/demo-hello-world + - name: revision + value: main + workspaces: + - name: shared-workspace + volumeClaimTemplate: + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 256Mi +``` + +Importantly, this pipeline-run object will kick off a single run of the pipeline. The run will either succeed or fail. +The outcome will be written into the pipeline-run’s status. No later changes to the pipeline-run object will change +those outcomes; the run happens once. + +To see this in action, let’s apply the above pipeline-run. If we watch the object, we’ll soon see that it succeeds. + +```shell +$ watch 'kubectl get -o yaml pipelinerun linter-pipeline-run | yq .status.conditions' +``` + +Eventually yields the result: + +```yaml +- lastTransitionTime: ... + message: "Tasks Completed: 2 (Failed: 0, Cancelled 0), Skipped: 0" + reason: "Succeeded" + status: "True" + type: "Succeeded" +``` + +### Templating Pipeline Runs + +This seems like a very easy step to encode in a supply chain. All we would need to do is ensure that the Tekton tasks +and pipeline are created beforehand. Then a supply chain could stamp out a templated pipeline-run object. This template +will pull the repository and revision value from the workload. But there’s a problem... what happens if an app dev +changes one of these values on the workload? Our supply chain would not properly reflect the change, because the +pipeline-run object cannot be updated! + +Fortunately there’s an easy fix. We’re going to use a pair of new Cartographer resources: Runnable and +ClusterRunTemplate. + +It will be the responsibility of the ClusterRunTemplate to template our desired object (in this case, our Tekton +pipeline-run). The Runnable will be responsible for providing values to the ClusterRunTemplate, the values that we +expect to vary (in our example, the url and revision of the source code). When the set of values from the Runnable +changes, a new object will be created (rather than the old object updated). + +The Runnable will also expose results. Of course, multiple results will exist, a result for each of the objects created. +Runnable will only expose the results from the most recently submitted successful object. + +In this manner, we get a wrapper (Runnable) that is updateable and updates results in its status. This is similar to the +default behavior of kuberenetes objects. By wrapping Tekton pipeline-runs (or any immutable resource) in a Runnable, we +will be able to use the resource in a supply chain as if it were mutable. + +Let’s see the Runnable and the ClusterRunTemplate at work. Once we’re solid on those, we’ll use Runnable in a Supply +Chain in the next tutorial. + +#### ClusterRunTemplate + +Let’s start with the ClusterRunTemplate. As can be expected from the name, there’s a `.spec.template` field in it, where +we will write something very similar to our pipeline-run above. In fact, let’s write that exact pipeline-run in and then +look at the values that will need to change: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterRunTemplate +metadata: + name: md-linting-pipelinerun +spec: + template: + apiVersion: tekton.dev/v1beta1 + kind: PipelineRun + metadata: + name: linter-pipeline-run # <=== Can’t all have the same name + spec: + pipelineRef: + name: linter-pipeline + params: # <=== These param values will change + - name: repository + value: https://github.com/waciumawanjohi/demo-hello-world + - name: revision + value: main + workspaces: + - name: shared-workspace + volumeClaimTemplate: + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 256Mi +``` + +Most fields are fine. The name field is not. Why not? If we want to change the values for the pipeline-run, we’re not +going to update the templated object. We’re going to create an entirely new object. And of course that new object can’t +have the same hardcoded name. To handle this, every object templated in a ClusterRunTemplate specifies a `generateName` +rather than a `name`. We can use `linter-pipeline-run-` and kubernetes will handle putting a unique suffix on the name +of each pipeline-run. + + +```yaml + metadata: + generateName: linter-pipeline-run- +``` + + +The other change we want to make is to the values on the params. It doesn’t do much good to hardcode +`https://github.com/waciumawanjohi/demo-hello-world` into the repository param; this is the value we want to update. As +we said, we intend to update the Runnable and have the value in that update stamped out in a new pipeline-run object. So +we replace the hardcoded value with a Cartographer parameter. We’ll find the value that we want on the runnable. That +will look like `$(runnable.spec.inputs.repository)$`. This specifies that the value we want will be found in the +runnable spec, in a field named inputs, one of which will have the name `repository`. + +There’s only one more thing that we need to specify on the ClusterRunTemplate: what the outputs will be! We said that +the Runnable status will reflect results of a successful run. The ClusterRunTemplate specifies what results. For now +let’s simply report the lastTransition time that we saw on the conditions. We use jsonpath to indicate the location of +this value on the objects that are being stamped: + + +```yaml + outputs: + lastTransitionTime: .status.conditions[0].lastTransitionTime +``` + + +Let’s look at our complete ClusterRunTemplate: + +{{< tutorial cluster-run-template.yaml >}} + +#### Runnable + +Now let’s make the Runnable. First we’ll specify which ClusterRunTemplate our Runnable works with. We do this in the +Runnable's `.spec.runTemplateRef.` field and we refer to the name of the ClusterRunTemplate we just created. + + +```yaml + runTemplateRef: + name: md-linting-pipelinerun +``` + + +Next we’ll fill in the inputs. These are the paths that we templated into the ClusterRunTemplate param values, the +`runnable.spec.inputs.repository` and `runnable.spec.inputs.revision`. We’ll fill these with the values we previously +hardcoded in the pipelinerun. With runnable we’ll be able to update them later as we like. + + +```yaml + inputs: + repository: https://github.com/waciumawanjohi/demo-hello-world + revision: main +``` + + +Finally, we need a serviceAccountName. Just like the supply-chain, with Runnable Cartographer could be stamping out +_anything_. Using RBAC we expect a service account to provide permissions to Cartographer and limit it to creating only +the types of objects we expect. We'll create a service account named `pipeline-run-management-sa`. We’ll put that name +in our Runnable object. The full object looks like this: + +{{< tutorial runnable.yaml >}} + +Let’s quickly create the service account we referenced: + +{{< tutorial service-account.yaml >}} + +Great! Let’s deploy these objects. + +## Observe + +Let’s observe the pipeline-run objects in the cluster: + +```shell +$ kubectl get pipelineruns +``` + +We can see that a new pipelinerun has been created with the `linter-pipeline-run-` prefix: + +```console +NAME SUCCEEDED REASON STARTTIME COMPLETIONTIME +linter-pipeline-run-123az True Succeeded 2m48s 2m35s +``` + +Examining the created object it’s a non-trivial 300 lines: + +```shell +$ kubectl get -o yaml pipelineruns linter-pipeline-run-123az +``` + +In the metadata we can see familiar labels indicating Carto objects used to create this templated object. We can also +see that the object is owned by the runnable. + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: linter-pipeline-run-123az + generateName: linter-pipeline-run- + labels: + carto.run/run-template-name: md-linting-pipelinerun + carto.run/runnable-name: linter + tekton.dev/pipeline: linter-pipeline + ownerReferences: + - apiVersion: carto.run/v1alpha1 + blockOwnerDeletion: true + controller: true + kind: Runnable + name: linter + uid: ... + ... +``` + +The spec contains the spec that we templated out. Looks great. + +```yaml +spec: + params: + - name: repository + value: https://github.com/waciumawanjohi/demo-hello-world + - name: revision + value: main + pipelineRef: + name: linter-pipeline + serviceAccountName: default + timeout: 1h0m0s + workspaces: + - name: shared-workspace + volumeClaimTemplate: + metadata: + creationTimestamp: null + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 256Mi + status: {} +``` + +The status contains fields expected of Tekton: + +```yaml +status: + completionTime: ... + conditions: + - lastTransitionTime: "2022-03-07T19:24:35Z" + message: "Tasks Completed: 2 (Failed: 0, Cancelled 0), Skipped: 0" + reason: Succeeded + status: "True" + type: Succeeded + pipelineSpec: ... + startTime: ... + taskRuns: ... +``` + +To learn more about Tekton’s behavior, readers will want to refer to [Tekton documentation](https://tekton.dev/docs/). + +Now we examine the Cartographer Runnable object. We expect it to expose values from our successful object. + +```shell +$ kubectl get -o yaml runnable linter +``` + +```yaml +apiVersion: carto.run/v1alpha1 +kind: Runnable +Metadata: ... +Spec: ... +status: + conditions: ... + observedGeneration: ... + outputs: + lastTransitionTime: "2022-03-07T19:24:35Z" +``` + +Wonderful! The value from the field we specified in the ClusterRunTemplate is now in the outputs of the Runnable. + +Finally, let's update our runnable with a new repository: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: Runnable +metadata: + name: linter +spec: + runTemplateRef: + name: md-linting-pipelinerun + inputs: + repository: https://github.com/kelseyhightower/nocode # <=== new repo + revision: master # <=== new revision + serviceAccountName: pipeline-run-management-sa +``` + +When we apply this to the cluster, we can observe: + +- The spec of our runnable is updated +- The runnable causes the creation of a new pipelinerun +- The new pipeline run fails because the new repo does not pass linting +- Since the result of the new pipeline run is failure, our runnable output remains the output of our previous + (successful) pipeline run + +## Wrap Up + +In this tutorial you learned: + +- Some useful kubernetes objects cannot be updated +- Runnable allows you to add updateable behavior to those objects +- How to write Runnables with input values for ClusterRunTemplates +- How to write ClusterRunTemplates that specify outputs for the Runnable +- How to read output values from the Runnable + +We’ve got an updateable object, Runnable, that can manage tekton pipelines and tasks. Our next step is going to be using +Runnable in a supply chain. diff --git a/site/content/docs/development/tutorials/using-params.md b/site/content/docs/development/tutorials/using-params.md new file mode 100644 index 000000000..0f821e2e2 --- /dev/null +++ b/site/content/docs/development/tutorials/using-params.md @@ -0,0 +1,231 @@ +# Using Params + +## Overview + +In this tutorial we’ll explore how to use params to pass information that isn’t anticipated by the standard workload +fields. We’ll see how params can either be set or delegated to other objects in the supply chain. + +## Environment setup + +For this tutorial you will need a kubernetes cluster with Cartographer installed. You may follow +[the installation instructions here](https://github.com/vmware-tanzu/cartographer#installation). + +Alternatively, you may choose to use the +[./hack/setup.sh](https://github.com/vmware-tanzu/cartographer/blob/main/hack/setup.sh) script to install a kind cluster +with Cartographer. _This script is meant for our end-to-end testing and while we rely on it working in that role, no +user guarantees are made about the script._ + +Command to run from the Cartographer directory: + +```shell +$ ./hack/setup.sh cluster cartographer-latest +``` + +If you later wish to tear down this generated cluster, run + +```shell +$ ./hack/setup.sh teardown +``` + +## Scenario + +As in the tutorial ["Build Your First Supply Chain"](first-supply-chain.md), we will act as both the app dev and app +operator in a company creating hello world web applications. Our applications will again have been built and stored in a +registry. But now each image will be in some private registry and each app dev will need to provide the appropriate +service account with imagePullCredentials for our deployment. We will assume that the app devs have already created the +necessary service account in their namespace. We will see how to write our templates to accept a parameter with the +service account name as well as how to write a workload to supply that value. + +## Steps + +### App Operator Steps + +A new field must be added to our template of a deployment: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + name: app-deploy +spec: + template: + apiVersion: apps/v1 + kind: Deployment + metadata: + name: $(workload.metadata.name)$-deployment + labels: + app: $(workload.metadata.name)$ + spec: + replicas: 3 + selector: + matchLabels: + app: $(workload.metadata.name)$ + template: + metadata: + labels: + app: $(workload.metadata.name)$ + spec: + serviceAccountName: # <=== NEW FIELD + containers: + - name: $(workload.metadata.name)$ + image: $(workload.spec.image)$ +``` + +Let’s assume that the app operator has created an image registry in which they have expansive read credentials. The +operator can reasonably expect that many devs will use this registry. So the operator can be responsible for the +creation of a service account with the correct imagePullCredentials and can make sure this object is in the expected +namespaces with the expected name. Let’s set this expected name as "expected-service-account". + +While this will be the default value, we will allow the developer to override this choice. We’ll do this by setting the +name as a param for the ClusterTemplate: + +```yaml +apiVersion: carto.run/v1alpha1 +kind: ClusterTemplate +metadata: + ... +spec: + template: + ... + spec: + ... + template: + ... + spec: + serviceAccountName: $(params.image-pull-sa-name)$ + ... + params: + - name: image-pull-sa-name + default: expected-service-account +``` + +Here we see two changes to the ClusterTemplate’s spec: + +- The first is a field in the template. The field has been filled with a reference to a param: + `$(params.image-pull-sa-name)$` +- Second, we see that a new field has been introduced to the spec: `params`. This is a list of params, each of which + requires a name and a default value. Here, the app operator indicates that many devs will have a service account named + `expected-service-account` in their namespace. + +We can see here the full object we'll apply to the cluster: + +{{< tutorial app-deploy-template.yaml >}} + +We apply this template to the cluster along with the same supply chain from the +["Build Your First Supply Chain"](first-supply-chain.md) tutorial. + +### App Dev Steps + +As app devs, we’ve decided not to follow the app operator service account naming convention. We’ve created a service +account named "unconventionally-named-service-account" which has the imagePullSecrets to get our app image. In the +workload, we'll create a param. Params in workloads require two fields, a name and a value. We must give our param the +same name as the param in the template, `image-pull-sa-name`. And for the value, we'll provide our chosen service +account's name. + +```yaml +spec: + params: + - name: image-pull-sa-name + value: unconventionally-named-service-account +``` + +We can now apply this workload to the cluster: + +{{< tutorial workload.yaml >}} + +## Observe + +When we observe the created deployment, we can see that the value specified by the workload is present: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: hello-again-deployment + ... +spec: + template: + metadata: ... + spec: + Containers: ... + dnsPolicy: ... + restartPolicy: ... + schedulerName: ... + securityContext: {} + serviceAccount: unconventionally-named-service-account # <=== huzzah! + serviceAccountName: unconventionally-named-service-account + terminationGracePeriodSeconds: 30 + ... +status: ... +``` + +## Further information + +Params are created to allow delegation between different personas. The individual writing a template may be different +from the person writing the supply chain. Perhaps the template author is unsure of a parameter’s value, but the supply +chain author knows exactly the desired value and does not want the workload author to be able to override their choice. +The supply chain can be altered to specify a `value`. + +```yaml +--- +apiVersion: carto.run/v1alpha1 +kind: ClusterSupplyChain +metadata: + name: supply-chain +spec: + selector: + workload-type: pre-built + + resources: + - name: deploy + templateRef: + kind: ClusterTemplate + name: app-deploy + params: # <=== New field + - name: image-pull-sa-name + value: inevitable-service-account +``` + +If the supply chain is redeployed with this definition, we can observe that the deployment changes: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: hello-deployment + ... +spec: + template: + metadata: ... + spec: + Containers: ... + dnsPolicy: ... + restartPolicy: ... + schedulerName: ... + securityContext: {} + serviceAccount: inevitable-service-account + serviceAccountName: inevitable-service-account + terminationGracePeriodSeconds: 30 + ... +status: ... +``` + +Further information about params and the order of precedence can be found in the +["Parameter Heirarchy"](../architecture/#parameter-hierarchy) architecture documentation. + +In the Cartographer tests you can find an example of creating an object with numerous parameters which demonstrates the +precedence rules: + +- [A template with many param fields, providing default values for some](https://github.com/vmware-tanzu/cartographer/tree/main/tests/kuttl/supplychain/params-supply-chain/00-proper-templates.yaml) +- [A supply chain providing some defaults and some values for params](https://github.com/vmware-tanzu/cartographer/tree/main/tests/kuttl/supplychain/params-supply-chain/01-supply-chain.yaml) +- [A workload providing some values for params](https://github.com/vmware-tanzu/cartographer/tree/main/tests/kuttl/supplychain/params-supply-chain/02-workload.yaml) +- [The expected object which will be created when that trio is submitted to the cluster](https://github.com/vmware-tanzu/cartographer/tree/main/tests/kuttl/supplychain/params-supply-chain/02-assert.yaml) + +## Wrap Up + +Congratulations, you’ve used parameters in your supply chain! You’ve learned: + +- How a template requests information from a workload not available in the standard workload fields +- How to provide default values for params +- How the supply chain can provide mandatory values for params +- How to find more information on params diff --git a/site/data/docs/development-toc.yml b/site/data/docs/development-toc.yml index 440001628..1d7ddc712 100644 --- a/site/data/docs/development-toc.yml +++ b/site/data/docs/development-toc.yml @@ -37,6 +37,18 @@ toc: url: /reference/template - page: Runnable url: /reference/runnable + - title: Tutorials + subfolderitems: + - page: First Supply Chain + url: /tutorials/first-supply-chain + - page: Using Params + url: /tutorials/using-params + - page: Extending a Supply Chain + url: /tutorials/extending-a-supply-chain + - page: Runnable + url: /tutorials/runnable + - page: Runnable in a Supply Chain + url: /tutorials/runnable-in-a-supply-chain - title: Examples subfolderitems: - page: Multi-Cluster diff --git a/site/themes/template/layouts/shortcodes/tutorial.html b/site/themes/template/layouts/shortcodes/tutorial.html new file mode 100644 index 000000000..367b7b108 --- /dev/null +++ b/site/themes/template/layouts/shortcodes/tutorial.html @@ -0,0 +1,5 @@ +{{ $version := .Page.CurrentSection.Params.version }} +{{ $tutorialTitle := .Page.File.TranslationBaseName }} +{{ $filename := (path.Join "/content/docs" $version "tutorials/files" $tutorialTitle (.Get 0)) }} +{{ $file := $filename | readFile }} +{{ (print "```yaml\n" $file "\n```") | markdownify }} \ No newline at end of file