From 17c1790187e33441343afbff1f322e5801f434e4 Mon Sep 17 00:00:00 2001 From: Souvik Sarkar Date: Wed, 16 Mar 2022 17:29:28 +0530 Subject: [PATCH] pipelines as code pipelines as code pipelines as code pipelines as code intermediate check in intermediate check in intermediate check in complete draft complete draft minor tweaks attribute substitution and some minor fixes review comments added links to tkn-pac binaries rebased minor peer review comments intermediate work intermediate revisions incorporated Shivam's comment intermediate revisions SME review comments forget saving a file peer review comments fixing build error peer review comments --- _attributes/common-attributes.adoc | 1 + _topic_maps/_topic_map.yml | 2 + ...ting-applications-with-cicd-pipelines.adoc | 1 + cicd/pipelines/using-pipelines-as-code.adoc | 50 ++++++++ ...t-repository-hosting-service-provider.adoc | 21 ++++ ...ng-pipelines-as-code-for-a-github-app.adoc | 97 ++++++++++++++ ...izing-pipelines-as-code-configuration.adoc | 30 +++++ .../op-installing-pipelines-as-code-cli.adoc | 61 +++++++++ ...lines-as-code-on-an-openshift-cluster.adoc | 76 +++++++++++ ...p-pipelines-as-code-command-reference.adoc | 118 ++++++++++++++++++ 10 files changed, 457 insertions(+) create mode 100644 cicd/pipelines/using-pipelines-as-code.adoc create mode 100644 modules/op-configuring-pipelines-as-code-for-a-git-repository-hosting-service-provider.adoc create mode 100644 modules/op-configuring-pipelines-as-code-for-a-github-app.adoc create mode 100644 modules/op-customizing-pipelines-as-code-configuration.adoc create mode 100644 modules/op-installing-pipelines-as-code-cli.adoc create mode 100644 modules/op-installing-pipelines-as-code-on-an-openshift-cluster.adoc create mode 100644 modules/op-pipelines-as-code-command-reference.adoc diff --git a/_attributes/common-attributes.adoc b/_attributes/common-attributes.adoc index da708eddbdcf..1fdbc5d05868 100644 --- a/_attributes/common-attributes.adoc +++ b/_attributes/common-attributes.adoc @@ -76,6 +76,7 @@ endif::[] :pipelines-ver: pipelines-1.7 :tekton-chains: Tekton Chains :tekton-hub: Tekton Hub +:pac: Pipelines as Code //odo :odo-title: odo //alibaba cloud diff --git a/_topic_maps/_topic_map.yml b/_topic_maps/_topic_map.yml index f931709da935..7929d63fcefc 100644 --- a/_topic_maps/_topic_map.yml +++ b/_topic_maps/_topic_map.yml @@ -1582,6 +1582,8 @@ Topics: File: managing-nonversioned-and-versioned-cluster-tasks - Name: Using Tekton Hub with OpenShift Pipelines File: using-tekton-hub-with-openshift-pipelines + - Name: Using Pipelines as Code + File: using-pipelines-as-code - Name: Working with OpenShift Pipelines using the Developer perspective File: working-with-pipelines-using-the-developer-perspective - Name: Reducing resource consumption of OpenShift Pipelines diff --git a/cicd/pipelines/creating-applications-with-cicd-pipelines.adoc b/cicd/pipelines/creating-applications-with-cicd-pipelines.adoc index 7811e689bcf4..68682192b55f 100644 --- a/cicd/pipelines/creating-applications-with-cicd-pipelines.adoc +++ b/cicd/pipelines/creating-applications-with-cicd-pipelines.adoc @@ -74,6 +74,7 @@ include::modules/op-enabling-monitoring-of-event-listeners-for-triggers-for-user [id="pipeline-addtl-resources"] == Additional resources +* To include pipelines as code along with with the application source code in the same repository, see xref:../../cicd/pipelines/using-pipelines-as-code.adoc#using-pipelines-as-code[Using Pipelines as code]. * For more details on pipelines in the *Developer* perspective, see the xref:../../cicd/pipelines/working-with-pipelines-using-the-developer-perspective.adoc#working-with-pipelines-using-the-developer-perspective[working with pipelines in the *Developer* perspective] section. * To learn more about Security Context Constraints (SCCs), see the xref:../../authentication/managing-security-context-constraints.adoc#managing-pod-security-policies[Managing Security Context Constraints] section. * For more examples of reusable tasks, see the link:https://github.com/openshift/pipelines-catalog[OpenShift Catalog] repository. Additionally, you can also see the Tekton Catalog in the Tekton project. diff --git a/cicd/pipelines/using-pipelines-as-code.adoc b/cicd/pipelines/using-pipelines-as-code.adoc new file mode 100644 index 000000000000..29541f5b6044 --- /dev/null +++ b/cicd/pipelines/using-pipelines-as-code.adoc @@ -0,0 +1,50 @@ +:_content-type: ASSEMBLY +[id="using-pipelines-as-code"] += Using Pipelines as Code +include::_attributes/common-attributes.adoc[] +:context: using-pipelines-as-code + +toc::[] + +:FeatureName: Pipelines as Code +include::snippets/technology-preview.adoc[] + +[role="_abstract"] +With {pac}, cluster administrators and users with the required privileges can define pipeline templates as part of source code Git repositories. When triggered by a source code push or a pull request for the configured Git repository, the feature runs the pipeline and reports the status. + +[id="pac-key-features"] +== Key features +{pac} supports the following features: + +* Pull request status and control on the platform hosting the Git repository. +* GitHub Checks API to set the status of a pipeline run, including rechecks. +* GitHub pull request and commit events. +* Pull request actions in comments, such as `/retest`. +* Git events filtering and a separate pipeline for each event. +* Automatic task resolution in {pipelines-shortname}, including local tasks, Tekton Hub, and remote URLs. +* Retrieval of configurations using GitHub blobs and objects API. +* Access Control List (ACL) over a GitHub organization, or using a Prow style `OWNER` file. +* The `tkn-pac` CLI plugin for managing bootstrapping and {pac} repositories. +* Support for GitHub App, GitHub Webhook, Bitbucket Server, and Bitbucket Cloud. + +include::modules/op-installing-pipelines-as-code-on-an-openshift-cluster.adoc[leveloffset=+1] + +include::modules/op-installing-pipelines-as-code-cli.adoc[leveloffset=+1] + +include::modules/op-configuring-pipelines-as-code-for-a-git-repository-hosting-service-provider.adoc[leveloffset=+1] + +include::modules/op-configuring-pipelines-as-code-for-a-github-app.adoc[leveloffset=+2] + +include::modules/op-pipelines-as-code-command-reference.adoc[leveloffset=+1] + +include::modules/op-customizing-pipelines-as-code-configuration.adoc[leveloffset=+1] + +[role="_additional-resources"] +[id="additional-resources-pac"] +== Additional resources + +* xref:../../cicd/pipelines/installing-pipelines.adoc#installing-pipelines[Installing OpenShift Pipelines] + +* xref:../../cli_reference/tkn_cli/installing-tkn.adoc#installing-tkn[Installing tkn] + +* xref:../../cicd/pipelines/op-release-notes.adoc#op-release-notes[{pipelines-title} release notes] \ No newline at end of file diff --git a/modules/op-configuring-pipelines-as-code-for-a-git-repository-hosting-service-provider.adoc b/modules/op-configuring-pipelines-as-code-for-a-git-repository-hosting-service-provider.adoc new file mode 100644 index 000000000000..d867e670efa8 --- /dev/null +++ b/modules/op-configuring-pipelines-as-code-for-a-git-repository-hosting-service-provider.adoc @@ -0,0 +1,21 @@ +// This module is included in the following assembly: +// +// *cicd/pipelines/using-pipelines-as-code.adoc + +:_content-type: REFERENCE +[id="configuring-pipelines-as-code-for-a-git-repository-hosting-service-provider_{context}"] += Configuring {pac} for a Git repository hosting service provider + +[role="_abstract"] +After installing {pac}, cluster administrators can configure a Git repository hosting service provider. Currently, the following services are supported: + +* Github App +* Github Webhook +* Bitbucket Server +* Bitbucket Cloud + +[NOTE] +==== +GitHub App is the recommended service for using Pipelines as Code. +==== + diff --git a/modules/op-configuring-pipelines-as-code-for-a-github-app.adoc b/modules/op-configuring-pipelines-as-code-for-a-github-app.adoc new file mode 100644 index 000000000000..645b81a27711 --- /dev/null +++ b/modules/op-configuring-pipelines-as-code-for-a-github-app.adoc @@ -0,0 +1,97 @@ +// This module is included in the following assembly: +// +// *cicd/pipelines/using-pipelines-as-code.adoc + +:_content-type: PROCEDURE +[id="configuring-pipelines-as-code-for-a-github-app_{context}"] += Configuring {pac} for a GitHub App + +[role="_abstract"] +GitHub Apps act as a point of integration with {pipelines-title} and bring the advantage of Git-based workflows to OpenShift Pipelines. Cluster administrators can configure a single GitHub App for all cluster users. For GitHub Apps to work with Pipelines as Code, ensure that the webhook of the GitHub App points to the Pipelines as Code event listener route (or ingress endpoint) that listens for GitHub events. + +[id="configuring-github-app-for-pac"] +== Configuring a GitHub App + +Cluster administrators can create a GitHub App by running the following command: + +[source,terminal] +---- +$ tkn pac bootstrap github-app +---- + +If the `tkn pac` CLI plugin is not installed, you can create the GitHub App manually. + +[discrete] +.Procedure + +To create and configure a GitHub App manually for {pac}, perform the following steps: + +. Sign in to your GitHub account. + +. Go to **Settings** --> **Developer settings** --> **GitHub Apps**, and click **New GitHub App**. + +. Provide the following information in the GitHub App form: + +* **GitHub Application Name**: `OpenShift Pipelines` +* **Homepage URL**: OpenShift Console URL +* **Webhook URL**: The {pac} route or ingress URL. You can find it by executing the command `echo https://$(oc get route -n pipelines-as-code el-pipelines-as-code-interceptor -o jsonpath='{.spec.host}')`. ++ +[NOTE] +==== +For {pac} installated by default using the {pipelines-title} Operator, use the `openshift-pipelines` namespace instead of `pipelines-as-code`. +==== ++ +* **Webhook secret**: An arbitrary secret. You can generate a secret by executing the command `openssl rand -hex 20`. + +. Select the following **Repository permissions**: + +* **Checks**: `Read & Write` +* **Contents**: `Read & Write` +* **Issues**: `Read & Write` +* **Metadata**: `Read-only` +* **Pull request**: `Read & Write` + +. Select the following **Organization permissions**: + +* **Members**: `Readonly` +* **Plan**: `Readonly` + +. Select the following **User permissions**: + +* **Commit comment** +* **Issue comment** +* **Pull request** +* **Pull request review** +* **Pull request review comment** +* **Push** + +. Click **Create GitHub App**. + +. On the **Details** page of the newly created GitHub App, note the **App ID** displayed at the top. + +. In the **Private keys** section, click **Generate Private key** to automatically generate and download a private key for the GitHub app. Securely store the private key for future reference and usage. + + +[id="configuring-pac-for-github-app"] +== Configuring {pac} to access a GitHub App + +To configure {pac} to access the newly created GitHub App, execute the following command: ++ +[source,terminal] +---- +$ oc -n create secret generic pipelines-as-code-secret \ <1> + --from-literal github-private-key="$(cat )" \ <2> + --from-literal github-application-id="" \ <3> + --from-literal webhook.secret="" <4> +---- +<1> For {pac} installated by default using the {pipelines-title} Operator, use the `openshift-pipelines` namespace instead of `pipelines-as-code`. +<2> The path to the private key you downloaded while configuring the GitHub App. +<3> The **App ID** of the GitHub App. +<4> The webhook secret provided when you created the GitHub App. + + +[NOTE] +==== +{pac} works automatically with GitHub Enterprise by detecting the header set from GitHub Enterprise and using it for the GitHub Enterprise API authorization URL. +==== + diff --git a/modules/op-customizing-pipelines-as-code-configuration.adoc b/modules/op-customizing-pipelines-as-code-configuration.adoc new file mode 100644 index 000000000000..61eec1456ca2 --- /dev/null +++ b/modules/op-customizing-pipelines-as-code-configuration.adoc @@ -0,0 +1,30 @@ +// This module is included in the following assembly: +// +// *cicd/pipelines/using-pipelines-as-code.adoc + +:_content-type: REFERENCE +[id="customizing-pipelines-as-code-configuration_{context}"] += Customizing {pac} configuration + +[role="_abstract"] +To customize {pac}, cluster administrators can configure the following parameters using the `pipelines-as-code` config map in the `pipelines-as-code` namespace: + +.Customizing {pac} configuration +[options="header"] +|=== + +| Parameter | Description | Default + +| `application-name` | The name of the application. For example, the name displayed in the GitHub Checks labels. | `"Pipelines as Code CI"` + +| `max-keep-days` | The number of the days for which the executed pipeline runs are kept in the `pipelines-as-code namespace`. + +Note that this configmap setting does not affect the cleanups of a user's pipeline runs, which are controlled by the annotations on the pipeline run definition in the user's GitHub repository. | + +| `secret-auto-create` | Indicates whether or not a secret should be automatically created using the token generated in the GitHub application. This secret can then be used with private repositories. | `enabled` + +| `remote-tasks` | When enabled, allows remote tasks from pipeline run annotations. | `enabled` + +| `hub-url` | The base URL for the link:https://api.hub.tekton.dev/v1[Tekton Hub API]. | `https://hub.tekton.dev/` + +|=== \ No newline at end of file diff --git a/modules/op-installing-pipelines-as-code-cli.adoc b/modules/op-installing-pipelines-as-code-cli.adoc new file mode 100644 index 000000000000..35eb5f90fb4b --- /dev/null +++ b/modules/op-installing-pipelines-as-code-cli.adoc @@ -0,0 +1,61 @@ +// This module is included in the following assembly: +// +// *cicd/pipelines/using-pipelines-as-code.adoc + +:_content-type: PROCEDURE +[id="installing-pipelines-as-code-cli_{context}"] += Installing {pac} CLI + +[role="_abstract"] +Cluster administrators can use the `tkn-pac` CLI tool on local machines or as containers for testing. The `tkn-pac` CLI tool is installed automatically when you install the `tkn` CLI for {pipelines-title}. + +You can also install the `tkn-pac` `tkn-pac` version `0.23.1` binaries for the supported platforms: + +* link:https://mirror.openshift.com/pub/openshift-v4/clients/pipeline/0.23.1/tkn-pac-linux-amd64-0.23.1.tar.gz[Linux (x86_64, amd64)] +* link:https://mirror.openshift.com/pub/openshift-v4/clients/pipeline/0.23.1/tkn-pac-linux-s390x-0.23.1.tar.gz[Linux on IBM Z and LinuxONE (s390x)] +* link:https://mirror.openshift.com/pub/openshift-v4/clients/pipeline/0.23.1/tkn-pac-linux-ppc64le-0.23.1.tar.gz[Linux on IBM Power Systems (ppc64le)] +* link:https://mirror.openshift.com/pub/openshift-v4/clients/pipeline/0.23.1/tkn-pac-macos-amd64-0.23.1.tar.gz[Mac] +* link:https://mirror.openshift.com/pub/openshift-v4/clients/pipeline/0.23.1/tkn-pac-windows-amd64-0.23.1.zip[Windows] ++ +[NOTE] +==== +The binaries are compatible with `tkn` version `0.23.1`. +==== + +In addition, you can install `tkn-pac` using the following methods: + +[CAUTION] +==== +The `tkn-pac` CLI tool available using these methods is _not updated regularly_. +==== + +* Install on Linux or Mac OS using the `brew` package manager: ++ +[source,terminal] +---- +$ brew install openshift-pipelines/pipelines-as-code/tektoncd-pac +---- ++ +You can upgrade the package by running the following command: ++ +[source,terminal] +---- +$ brew upgrade openshift-pipelines/pipelines-as-code/tektoncd-pac +---- + +* Install as a container using `podman`: ++ +[source,terminal] +---- +$ podman run -e KUBECONFIG=/tmp/kube/config -v ${HOME}/.kube:/tmp/kube \ + -it quay.io/openshift-pipeline/pipelines-as-code tkn-pac help +---- ++ +You can also use `docker` as a substitute for `podman`. + +* Install from the GitHub repository using `go`: ++ +[source,terminal] +---- +$ go install github.com/openshift-pipelines/pipelines-as-code/cmd/tkn-pac +---- \ No newline at end of file diff --git a/modules/op-installing-pipelines-as-code-on-an-openshift-cluster.adoc b/modules/op-installing-pipelines-as-code-on-an-openshift-cluster.adoc new file mode 100644 index 000000000000..00a0bcbd1d5a --- /dev/null +++ b/modules/op-installing-pipelines-as-code-on-an-openshift-cluster.adoc @@ -0,0 +1,76 @@ +// This module is included in the following assembly: +// +// *cicd/pipelines/using-pipelines-as-code.adoc + +:_content-type: PROCEDURE +[id="installing-pipelines-as-code-on-an-openshift-cluster_{context}"] += Installing {pac} on an {product-title} + +[role="_abstract"] +{pac} is installed by default when you install the {pipelines-title} Operator. If you are using {pipelines-shortname} 1.7 or later versions, skip the procedure for manual installation of {pac}. + +However, if you want to disable the default installation of {pac} with the {pipelines-title} Operator, set the value of the `enablePipelinesAsCode` field as `false` in the `TektonConfig` custom resource. + +[source,yaml] +---- +... +spec: + addon: + enablePipelinesAsCode: false +... +---- + +To install {pac} using the Operator, set the value of the `enablePipelinesAsCode` field to `true`. + +[discrete] +.Procedure + +. To _manually_ install {pac} on a {product-title} cluster instead of the default installation with the {pipelines-title} Operator, run the following command: ++ +[source,terminal] +---- +$ VERSION=0.5.4 +$ oc apply -f https://raw.githubusercontent.com/openshift-pipelines/pipelines-as-code/release-$VERSION/release-$VERSION.yaml +---- ++ +[NOTE] +==== +For the latest stable version, check the link:https://github.com/openshift-pipelines/pipelines-as-code/releases[release page]. In addition, check the {pipelines-title} release notes to ensure that the {pac} version is compatible with the {pipelines-title} version. +==== ++ +This command installs {pac} in the `pipelines-as-code` namespace and creates user roles and the route URL for the {pac} event listener. + +. Note the route URL for the {pac} controller created on the cluster: ++ +[source,terminal] +---- +$ echo https://$(oc get route -n pipelines-as-code el-pipelines-as-code-interceptor -o jsonpath='{.spec.host}') +---- ++ +This URL will be needed later when you configure the Git repository hosting service provider. + +. (Optional) To allow non-administrative users to create repository custom resource definitions (CRDs) in their respective namespaces, create a `RoleBinding` object with the name `openshift-pipeline-as-code-clusterrole` in the namespace. For example, to allow a user to create a repository CRD in the `user-ci` namespace, run the following command: ++ +[source,terminal] +---- +$ oc adm policy add-role-to-user openshift-pipeline-as-code-clusterrole user -n user-ci +---- ++ +Alternatively, apply the following YAML file using the `oc apply -f ` command: ++ +[source,yaml] +---- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: openshift-pipeline-as-code-clusterrole + namespace: user-ci +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: openshift-pipeline-as-code-clusterrole +subjects: +- apiGroup: rbac.authorization.k8s.io + kind: User + name: user +---- diff --git a/modules/op-pipelines-as-code-command-reference.adoc b/modules/op-pipelines-as-code-command-reference.adoc new file mode 100644 index 000000000000..772de5dca0ec --- /dev/null +++ b/modules/op-pipelines-as-code-command-reference.adoc @@ -0,0 +1,118 @@ +// This module is included in the following assembly: +// +// *cicd/pipelines/using-pipelines-as-code.adoc + +:_content-type: REFERENCE +[id="pipelines-as-code-command-reference_{context}"] += {pac} command reference + +[role="_abstract"] +The `tkn-pac` CLI tool offers the following capabilities: + +* Bootstrap {pac} installation and configuration. +* Create a new {pac} repository. +* List all {pac} repositories. +* Describe a {pac} repository and the associated runs. +* Generate a simple pipeline run to get started. +* Resolve a pipeline run as if it was executed by {pac}. + +[TIP] +==== +You can use the commands corresponding to the capabilities for testing and experimentation, so that you don't have to make changes to the Git repository containing the application source code. +==== + +== Basic syntax + +[source,terminal] +---- +$ tkn pac [command or options] [arguments] +---- + +== Global options + +[source,terminal] +---- +$ tkn pac --help +---- + +== Utility commands + +=== bootstrap + +.Bootstrapping {pac} installation and configuration +[options="header"] +|=== + +| Command | Description + +| `tkn pac bootstrap` | Installs and configures {pac} for Git repository hosting service providers, such as GitHub and GitHub Enterprise. + +| `tkn pac bootstrap --nightly` | Installs the nightly build of {pac}. + +| `tkn pac bootstrap --route-url ` | Overrides the OpenShift route URL. + +By default, `tkn pac bootstrap` detects the OpenShift route, which is automatically associated with the {pac} controller service. + +If you do not have an {product-title} cluster, it asks you for the public URL that points to the ingress endpoint. + +| `tkn pac bootstrap github-app` | Create a GitHub application and secrets in the `pipelines-as-code` namespace. + +|=== + +=== repository + +.Managing {pac} repositories +[options="header"] +|=== + +| Command | Description + +| `tkn pac repo create` | Creates a new {pac} repository and a namespace based on the pipeline run template. + +| `tkn pac repo list` | Lists all the {pac} repositories and displays the last status of the associated runs. + +| `tkn pac repo describe` | Describes a {pac} repository and the associated runs. + +|=== + +=== generate + +.Generating pipeline runs using Pipelines as Code +[options="header"] +|=== + +| Command | Description + +| `tkn pac generate` | Generates a simple pipeline run. + +When executed from the directory containing the source code, it automatically detects current Git information. + +In addition, it uses basic language detection capability and adds extra tasks depending on the language. + +For example, if it detects a `setup.py` file at the repository root, the link:https://hub.tekton.dev/tekton/task/pylint[pylint task] is automatically added to the generated pipeline run. + +|=== + +=== resolve + +.Resolving and executing pipeline runs using {pac} +[options="header"] +|=== + +| Command | Description + +| `tkn pac resolve` | Executes a pipeline run as if it is owned by the {pac} on service. + +| `tkn pac resolve -f .tekton/pull-request.yaml \| oc apply -f -` | Displays the status of a live pipeline run that uses the template in `.tekton/pull-request.yaml`. + +Combined with a Kubernetes installation running on your local machine, you can observe the pipeline run without generating a new commit. + +If you run the command from a source code repository, it attempts to detect the current Git information and automatically resolve parameters such as current revision or branch. + +| `tkn pac resolve -f .tekton/pr.yaml -p revision=main -p repo_name=` | Executes a pipeline run by overriding default parameter values derived from the Git repository. + +The `-f` option can also accept a directory path and apply the `tkn pac resolve` command on all `.yaml` or `.yml` files in that directory. You can also use the `-f` flag multiple times in the same command. + +You can override the default information gathered from the Git repository by specifying parameter values using the `-p` option. For example, you can use a Git branch as a revision and a different repository name. + +|=== \ No newline at end of file