Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add guide to run Piped on CloudRun #2231

Merged
merged 5 commits into from Jul 13, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
@@ -0,0 +1,9 @@
---
title: "Installation"
linkTitle: "Installation"
weight: 1
description: >
This page describes how to install a Piped.
---

Since Piped is a stateless agent, no database or storage is required to run. In addition, a Piped can interact with one or multiple cloud providers, so the number of Piped and where they should run is entirely up to your preference. For example, you can run your Pipeds in a Kubernetes cluster to deploy not just Kubernetes applications but your Terraform and CloudRun applications as well.
@@ -0,0 +1,121 @@
---
title: "Installing on CloudRun"
linkTitle: "Installing on CloudRun"
weight: 2
description: >
This page describes how to install Piped on CloudRun.
---

## Prerequisites

##### Having piped's ID and Key strings
- Ensure that the `piped` has been registered and you are having its PIPED_ID and PIPED_KEY strings.
- If you are not having them, this [page](/docs/operator-manual/control-plane/registering-a-piped/) guides you how to register a new one.

##### Preparing SSH key
- If your Git repositories are private, `piped` requires a private SSH key to access those repositories.
- Please checkout [this documentation](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository's Settings page.)

## Installation

- Preparing a piped configuration file as the following:

``` yaml
apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
projectID: {PROJECT_ID}
pipedID: {PIPED_ID}
pipedKeyData: {BASE64_ENCODED_PIPED_KEY}
webAddress: {CONTROL_PLANE_WEB_ADDRESS}
# Write in a format like "host:443" because the communication is done via gRPC.
apiAddress: {CONTROL_PLANE_API_ADDRESS}

git:
sshKeyData: {BASE64_ENCODED_PRIVATE_SSH_KEY}

repositories:
- repoId: {REPO_ID_OR_NAME}
remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
branch: {GIT_BRANCH}

# Optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Optional comments like these makes it clear what to configure at least.

# Enable this Piped to handle CLOUD_RUN application.
cloudProviders:
- name: cloudrun-in-project
type: CLOUDRUN
config:
project: {GCP_PROJECT_ID}
region: {GCP_PROJECT_REGION}

# Optional
# Uncomment this if you want to enable this Piped to handle Terraform application.
# - name: terraform-gcp
# type: TERRAFORM

# Optional
# Uncomment this if you want to enable SecretManagement feature.
# https://pipecd.dev/docs/user-guide/secret-management
# secretManagement:
# type: KEY_PAIR
# config:
# privateKeyData: {BASE64_ENCODED_PRIVATE_KEY}
# publicKeyData: {BASE64_ENCODED_PUBLIC_KEY}
```

- Creating a new secret in [SecretManager](https://cloud.google.com/secret-manager/docs/creating-and-accessing-secrets) to store above configuration data securely

``` console
gcloud secrets create cloudrun-piped-config --data-file={PATH_TO_CONFIG_FILE}
```

- Running Piped in CloudRun

khanhtc1202 marked this conversation as resolved.
Show resolved Hide resolved
Prepare a CloudRun service manifest file as below.

Note: Fields which set to '1' are strict to be set with that value to ensure piped work correctly.
nghialv marked this conversation as resolved.
Show resolved Hide resolved

``` yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: piped
spec:
template:
metadata:
annotations:
autoscaling.knative.dev/maxScale: '1' # This must be 1.
autoscaling.knative.dev/minScale: '1' # This must be 1.
run.googleapis.com/ingress: internal
run.googleapis.com/ingress-status: internal
spec:
containerConcurrency: 1 # This must be 1.
containers:
- image: gcr.io/pipecd/piped:v0.11.1
args:
- piped
- --metrics=true
- --config-file=/etc/piped-config/config.yaml
ports:
- containerPort: 9085
volumeMounts:
- mountPath: /etc/piped-config
name: piped-config
resources:
limits:
cpu: 1000m
memory: 512Mi
volumes:
- name: piped-config
secret:
secretName: cloudrun-piped-config
items:
- path: config.yaml
key: latest
```

nghialv marked this conversation as resolved.
Show resolved Hide resolved
``` console
gcloud beta run services replace cloudrun-piped-service.yaml
```

Note: Make sure that the created secret is accessible from this Piped service. See more [here](https://cloud.google.com/run/docs/configuring/secrets#access-secret).
@@ -1,9 +1,9 @@
---
title: "Installation"
linkTitle: "Installation"
title: "Installing on Kubernetes cluster"
linkTitle: "Installing on Kubernetes cluster"
weight: 1
description: >
This page describes how to install a piped.
This page describes how to install Piped on Kubernetes cluster.
---

## Prerequisites
Expand All @@ -14,10 +14,9 @@ description: >

##### Preparing SSH key
- If your Git repositories are private, `piped` requires a private SSH key to access those repositories.
- Please checkout [this documentation](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for how to generate a new SSH key. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository's Settings page.)
- Please checkout [this documentation](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository's Settings page.)

## Installation on Kubernetes cluster
### In the cluster-wide mode
## In the cluster-wide mode
This way requires installing cluster-level resources. Piped installed with this way can perform deployment workloads against any other namespaces than the where Piped runs on.

- Adding `pipecd` helm chart repository
Expand All @@ -32,18 +31,18 @@ This way requires installing cluster-level resources. Piped installed with this
apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
projectID: {YOUR_PROJECT_ID}
pipedID: {YOUR_PIPED_ID}
projectID: {PROJECT_ID}
pipedID: {PIPED_ID}
pipedKeyFile: /etc/piped-secret/piped-key
webAddress: {CONTROL_PLANE_WEB_ADDRESS}
# Write in a format like "host:443" because the communication is done via gRPC.
apiAddress: {YOUR_CONTROL_PLANE_ADDRESS}
webAddress: http://{YOUR_CONTROL_PLANE_ADDRESS}
apiAddress: {CONTROL_PLANE_API_ADDRESS}
git:
sshKeyFile: /etc/piped-secret/ssh-key
repositories:
- repoId: {REPO_ID_OR_NAME}
remote: git@github.com:{YOUR_GIT_ORG}/{YOUR_GIT_REPO}.git
branch: master
remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
branch: {GIT_BRANCH}
syncInterval: 1m
```

Expand All @@ -62,7 +61,7 @@ Note: Be sure to set `--set args.insecure=true` if your control-plane has not TL

See [values.yaml](https://github.com/pipe-cd/manifests/blob/master/manifests/piped/values.yaml) for the full values.

### In the namespaced mode
## In the namespaced mode
The previous way requires installing cluster-level resources. If you want to restrict Piped's permission within the namespace where Piped runs on, this way is for you.
Most parts are identical to the previous way, but some are slightly different.

Expand All @@ -72,26 +71,26 @@ Most parts are identical to the previous way, but some are slightly different.
apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
projectID: {YOUR_PROJECT_ID}
pipedID: {YOUR_PIPED_ID}
projectID: {PROJECT_ID}
pipedID: {PIPED_ID}
pipedKeyFile: /etc/piped-secret/piped-key
webAddress: {CONTROL_PLANE_WEB_ADDRESS}
# Write in a format like "host:443" because the communication is done via gRPC.
apiAddress: {YOUR_CONTROL_PLANE_ADDRESS}
webAddress: http://{YOUR_CONTROL_PLANE_ADDRESS}
apiAddress: {CONTROL_PLANE_API_ADDRESS}
git:
sshKeyFile: /etc/piped-secret/ssh-key
repositories:
- repoId: REPO_ID_OR_NAME
remote: git@github.com:{YOUR_GIT_ORG}/{YOUR_GIT_REPO}.git
branch: master
remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
branch: {GIT_BRANCH}
syncInterval: 1m
# This is needed to restrict to limit the access range to within a namespace.
cloudProviders:
- name: my-kubernetes
type: KUBERNETES
config:
appStateInformer:
namespace: {YOUR_NAMESPACE}
namespace: {NAMESPACE}
```

- Then installing it with the following options:
Expand Down Expand Up @@ -122,42 +121,8 @@ Therefore if you are running on OpenShift with a version before 4.2, please use
--set rbac.scope=namespace
--set args.addLoginUserToPasswd=true \
--set securityContext.runAsNonRoot=true \
--set securityContext.runAsUser={YOUR_UID} \
--set securityContext.fsGroup={YOUR_FS_GROUP} \
--set securityContext.runAsUser={UID} \
--set securityContext.fsGroup={FS_GROUP} \
--set securityContext.runAsGroup=0 \
--set image.repository="gcr.io/pipecd/piped-okd"
```

## Installing on single machine

- Downloading the latest `piped` binary for your machine

https://github.com/pipe-cd/pipe/releases

- Preparing a piped configuration file as the following:

``` yaml
apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
projectID: {YOUR_PROJECT_ID}
pipedID: {YOUR_PIPED_ID}
pipedKeyFile: {PATH_TO_PIPED_KEY_FILE}
# Write in a format like "host:443" because the communication is done via gRPC.
apiAddress: {YOUR_CONTROL_PLANE_ADDRESS}
webAddress: http://{YOUR_CONTROL_PLANE_ADDRESS}
git:
sshKeyFile: {PATH_TO_SSH_KEY_FILE}
repositories:
- repoId: {REPO_ID_OR_NAME}
remote: git@github.com:{YOUR_GIT_ORG}/{YOUR_GIT_REPO}.git
branch: {YOUR_GIT_BRANCH}
syncInterval: 1m
```

- Start running the `piped`

``` console
./piped piped --config-file={PATH_TO_PIPED_CONFIG_FILE}
```

@@ -0,0 +1,51 @@
---
title: "Installing on a single machine"
linkTitle: "Installing on a single machine"
weight: 3
description: >
This page describes how to install a Piped on a single machine.
---

## Prerequisites

##### Having piped's ID and Key strings
- Ensure that the `piped` has been registered and you are having its PIPED_ID and PIPED_KEY strings.
- If you are not having them, this [page](/docs/operator-manual/control-plane/registering-a-piped/) guides you how to register a new one.

##### Preparing SSH key
- If your Git repositories are private, `piped` requires a private SSH key to access those repositories.
- Please checkout [this documentation](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository's Settings page.)

## Installation

- Downloading the latest `piped` binary for your machine

https://github.com/pipe-cd/pipe/releases

- Preparing a piped configuration file as the following:

``` yaml
apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
projectID: {PROJECT_ID}
pipedID: {PIPED_ID}
pipedKeyFile: {PATH_TO_PIPED_KEY_FILE}
webAddress: {CONTROL_PLANE_WEB_ADDRESS}
# Write in a format like "host:443" because the communication is done via gRPC.
apiAddress: {CONTROL_PLANE_API_ADDRESS}
git:
sshKeyFile: {PATH_TO_SSH_KEY_FILE}
repositories:
- repoId: {REPO_ID_OR_NAME}
remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
branch: {GIT_BRANCH}
syncInterval: 1m
```

- Start running the `piped`

``` console
./piped piped --config-file={PATH_TO_PIPED_CONFIG_FILE}
```