Skip to content
This repository has been archived by the owner on Mar 24, 2022. It is now read-only.

Commit

Permalink
Refactor location of sample tasks and pipelines for pks
Browse files Browse the repository at this point in the history
  • Loading branch information
Luciano Silva committed Mar 2, 2018
1 parent 33b1302 commit ada3dd4
Show file tree
Hide file tree
Showing 15 changed files with 881 additions and 4 deletions.
4 changes: 2 additions & 2 deletions pipelines/pcf/pks/configure-pks-cluster/pipeline.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
plan:
- get: pcf-pipelines-utils
- task: create-pks-cluster
file: pcf-pipelines-utils/tasks/pcf/create-pks-cluster/task.yml
file: pcf-pipelines-utils/tasks/pcf/pks/create-pks-cluster/task.yml
params:
PCF_PKS_API: ((pcf_pks_api))
PKS_CLI_USERNAME: ((pks_cli_username))
Expand All @@ -32,7 +32,7 @@ jobs:
plan:
- get: pcf-pipelines-utils
- task: delete-pks-cluster
file: pcf-pipelines-utils/tasks/pcf/delete-pks-cluster/task.yml
file: pcf-pipelines-utils/tasks/pcf/pks/delete-pks-cluster/task.yml
params:
PCF_PKS_API: ((pcf_pks_api))
PKS_CLI_USERNAME: ((pks_cli_username))
Expand Down
104 changes: 104 additions & 0 deletions pipelines/pcf/pks/install-pks/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
<img src="https://pivotal.gallerycdn.vsassets.io/extensions/pivotal/vscode-concourse/0.1.3/1517353139519/Microsoft.VisualStudio.Services.Icons.Default" alt="Concourse" height="70"/>&nbsp;&nbsp;<img src="https://dtb5pzswcit1e.cloudfront.net/assets/images/product_logos/icon_pivotalcontainerservice@2x.png" alt="PCF Knowledge Depot" height="70"/>

# Install PKS pipeline

This pipeline installs the PKS tile on top of an existing PCF Ops Manager deployment.

<img src="https://raw.githubusercontent.com/lsilvapvt/misc-support-files/master/docs/images/install-pks-tile.png" alt="Concourse" width="100%"/>


The parameters file of this pipeline implements the concept of "externalized tile parameters", where all the available tile configuration options are fed to the pipeline tasks as a YAML object containing the parameter names expected by Ops Manager for the tile.

For example:
```
properties: |
######## Configuration for Plan 1
.properties.plan1_selector:
value: "Plan Active"
.properties.plan1_selector.active.name:
value: "Small plan" # the name that appears for end users to choose
```

This approach allows for the `configure-tile` task of this pipeline to be generic and *tile-agnostic*, by delegating the tile configuration options to the content of the main three parameters `networks`, `properties` and `resources`.

---

## How to use this pipeline

1) Update [`pks_params.yml`](pks_params.yml) by following the instructions in the file.
The order of tile parameters in that file follows the same order as parameters are presented in Ops Manager and in the tile documentation ([vSphere](https://docs.pivotal.io/runtimes/pks/1-0/installing-pks-vsphere.html) or [GCP](https://docs.pivotal.io/runtimes/pks/1-0/installing-pks-gcp.html)).

If you use `Vault` or `CredHub` for credentials management, you can use the provided script [`pks_vault_params.sh`](pks_vault_params.sh) to automatically create the pipeline secrets in those systems.

Also, note that the pipeline can automatically generate certificates for the PKS API. See more details in comments for parameter `.pivotal-container-service.pks_tls` in [`pks_params.yml`](pks_params.yml).


2) Update [`pcf_params.yml`](pcf_params.yml) by following the instructions in the file.

This parameter file contains information about the PCF foundation (e.g. Ops Manager and Director) to which the tile will be deployed to.

This parameters file is separate from the others for reuse purposes, since any other PCF tile install or upgrade pipeline will use the same properties. If you already have this type of file created for another PCF tile pipeline, you can reuse it here. See [`Appendix A`](#appendix-a---pcf-pipelines-parameter-files-tiers) section below for a sample diagram of this parameters files structure pattern.

3) Update [`global_params.yml`](global_params.yml) by following the instructions in the file.

This parameter file contains information about global properties that typically apply to any PCF pipeline (e.g. Pivotal Network token).

This parameters file is separate from the others for reuse purposes, since any other PCF tile install or upgrade pipeline will use the same properties. If you already have this type of file created for another PCF tile pipeline, you can reuse it here. See [`Appendix A`](#appendix-a---pcf-pipelines-parameter-files-tiers) section below for a sample diagram of this parameters files structure pattern.

4) Create the pipeline in Concourse:

`fly -t <target> set-pipeline -p install-pks -c pipeline.yml -l global_params.yml -l pcf_params.yml -l pks_params.yml`

5) Un-pause and run pipeline `install-pks`


---


## Post PKS tile deploy steps

### PKS CLI client ID creation

Once the PKS tile is successfully deployed, a PKS CLI client ID is required to be created ([see documentation](https://docs.pivotal.io/runtimes/pks/1-0/manage-users.html#uaa-scopes)).

For that step, the pipeline also provides a job to automate it: `create-pks-cli-user`. Simply manually run that pipeline job to get the PKS CLI client ID created.

*Note:* in order for that task to work, the configured PKS API endpoint needs to be reachable from a DNS/network standpoint (see docs for [vSphere](https://docs.pivotal.io/runtimes/pks/1-0/installing-pks-vsphere.html#loadbalancer-pks-api) and [GCP](https://docs.pivotal.io/runtimes/pks/1-0/installing-pks-gcp.html#loadbalancer-pks-api])).

### Using PKS

Once the PKS CLI client ID created, proceed with [creating K8s clusters with PKS](https://docs.pivotal.io/runtimes/pks/1-0/create-cluster.html) and [deploying K8s workloads with `kubectl`](https://docs.pivotal.io/runtimes/pks/1-0/deploy-workloads.html).


---

## Appendix A - PCF pipelines parameter files tiers

```
┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
│ │ │ │
Pipelines for Foundation 1 Pipelines for Foundation 2
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
│ │ │ │ │ │ │ │ │ │ │ │
│ │ ERT tile │ │Redis tile │ │MySQL tile │ │ │ │ ERT tile │ │Redis tile │ │MySQL tile │ │ 1 params file per tile pipeline
│ params │ │ params │ │ params │ │ params │ │ params │ │ params │
│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
└───────────┘ └───────────┘ └───────────┘ └───────────┘ └───────────┘ └───────────┘
│ ┌───────────────────────────────────────┐ │ │ ┌───────────────────────────────────────┐ │
│ │ │ │
│ │ PCF foundation 1 params │ │ │ │ PCF foundation 2 params │ │ 1 params file per foundation
│ e.g. OpsMgr info │ │ e.g. OpsMgr info │
│ │ │ │ │ │ │ │
│ │ │ │
│ └───────────────────────────────────────┘ │ │ └───────────────────────────────────────┘ │
┌───────────────────────────────────────────────────────────────────────────────────────┐
│ │ │ │ │ │
│ Global parameters │ 1 params file for all
│ │ e.g. PivNet token │ │ │ │ foundations
│ │
│ │ │ │ │ │
└───────────────────────────────────────────────────────────────────────────────────────┘
│ │ │ │
─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
```
5 changes: 5 additions & 0 deletions pipelines/pcf/pks/install-pks/global_params.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
######## Generic global parameters
######## typically apply to all foundations and all tiles

# Pivotal Network token for tile release download and pcf-pipelines download
pivnet_token: ((pivnet_token))
15 changes: 15 additions & 0 deletions pipelines/pcf/pks/install-pks/pcf_params.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
######## PCF Foundation-specific parameters

# Ops Manager information and admin credentials
opsman_domain_or_ip_address: ((opsman_domain_or_ip_address))
opsman_admin_username: ((opsman_admin_username)) # Username for Ops Manager admin account
opsman_admin_password: ((opsman_admin_password)) # Password for Ops Manager admin account
# Either opsman_client_id/opsman_client_secret or opsman_admin_username/opsman_admin_password needs to be specified.
# If you are using opsman_admin_username/opsman_admin_password, edit opsman_client_id/opsman_client_secret to be an empty value.
# If you are using opsman_client_id/opsman_client_secret, edit opsman_admin_username/opsman_admin_password to be an empty value.
opsman_client_id: ((opsman_client_id))
opsman_client_secret: ((opsman_client_secret))

# pcf-pipelines legacy parameters - only used for when no_proxy option is used
company_proxy_domain:
om_ip:
220 changes: 220 additions & 0 deletions pipelines/pcf/pks/install-pks/pipeline.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
groups: []
resources:
- name: pcf-pipelines-tarball
type: pivnet
source:
api_token: {{pivnet_token}}
product_slug: pcf-automation
product_version: v0.23.0
- name: pcf-pipelines-utils
type: git
source:
uri: https://github.com/pivotalservices/concourse-pipeline-samples.git
- name: product-release
type: pivnet
source:
api_token: {{pivnet_token}}
product_slug: pivotal-container-service
product_version: {{product_version}}
sort_by: semver


resource_types:
- name: pivnet
type: docker-image
source:
repository: pivotalcf/pivnet-resource
tag: latest-final

jobs:
- name: upload-product
plan:
- aggregate:
- do:
- get: pcf-pipelines-tarball
- task: unpack-tarball
config:
platform: linux
image_resource:
type: docker-image
source:
repository: czero/rootfs
run:
path: bash
args:
- -c
- tar -xvf pcf-pipelines-tarball/*.tgz
inputs:
- name: pcf-pipelines-tarball
path: ""
outputs:
- name: pcf-pipelines
path: ""
- get: pivnet-product
resource: product-release
params:
globs:
- "*.pivotal"
- task: upload-tile
file: pcf-pipelines/tasks/upload-product-and-stemcell/task.yml
params:
IAAS: vsphere
NO_PROXY: {{company_proxy_domain}}
OM_IP: {{om_ip}}
OPS_MGR_PWD: {{opsman_admin_password}}
OPS_MGR_USR: {{opsman_admin_username}}
OPSMAN_CLIENT_ID: {{opsman_client_id}}
OPSMAN_CLIENT_SECRET: {{opsman_client_secret}}
OPSMAN_DOMAIN_OR_IP_ADDRESS: {{opsman_domain_or_ip_address}}
PIVNET_API_TOKEN: {{pivnet_token}}
- name: stage-product
plan:
- aggregate:
- do:
- get: pcf-pipelines-tarball
- task: unpack-tarball
config:
platform: linux
image_resource:
type: docker-image
source:
repository: czero/rootfs
run:
path: bash
args:
- -c
- tar -xvf pcf-pipelines-tarball/*.tgz
inputs:
- name: pcf-pipelines-tarball
path: ""
outputs:
- name: pcf-pipelines
path: ""
- get: pivnet-product
passed:
- upload-product
trigger: true
resource: product-release
params:
globs: ["*.pivotal"]

- task: stage
file: pcf-pipelines/tasks/stage-product/task.yml
params:
OPSMAN_CLIENT_ID: {{opsman_client_id}}
OPSMAN_CLIENT_SECRET: {{opsman_client_secret}}
OPSMAN_DOMAIN_OR_IP_ADDRESS: {{opsman_domain_or_ip_address}}
OPSMAN_PASSWORD: {{opsman_admin_password}}
OPSMAN_USERNAME: {{opsman_admin_username}}

- name: configure-product
plan:
- aggregate:
- do:
- get: pcf-pipelines-tarball
- task: unpack-tarball
config:
platform: linux
image_resource:
type: docker-image
source:
repository: czero/rootfs
run:
path: bash
args:
- -c
- tar -xvf pcf-pipelines-tarball/*.tgz
inputs:
- name: pcf-pipelines-tarball
path: ""
outputs:
- name: pcf-pipelines
path: ""
- get: pivnet-product
passed:
- stage-product
resource: product-release
trigger: true
params:
globs: []
- get: pcf-pipelines-utils

- task: configure-tile
file: pcf-pipelines-utils/tasks/pcf/configure-tile/task.yml
params:
OPSMAN_DOMAIN_OR_IP_ADDRESS: {{opsman_domain_or_ip_address}}
OPSMAN_USERNAME: {{opsman_admin_username}}
OPSMAN_PASSWORD: {{opsman_admin_password}}
OPSMAN_CLIENT_ID: {{opsman_client_id}}
OPSMAN_CLIENT_SECRET: {{opsman_client_secret}}
TILE_PROPERTIES: {{properties}}
TILE_NETWORK: {{networks}}
TILE_RESOURCES: {{resources}}
TILE_PRODUCT_NAME: {{product_name}}

- task: disable-errands
file: pcf-pipelines/tasks/disable-errands/task.yml
params:
ERRANDS_TO_DISABLE: {{errands_to_disable}}
OPSMAN_CLIENT_ID: {{opsman_client_id}}
OPSMAN_CLIENT_SECRET: {{opsman_client_secret}}
OPSMAN_DOMAIN_OR_IP_ADDRESS: {{opsman_domain_or_ip_address}}
OPSMAN_PASSWORD: {{opsman_admin_password}}
OPSMAN_USERNAME: {{opsman_admin_username}}
PRODUCT_NAME: {{product_name}}

- name: deploy-product
plan:
- aggregate:
- do:
- get: pcf-pipelines-tarball
- task: unpack-tarball
config:
platform: linux
image_resource:
type: docker-image
source:
repository: czero/rootfs
run:
path: bash
args:
- -c
- tar -xvf pcf-pipelines-tarball/*.tgz
inputs:
- name: pcf-pipelines-tarball
path: ""
outputs:
- name: pcf-pipelines
path: ""
- get: pivnet-product
passed:
- configure-product
resource: product-release
trigger: true
params:
globs: []
- task: apply-changes
file: pcf-pipelines/tasks/apply-changes/task.yml
params:
OPSMAN_CLIENT_ID: {{opsman_client_id}}
OPSMAN_CLIENT_SECRET: {{opsman_client_secret}}
OPSMAN_DOMAIN_OR_IP_ADDRESS: {{opsman_domain_or_ip_address}}
OPSMAN_PASSWORD: {{opsman_admin_password}}
OPSMAN_USERNAME: {{opsman_admin_username}}

- name: create-pks-cli-user
plan:
- aggregate:
- get: pcf-pipelines-utils
- task: configure-pks-cli-user
file: pcf-pipelines-utils/tasks/pcf/pks/configure-pks-cli-user/task.yml
params:
OPSMAN_DOMAIN_OR_IP_ADDRESS: {{opsman_domain_or_ip_address}}
OPSMAN_USERNAME: {{opsman_admin_username}}
OPSMAN_PASSWORD: {{opsman_admin_password}}
OPSMAN_CLIENT_ID: {{opsman_client_id}}
OPSMAN_CLIENT_SECRET: {{opsman_client_secret}}
PKS_DOMAIN: {{pcf_pks_domain}}
PKS_CLI_USERNAME: {{pks_cli_username}}
PKS_CLI_USEREMAIL: {{pks_cli_useremail}}
PKS_CLI_PASSWORD: {{pks_cli_password}}
Loading

0 comments on commit ada3dd4

Please sign in to comment.