Skip to content

Commit

Permalink
update strategies and docs
Browse files Browse the repository at this point in the history
  • Loading branch information
qu1queee committed May 31, 2021
1 parent fb0b994 commit 410baa6
Show file tree
Hide file tree
Showing 7 changed files with 174 additions and 76 deletions.
64 changes: 63 additions & 1 deletion docs/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ SPDX-License-Identifier: Apache-2.0
- [Configuring a Build](#configuring-a-build)
- [Defining the Source](#defining-the-source)
- [Defining the Strategy](#defining-the-strategy)
- [Defining Params](#defining-params)
- [Defining the Builder or Dockerfile](#defining-the-builder-or-dockerfile)
- [Defining the Output](#defining-the-output)
- [Runtime-Image](#Runtime-Image)
Expand All @@ -24,6 +25,7 @@ A `Build` resource allows the user to define:
- source
- sources
- strategy
- params
- builder
- dockerfile
- output
Expand All @@ -39,6 +41,7 @@ The controller watches for:
When the controller reconciles it:

- Validates if the referenced `StrategyRef` exists.
- Validates if the specified `params` exists on the referenced strategy parameters. It also validates if the `params` names collide with the Shipwright reserved names.
- Validates if the container `registry` output secret exists.
- Validates if the referenced `spec.source.url` endpoint exists.

Expand All @@ -56,6 +59,8 @@ In order to prevent users from triggering `BuildRuns` (_execution of a Build_) t
| SpecBuilderSecretRefNotFound | The secret used to authenticate to the container registry doesn't exist.|
| MultipleSecretRefNotFound | More than one secret is missing. At the moment, only three paths on a Build can specify a secret. |
| RuntimePathsCanNotBeEmpty | The Runtime feature is used, but the runtime path was not defined. This is mandatory. |
| RestrictedParametersInUse | One or many defined `params` are colliding with Shipwright reserved parameters. See [Defining Params](#defining-params) for more information. |
| UndefinedParameter | One or many defined `params` are not defined in the referenced strategy. Please ensure that the strategy defines them under it´s `spec.parameters` list. |
| RemoteRepositoryUnreachable | The defined `spec.source.url` was not found. This validation only take place for http/https protocols. |

## Configuring a Build
Expand All @@ -73,7 +78,7 @@ The `Build` definition supports the following fields:
- `spec.output.credentials.name`- Reference an existing secret to get access to the container registry.

- Optional:
- `spec.parameters` - Refers to a list of `name-value` that could be used to loosely type parameters in the `BuildStrategy`.
- `spec.params` - Refers to a list of `name-value` that could be used to loosely type `parameters` in the `BuildStrategy`.
- `spec.dockerfile` - Path to a Dockerfile to be used for building an image. (_Use this path for strategies that require a Dockerfile_)
- `spec.sources` - [Sources](#Sources) describes a slice of artifacts that will be imported into project context, before the actual build process starts.
- `spec.runtime` - Runtime-Image settings, to be used for a multi-stage build.
Expand Down Expand Up @@ -171,6 +176,63 @@ spec:
kind: ClusterBuildStrategy
```

### Defining Params

A `Build` resource can specify _params_, these allow users to modify the behaviour of the referenced `BuildStrategy` steps.

When using _params_, users should avoid:

- Defining a `spec.param` name that doesn´t match one of the `spec.parameters` defined in the `BuildStrategy`.
- Defining a `spec.param` name that collides with the Shipwright reserved parameters. These are _BUILDER_IMAGE_,_DOCKERFILE_,_CONTEXT_DIR_ and any name starting with _shp-_.

In general, _params_ are tighly bound to Strategy _parameters_, please make sure you understand the contents of your strategy of choice, before defining _params_ in the _Build_. `BuildRun` resources allow users to override `Build` _params_, see the related [docs](./buildrun.md#defining-params) for more information.

#### Example

The following `BuildStrategy` contains a single step ( _a-strategy-step_ ) with a command and arguments. The strategy defines a parameter( _sleep-time_ ) with a reasonable default, that is used in the step arguments, see _$(params.sleep-time)_.

```yaml
---
apiVersion: shipwright.io/v1alpha1
kind: BuildStrategy
metadata:
name: sleepy-strategy
spec:
parameters:
- name: sleep-time
description: "time in seconds for sleeping"
default: "1"
buildSteps:
- name: a-strategy-step
image: alpine:latest
command:
- sleep
args:
- $(params.sleep-time)
```

If users would like the above strategy to change its behaviour, e.g. _allow the step to trigger a sleep cmd longer than 1 second_, then users can modify the default behaviour, via their `Build` `spec.params` definition. For example:

```yaml
---
apiVersion: shipwright.io/v1alpha1
kind: Build
metadata:
name: a-build
spec:
source:
url: https://github.com/shipwright-io/sample-go
contextDir: docker-build/
params:
- name: sleep-time
value: "60"
strategy:
name: sleepy-strategy
kind: BuildStrategy
```

The above `Build` definition uses _sleep-time_ param, a well-defined _parameter_ under it´s referenced `BuildStrategy`. By doing this, the user signalize to the referenced sleepy-strategy, the usage of a different value for it´s _sleep-time_ parameter.

### Defining the Builder or Dockerfile

A `Build` resource can specify an image containing the tools to build the final image. Users can do this via the `spec.builder` or the `spec.dockerfile`. For example, the user choose the `Dockerfile` file under the source repository.
Expand Down
42 changes: 41 additions & 1 deletion docs/buildrun.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ SPDX-License-Identifier: Apache-2.0
- [BuildRun Controller](#buildrun-controller)
- [Configuring a BuildRun](#configuring-a-buildrun)
- [Defining the BuildRef](#defining-the-buildref)
- [Defining params](#defining-params)
- [Defining the ServiceAccount](#defining-the-serviceaccount)
- [BuildRun Status](#buildrun-status)
- [Understanding the state of a BuildRun](#understanding-the-state-of-a-BuildRun)
Expand All @@ -18,7 +19,7 @@ SPDX-License-Identifier: Apache-2.0

## Overview

The resource `BuildRun` (`buildruns.dev/v1alpha1`) is the build process of a `Build` resource definition which is executed in Kubernetes.
The resource `BuildRun` (`buildruns.shipwright.io/v1alpha1`) is the build process of a `Build` resource definition which is executed in Kubernetes.

A `BuildRun` resource allows the user to define:

Expand Down Expand Up @@ -55,6 +56,7 @@ The `BuildRun` definition supports the following fields:
- Optional:
- `spec.serviceAccount` - Refers to the SA to use when building the image. (_defaults to the `default` SA_)
- `spec.timeout` - Defines a custom timeout. The value needs to be parsable by [ParseDuration](https://golang.org/pkg/time/#ParseDuration), for example `5m`. The value overwrites the value that is defined in the `Build`.
- `spec.params` - Override any _params_ defined in the referenced `Build`, as long as their name matches.
- `spec.output.image` - Refers to a custom location where the generated image would be pushed. The value will overwrite the `output.image` value which is defined in `Build`. ( Note: other properties of the output, for example, the credentials cannot be specified in the buildRun spec. )
- `spec.output.credentials.name` - Reference an existing secret to get access to the container registry. This secret will be added to the service account along with the ones requested by the `Build`.

Expand All @@ -72,6 +74,44 @@ spec:
name: buildpack-nodejs-build-namespaced
```
### Defining Params
A `BuildRun` resource can override _params_ defined in it´s referenced `Build`, as long as the `Build` defines the same _params_ name.

For example, the following `BuildRun` overrides the value for _sleep-time_ param, that is defined in the _a-build_ `Build` resource.

```yaml
---
apiVersion: shipwright.io/v1alpha1
kind: BuildRun
metadata:
name: a-buildrun
spec:
buildRef:
name: a-build
params:
- name: sleep-time
value: "30"
---
apiVersion: shipwright.io/v1alpha1
kind: Build
metadata:
name: a-build
spec:
source:
url: https://github.com/shipwright-io/sample-go
contextDir: docker-build/
params:
- name: sleep-time
value: "60"
strategy:
name: sleepy-strategy
kind: BuildStrategy
```

See more about `params` usage in the related [Build](./build.md#defining-params) resource docs.

### Defining the ServiceAccount

A `BuildRun` resource can define a serviceaccount to use. Usually this SA will host all related secrets referenced on the `Build` resource, for example:
Expand Down
47 changes: 44 additions & 3 deletions docs/buildstrategies.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@ SPDX-License-Identifier: Apache-2.0
- [Source to Image](#source-to-image)
- [Installing Source to Image Strategy](#installing-source-to-image-strategy)
- [Build Steps](#build-steps)
- [Strategy parameters](#strategy-parameters)
- [System parameters](#system-parameters)
- [System parameters vs Strategy parameters comparison](#system-parameters-vs-strategy-parameters-comparison)
- [System results](#system-results)
- [Steps Resource Definition](#steps-resource-definition)
- [Strategies with different resources](#strategies-with-different-resources)
Expand Down Expand Up @@ -122,8 +124,6 @@ kubectl apply -f samples/buildstrategy/kaniko/buildstrategy_kaniko_cr.yaml

[BuildKit](https://github.com/moby/buildkit) is composed of the `buildctl` client and the `buildkitd` daemon. For the `buildkit` ClusterBuildStrategy, it runs on a [daemonless](https://github.com/moby/buildkit#daemonless) mode, where both client and ephemeral daemon run in a single container. In addition, it runs without privileges (_[rootless](https://github.com/moby/buildkit/blob/master/docs/rootless.md)_).

The `buildkit-insecure` ClusterBuildStrategy exists to support users pushing to an insecure container registry. We use this strategy at the moment only for testing purposes against a local in-cluster registry. In the future, this strategy will be removed in favor of a single one where users can parameterize the secure/insecure behaviour.

### Cache Exporters

By default, the `buildkit` ClusterBuildStrategy will use caching to optimize the build times. When pushing an image to a registry, it will use the `inline` export cache, which pushes the image and cache together. Please refer to [export-cache docs](https://github.com/moby/buildkit#export-cache) for more information.
Expand Down Expand Up @@ -202,16 +202,57 @@ kubectl apply -f samples/buildstrategy/source-to-image/buildstrategy_source-to-i
[s2i]: https://github.com/openshift/source-to-image
[buildah]: https://github.com/containers/buildah

## Strategy parameters

Strategy parameters allow users to parameterize their strategy definition, by allowing users to control the _parameters_ values via the `Build` or `BuildRun` resources.

Users defining _parameters_ under their strategies require to understand the following:

- **Definition**: A list of parameters should be defined under `spec.parameters`. Each list item should consist of a _name_, a _description_ and a reasonable _default_ value (_type string_).
- **Usage**: In order to use a parameter in the strategy steps, users should follow the following syntax: `$(params.your-parameter-name)`
- **Parameterize**: Any `Build` or `BuildRun` referencing your strategy, can set a value for _your-parameter-name_ parameter if needed.

The following is an example of a strategy that defines and uses the `sleep-time` parameter:

```yaml
---
apiVersion: shipwright.io/v1alpha1
kind: BuildStrategy
metadata:
name: sleepy-strategy
spec:
parameters:
- name: sleep-time
description: "time in seconds for sleeping"
default: "1"
buildSteps:
- name: a-strategy-step
image: alpine:latest
command:
- sleep
args:
- $(params.sleep-time)
```
See more information on how to use this parameter in a `Build` or `BuildRun` in the related [docs](./build.md#defining-params).

## System parameters

You can use parameters when defining the steps of a build strategy to access system information as well as information provided by the user in his Build or BuildRun. The following parameters are available:
Contrary to the strategy `spec.parameters`, you can use system parameters and their values defined at runtime when defining the steps of a build strategy to access system information as well as information provided by the user in their Build or BuildRun. The following parameters are available:

| Parameter | Description |
| ------------------------------ | ----------- |
| `$(params.shp-source-root)` | The absolute path to the directory that contains the user's sources. |
| `$(params.shp-source-context)` | The absolute path to the context directory of the user's sources. If the user specified no value for `spec.source.contextDir` in his Build, then this value will equal the value for `$(params.shp-source-root)`. Note that this directory is not guaranteed to exist at the time the container for your step is started, you can therefore not use this parameter as a step's working directory. |
| `$(params.shp-output-image)` | The URL of the image that the user wants to push as specified in the Build's `spec.output.image`, or the override from the BuildRun's `spec.output.image`. |

## System parameters vs Strategy Parameters Comparison

| Parameter Type | User Configurable | Definition |
| ------------------ | ------------ | ------------- |
| System Parameter | No | At run-time, by the `BuildRun` controller. |
| Strategy Parameter | Yes | At build-time, during the `BuildStrategy` creation. |

## System results

You can optionally store the size and digest of the image your build strategy created to some files. This information will eventually be made available in the status of the BuildRun.
Expand Down
32 changes: 16 additions & 16 deletions docs/proposals/parameterize-strategies.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ approvers:
- "@ImJasonH"
creation-date: 2021-03-25
last-updated: 2021-04-20
status: implementable
status: implemented
---

# Parameterize Build Strategies
Expand Down Expand Up @@ -58,13 +58,13 @@ There are several reasons for the need of a more well define parameterization me

### Goals

- Introduce a `spec.params` field across Build and BuildRun API´s, as a way for users to provide N parameters according to their preferred strategy of choice. Defining `spec.params` in a BuildRun overrides any definition of the same parameter in Build´s.
- Introduce a `spec.params` field across Build and BuildRun API´s, as a way for users to provide N parameters according to their preferred strategy of choice. Defining `spec.params` in a BuildRun overrides any definition of the same param in Build´s.

- Introduce a `spec.params` on Strategies, both cluster or namespaced scope. This allow strategy administrators to layout the definition of N parameters, as required for their strategies.
- Introduce a `spec.parameters` on Strategies, both cluster or namespaced scope. This allow strategy administrators to layout the definition of N parameters, as required for their strategies.

- Revisit what we consider today as default Parameters that are defined on runtime when creating `TaskRuns`. These are `DOCKERFILE`, `CONTEXT_DIR` and `BUILDER_IMAGE`. Decide if we need to reduce or extend the default definitions and properly document their usage.

- Expose the usage of Tekton Params directly on the strategies, in favor of simplicity and readability. This boils down to `inputs.params.DOCKERFILE`, instead of `build.dockerfile`.
- Expose the usage of Tekton Params directly on the strategies, in favor of simplicity and readability. This boils down to `params.DOCKERFILE`, instead of `build.dockerfile`.

### Non-Goals

Expand All @@ -76,7 +76,7 @@ There are several reasons for the need of a more well define parameterization me

### Part 1: Introduce spec.params

Introduce the `spec.params` field across Build, BuildRun and BuildStrategies resources. This will define a list of parameters definition, of the type `string`. This new `spec.params` does not provide support for Tekton params of the type `array`. This field can only be use, if the parameter is supported in the strategy of choice.
Introduce the `spec.params` field across Build and BuildRun, and the `spec.parameters` for BuildStrategies resources. This will define a list of parameters definition, of the type `string`. This new `spec.params` does not provide support for Tekton params of the type `array`. This field can only be use, if the parameter is supported in the strategy of choice.

For example:

Expand All @@ -87,10 +87,10 @@ metadata:
name: a-cluster-strategy
spec:
buildSteps: #Content omitted for this example
params:
parameters:
- name: a-param
description: A description of this parameter definition.
default: The default parameter value.
default: "The default parameter value"
```
```yaml
Expand Down Expand Up @@ -121,11 +121,11 @@ spec:
value: Another parameter value because my build is not up-to-date.
```
_Note_: If a **Buildrun** specifies `a-param` via its `spec.params` ,this will override the value defined in the `a-build`. In other words, BuildRuns have a higher priority when defining parameters.
_Note_: If a **Buildrun** specifies `a-param` via its `spec.params` ,this will override the value defined in the `a-build`. In other words, BuildRuns have a higher priority when defining params.

### Part 2: Runtime Parameters

As mentioned in the Goals section, we currently define three parameters that can be constructed on runtime, also know as _runtime parameters_. Runtime parameters are define during runtime, on the creation of a Taskrun, where the paremeter definition and the parameter value mapping takes place. These runtime parameters are:
As mentioned in the Goals section, we currently define three parameters that can be constructed on runtime, also know as _runtime parameters_ or _system parameters_. System parameters are define during runtime, on the creation of a Taskrun, where the paremeter definition and the parameter value mapping takes place. These runtime parameters are:

- DOCKERFILE
- CONTEXT_DIR
Expand Down Expand Up @@ -166,22 +166,22 @@ Instead of doing:
we should do:

```yaml
--context=/workspace/source/$(PARAMS.CONTEXT_DIR)
--context=/workspace/source/$(params.CONTEXT_DIR)
```

### Part 4: Sanity Checks

We will require a sanity check mechanism, in order to validate the quality of the defined user parameters. This belongs to implementation details, but generic examples are:
We will require a sanity check mechanism, in order to validate the quality of the defined user params. This belongs to implementation details, but generic examples are:

- Decide how to handle parameters that have none default and that are not specified at the Build/BuildRun level.
- Validate that the defined parameter in the strategy is not a reserved runtime parameter.
- Validate if the specified user parameter was defined in the strategy.
- Validate if the specified user params was defined in the strategy.

### Part 5: Documentation Enhancement

Currently we do not have proper documentation on:

- Tutorilas on how to Build Strategies.
- Tutorials on how to build Strategies.
- Which are the runtime parameters and how they are used.

This ensures that as part of this EP implementation, we can provide a set of documents to fulfill the above missing points.
Expand All @@ -203,7 +203,7 @@ kind: ClusterBuildStrategy
metadata:
name: buildkit
spec:
params:
parameters:
- name: DOCKERFILE_NAME
description: Name of your Dockerfile inside the DOCKERFILE context
default: "Dockerfile"
Expand Down Expand Up @@ -261,7 +261,7 @@ kind: ClusterBuildStrategy
metadata:
name: buildkit
spec:
params:
parameters:
- name: DOCKERFILE_NAME
description: Name of your Dockerfile inside the DOCKERFILE context
default: "Dockerfile"
Expand All @@ -285,7 +285,7 @@ kind: ClusterBuildStrategy
metadata:
name: buildkit
spec:
params:
parameters:
- name: INSECURE_REGISTRY
description: Defines if an image should be pushed to an insecure registry
default: false
Expand Down
Loading

0 comments on commit 410baa6

Please sign in to comment.