Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add bundle contents validation #146

Merged
merged 3 commits into from Jan 21, 2020
Merged

Add bundle contents validation #146

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7673f8e
Add validate bundle contents
exdx Jan 5, 2020
1f920b9
Refactor bundle validation code to seperate format and contend valida…
dinhxuanvu Jan 15, 2020
0d31556
Address feedbacks from comments/reviews
dinhxuanvu Jan 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Expand Up @@ -449,6 +449,7 @@ bin

# Ignore sqlite
*.db
*.db-journal

# Ignore vscode
.vscode
Expand Down
10 changes: 8 additions & 2 deletions cmd/opm/alpha/bundle/validate.go
View file
Open in desktop
Expand Up @@ -3,6 +3,7 @@ package bundle
import (
"io/ioutil"
"os"
"path/filepath"

"github.com/operator-framework/operator-registry/pkg/lib/bundle"
log "github.com/sirupsen/logrus"
Expand Down Expand Up @@ -54,9 +55,14 @@ func validateFunc(cmd *cobra.Command, args []string) error {
return err
}

logger.Info("Unpacked image layers, validating bundle image contents")
logger.Info("Unpacked image layers, validating bundle image format & contents")

err = imageValidator.ValidateBundle(dir)
err = imageValidator.ValidateBundleFormat(dir)
if err != nil {
return err
}

err = imageValidator.ValidateBundleContent(filepath.Join(dir, bundle.ManifestsDir))
if err != nil {
return err
}
Expand Down
65 changes: 42 additions & 23 deletions docs/design/operator-bundle.md
View file
Open in desktop
Expand Up @@ -91,40 +91,34 @@ $ tree

## Operator Bundle Commands

Operator SDK CLI is available to generate Bundle annotations and Dockerfile based on provided operator manifests.
`opm` (Operator Package Manager) is a CLI tool to generate bundle annotations, build bundle manifests image, validate bundle manifests image and other functionalities. Please note that the `generate`, `build` and `validate` features of `opm` CLI are currently in alpha and only meant for development use.

### Operator SDK CLI
### `opm` (Operator Package Manager)

In order to use Operator SDK CLI, follow the operator-SDK installation instruction:
In order to use `opm` CLI, follow the `opm` build instruction:

1. Install the [Operator SDK CLI](https://github.com/operator-framework/operator-sdk/blob/master/doc/user/install-operator-sdk.md)
1. Clone the operator registry repository:

Now, a binary named `operator-sdk` is available in OLM's directory to use.
```bash
$ ./operator-sdk
An SDK for building operators with ease

Usage:
operator-sdk [command]

Available Commands:
bundle Operator bundle commands
$ git clone https://github.com/operator-framework/operator-registry
```

Flags:
-h, --help help for operator-sdk
--verbose Enable verbose logging
2. Build `opm` binary using this command:

Use "operator-sdk [command] --help" for more information about a command.
```bash
$ go build ./cmd/opm/
```

Now, a binary named `opm` is now built in current directory and ready to be used.

### Generate Bundle Annotations and DockerFile
*Notes:*
* If there are `annotations.yaml` and `Dockerfile` existing in the directory, they will be overwritten.

Using `operator-sdk` CLI, bundle annotations can be generated from provided operator manifests. The overall `bundle generate` command usage is:
Using `opm` CLI, bundle annotations can be generated from provided operator manifests. The overall `bundle generate` command usage is:
```bash
Usage:
operator-sdk bundle generate [flags]
opm alpha bundle generate [flags]

Flags:
-c, --channels string The list of channels that bundle image belongs to
Expand All @@ -141,7 +135,7 @@ The `--directory/-d`, `--channels/-c`, `--package/-p` are required flags while `

The command for `generate` task is:
```bash
$ ./operator-sdk bundle generate --directory /test --package test-operator \
$ ./opm alpha bundle generate --directory /test --package test-operator \
--channels stable,beta --default stable
```

Expand Down Expand Up @@ -173,7 +167,7 @@ $ docker build -f /path/to/Dockerfile -t quay.io/test/test-operator:latest /path
Operator bundle image can be built from provided operator manifests using `build` command (see *Notes* below). The overall `bundle build` command usage is:
```bash
Usage:
operator-SDK bundle build [flags]
opm alpha bundle build [flags]

Flags:
-c, --channels string The list of channels that bundle image belongs to
Expand All @@ -192,15 +186,15 @@ Flags:

The command for `build` task is:
```bash
$ ./operator-sdk bundle build --directory /test --tag quay.io/coreos/test-operator.v0.1.0:latest \
$ ./opm alpha bundle build --directory /test --tag quay.io/coreos/test-operator.v0.1.0:latest \
--package test-operator --channels stable,beta --default stable
```

The `--directory` or `-d` specifies the directory where the operator manifests for a specific version are located. The `--tag` or `-t` specifies the image tag that you want the operator bundle image to have. By using `build` command, the `annotations.yaml` and `Dockerfile` are automatically generated in the background.

The default image builder is `Docker`. However, ` Buildah` and `Podman` are also supported. An image builder can specified via `--image-builder` or `-b` optional tag in `build` command. For example:
```bash
$ ./operator-sdk bundle build --directory /test/0.1.0/ --tag quay.io/coreos/test-operator.v0.1.0:latest \
$ ./opm alpha bundle build --directory /test/0.1.0/ --tag quay.io/coreos/test-operator.v0.1.0:latest \
--image-builder podman --package test-operator --channels stable,beta --default stable
```

Expand All @@ -209,3 +203,28 @@ The `--package` or `-p` is the name of package fo the operator such as `etcd` wh
*Notes:*
* If there is `Dockerfile` existing in the directory, it will be overwritten.
* If there is an existing `annotations.yaml` in `/metadata` directory, the cli will attempt to validate it and returns any found errors. If the ``annotations.yaml`` is valid, it will be used as a part of build process. The optional boolean `--overwrite/-o` flag can be enabled (false by default) to allow cli to overwrite the `annotations.yaml` if existed.

### Validate Bundle Image

Operator bundle image can validate bundle image that is publicly available in an image registry using `validate` command (see *Notes* below). The overall `bundle validate` command usage is:
```bash
Usage:
opm alpha bundle validate [flags]

Flags:
-t, --tag string The name of the bundle image will be built
-b, --image-builder string Tool to extract container images. One of: [docker, podman] (default "docker")
-h, --help help for build
```

The command for `validate` task is:
```bash
$ ./opm alpha bundle build --tag quay.io/coreos/test-operator.v0.1.0:latest --image-builder docker
```

The `validate` command will first extract the contents of the bundle image into a temporary directory after it pulls the image from its image registry. Then, it will validate the format of bundle image to ensure manifests and metadata are located in their appropriate directories (`/manifests/` for bundle manifests files such as CSV and `/metadata/` for metadata files such as `annotations.yaml`). Also, it will validate the information in `annotations.yaml` to confirm that metadata is matching the provided data. For example, the provided media type in annotations.yaml just matches the actual media type is provided in the bundle image.

After the bundle image format is confirmed, the command will validate the bundle contents such as manifests and metadata files if the bundle format is `RegistryV1` or "Plain" type. "RegistryV1" format means it contains `ClusterResourceVersion` and its associated Kubernetes objects while `PlainType` means it contains all Kubernetes objects. The content validation process will ensure the individual file in the bundle image is valid and can be applied to an OLM-enabled cluster provided all necessary permissions and configurations are met.

*Notes:*
* The bundle content validation is best effort which means it will not guarantee 100% accuracy due to nature of Kubernetes objects may need certain permissions and configurations, which users may not have, in order to be applied successfully in a cluster.
44 changes: 34 additions & 10 deletions go.mod
View file
Open in desktop
@@ -1,19 +1,22 @@
module github.com/operator-framework/operator-registry

go 1.13

require (
github.com/antihax/optional v0.0.0-20180407024304-ca021399b1a6
github.com/docker/distribution v2.7.1+incompatible
github.com/ghodss/yaml v1.0.0
github.com/golang-migrate/migrate/v4 v4.6.2
github.com/golang/mock v1.2.0
github.com/golang/mock v1.3.1
github.com/golang/protobuf v1.3.2
github.com/grpc-ecosystem/grpc-health-probe v0.2.1-0.20181220223928-2bf0a5b182db
github.com/mattn/go-sqlite3 v1.10.0
github.com/maxbrunsfeld/counterfeiter/v6 v6.2.2
github.com/onsi/ginkgo v1.10.1
github.com/onsi/gomega v1.7.0
github.com/operator-framework/api v0.0.0-20200120235816-80fd2f1a09c9
github.com/operator-framework/operator-lifecycle-manager v0.0.0-20191115003340-16619cd27fa5
github.com/otiai10/copy v1.0.1
github.com/otiai10/curr v0.0.0-20190513014714-f5a3d24e5776 // indirect
github.com/pkg/errors v0.8.1
github.com/sirupsen/logrus v1.4.2
github.com/spf13/cobra v0.0.5
Expand All @@ -23,14 +26,35 @@ require (
google.golang.org/grpc v1.24.0
gopkg.in/yaml.v2 v2.2.4
helm.sh/helm/v3 v3.0.1
k8s.io/api v0.0.0-20191016110408-35e52d86657a
k8s.io/apiextensions-apiserver v0.0.0-20191016113550-5357c4baaf65
k8s.io/apimachinery v0.0.0-20191004115801-a2eda9f80ab8
k8s.io/client-go v0.0.0-20191016111102-bec269661e48
k8s.io/api v0.0.0
k8s.io/apiextensions-apiserver v0.0.0
k8s.io/apimachinery v0.0.0
k8s.io/client-go v8.0.0+incompatible
k8s.io/klog v1.0.0
k8s.io/kubectl v0.0.0-20191016120415-2ed914427d51
k8s.io/kubectl v0.0.0
)

replace github.com/docker/docker => github.com/moby/moby v0.7.3-0.20190826074503-38ab9da00309 // Required by Helm

go 1.13
replace (
github.com/docker/docker => github.com/moby/moby v0.7.3-0.20190826074503-38ab9da00309 // Required by Helm
k8s.io/api => k8s.io/api v0.0.0-20190918155943-95b840bb6a1f
k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20190918161926-8f644eb6e783
k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20190913080033-27d36303b655
k8s.io/apiserver => k8s.io/apiserver v0.0.0-20191016112112-5190913f932d
k8s.io/cli-runtime => k8s.io/cli-runtime v0.0.0-20191016114015-74ad18325ed5
k8s.io/client-go => k8s.io/client-go v0.0.0-20190918160344-1fbdaa4c8d90
k8s.io/cloud-provider => k8s.io/cloud-provider v0.0.0-20191016115326-20453efc2458
k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.0.0-20191016115129-c07a134afb42
k8s.io/code-generator => k8s.io/code-generator v0.0.0-20191004115455-8e001e5d1894
k8s.io/component-base => k8s.io/component-base v0.0.0-20191016111319-039242c015a9
k8s.io/cri-api => k8s.io/cri-api v0.0.0-20190828162817-608eb1dad4ac
k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20191016115521-756ffa5af0bd
k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.0.0-20191016112429-9587704a8ad4
k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.0.0-20191016114939-2b2b218dc1df
k8s.io/kube-proxy => k8s.io/kube-proxy v0.0.0-20191016114407-2e83b6f20229
k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.0.0-20191016114748-65049c67a58b
k8s.io/kubectl => k8s.io/kubectl v0.0.0-20191016120415-2ed914427d51
k8s.io/kubelet => k8s.io/kubelet v0.0.0-20191016114556-7841ed97f1b2
k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.0.0-20191016115753-cf0698c3a16b
k8s.io/metrics => k8s.io/metrics v0.0.0-20191016113814-3b1a734dba6e
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.0.0-20191016112829-06bb3c9d77c9
)