Refactor bundle validation code to seperate format and contend valida…

…tion

1. Seperate bundle format and content validation into 2 seperate functions
2. Remove operator-sdk reference from registry docs. Using `opm` reference
instead.
3. Add bundle object validation to improve content validation

Signed-off-by: Vu Dinh <vdinh@redhat.com>

cmd/opm/alpha/bundle/validate.go

@@ -3,6 +3,7 @@ package bundle
 import (
 	"io/ioutil"
 	"os"
+	"path/filepath"
 
 	"github.com/operator-framework/operator-registry/pkg/lib/bundle"
 	log "github.com/sirupsen/logrus"
@@ -54,9 +55,14 @@ func validateFunc(cmd *cobra.Command, args []string) error {
 		return err
 	}
 
-	logger.Info("Unpacked image layers, validating bundle image contents")
+	logger.Info("Unpacked image layers, validating bundle image format & contents")
 
-	err = imageValidator.ValidateBundle(dir)
+	err = imageValidator.ValidateBundleFormat(dir)
+	if err != nil {
+		return err
+	}
+
+	err = imageValidator.ValidateBundleFormat(filepath.Join(dir, bundle.ManifestsDir))
 	if err != nil {
 		return err
 	}

docs/design/operator-bundle.md

@@ -91,40 +91,34 @@ $ tree
 
 ## Operator Bundle Commands
 
-Operator SDK CLI is available to generate Bundle annotations and Dockerfile based on provided operator manifests.
+`opm` (Operator Package Manager) is a CLI tool to generate bundle annotations, build bundle manifests image, validate bundle manifests image and other functionalities. Please note that the `generate`, `build` and `validate` features of `opm` CLI are currently in alpha and only meant for development use.
 
-### Operator SDK CLI
+### `opm` (Operator Package Manager)
 
-In order to use Operator SDK CLI, follow the operator-SDK installation instruction:
+In order to use `opm` CLI, follow the `opm` build instruction:
 
-1. Install the [Operator SDK CLI](https://github.com/operator-framework/operator-sdk/blob/master/doc/user/install-operator-sdk.md)
+1. Clone the operator registry repository:
 
-Now, a binary named `operator-sdk` is available in OLM's directory to use.
 ```bash
-$ ./operator-sdk
-An SDK for building operators with ease
-
-Usage:
-  operator-sdk [command]
+$ git clone https://github.com/operator-framework/operator-registry
+```
 
-Available Commands:
-    bundle      Operator bundle commands
+2. Build `opm` binary using this command:
 
-Flags:
-  -h, --help      help for operator-sdk
-      --verbose   Enable verbose logging
-
-Use "operator-sdk [command] --help" for more information about a command.
+```bash
+$ go build ./cmd/opm/
 ```
 
+Now, a binary named `opm` is now built in current directory and ready to be used.
+
 ### Generate Bundle Annotations and DockerFile
 *Notes:*
 * If there are `annotations.yaml` and `Dockerfile` existing in the directory, they will be overwritten.
 
-Using `operator-sdk` CLI, bundle annotations can be generated from provided operator manifests. The overall `bundle generate` command usage is:
+Using `opm` CLI, bundle annotations can be generated from provided operator manifests. The overall `bundle generate` command usage is:
 ```bash
 Usage:
-  operator-sdk bundle generate [flags]
+  opm alpha bundle generate [flags]
 
 Flags:
   -c, --channels string    The list of channels that bundle image belongs to
@@ -141,7 +135,7 @@ The `--directory/-d`, `--channels/-c`, `--package/-p` are required flags while `
 
 The command for `generate` task is:
 ```bash
-$ ./operator-sdk bundle generate --directory /test --package test-operator \
+$ ./opm alpha bundle generate --directory /test --package test-operator \
 --channels stable,beta --default stable
 ```
 
@@ -173,7 +167,7 @@ $ docker build -f /path/to/Dockerfile -t quay.io/test/test-operator:latest /path
 Operator bundle image can be built from provided operator manifests using `build` command (see *Notes* below). The overall `bundle build` command usage is:
 ```bash
 Usage:
-  operator-SDK bundle build [flags]
+  opm alpha bundle build [flags]
 
 Flags:
   -c, --channels string        The list of channels that bundle image belongs to
@@ -192,15 +186,15 @@ Flags:
 
 The command for `build` task is:
 ```bash
-$ ./operator-sdk bundle build --directory /test --tag quay.io/coreos/test-operator.v0.1.0:latest \
+$ ./opm alpha bundle build --directory /test --tag quay.io/coreos/test-operator.v0.1.0:latest \
 --package test-operator --channels stable,beta --default stable
 ```
 
 The `--directory` or `-d` specifies the directory where the operator manifests for a specific version are located. The `--tag` or `-t` specifies the image tag that you want the operator bundle image to have. By using `build` command, the `annotations.yaml` and `Dockerfile` are automatically generated in the background.
 
 The default image builder is `Docker`. However, ` Buildah` and `Podman` are also supported. An image builder can specified via `--image-builder` or `-b` optional tag in `build` command. For example:
 ```bash
-$ ./operator-sdk bundle build --directory /test/0.1.0/ --tag quay.io/coreos/test-operator.v0.1.0:latest \
+$ ./opm alpha bundle build --directory /test/0.1.0/ --tag quay.io/coreos/test-operator.v0.1.0:latest \
 --image-builder podman --package test-operator --channels stable,beta --default stable
 ```
 
@@ -215,7 +209,7 @@ The `--package` or `-p` is the name of package fo the operator such as `etcd` wh
 Operator bundle image can validate bundle image that is publicly available in an image registry using `validate` command (see *Notes* below). The overall `bundle validate` command usage is:
 ```bash
 Usage:
-  operator-SDK bundle validate [flags]
+  opm alpha bundle validate [flags]
 
 Flags:
   -t, --tag string             The name of the bundle image will be built
@@ -225,7 +219,7 @@ Flags:
 
 The command for `validate` task is:
 ```bash
-$ ./operator-sdk bundle build --tag quay.io/coreos/test-operator.v0.1.0:latest --image-builder docker
+$ ./opm alpha bundle build --tag quay.io/coreos/test-operator.v0.1.0:latest --image-builder docker
 ```
 
 The `validate` command will first extract the contents of the bundle image into a temporary directory after it pulls the image from its image registry. Then, it will validate the format of bundle image to ensure manifests and metadata are located in their appropriate directories (`/manifests/` for bundle manifests files such as CSV and `/metadata/` for metadata files such as `annotations.yaml`). Also, it will validate the information in `annotations.yaml` to confirm that metadata is matching the provided data. For example, the provided media type in annotations.yaml just matches the actual media type is provided in the bundle image.

pkg/lib/bundle/errors.go

@@ -6,27 +6,22 @@ import (
 )
 
 // ValidationError is an imlementation of the Error type
-// that defines a set of errors when validating the bundle
+// that defines a list of errors when validating the bundle
 type ValidationError struct {
-	AnnotationErrors []error
-	FormatErrors     []error
+	Errors []error
 }
 
 func (v ValidationError) Error() string {
 	var errs []string
-	for _, err := range v.AnnotationErrors {
-		errs = append(errs, err.Error())
-	}
-	for _, err := range v.FormatErrors {
+	for _, err := range v.Errors {
 		errs = append(errs, err.Error())
 	}
 	return fmt.Sprintf("Bundle validation errors: %s",
 		strings.Join(errs, ","))
 }
 
-func NewValidationError(annotationErrs, formatErrs []error) ValidationError {
+func NewValidationError(errs []error) ValidationError {
 	return ValidationError{
-		AnnotationErrors: annotationErrs,
-		FormatErrors:     formatErrs,
+		Errors: errs,
 	}
 }

pkg/lib/bundle/interfaces.go

@@ -14,7 +14,10 @@ type BundleImageValidator interface {
 	PullBundleImage(imageTag string, directory string) error
 	// Validate bundle takes a directory containing the contents of a bundle image
 	// and validates that the format is correct
-	ValidateBundle(directory string) error
+	ValidateBundleFormat(directory string) error
+	// Validate bundle takes a directory containing the contents of a bundle image
+	// and validates that the content is correct
+	ValidateBundleContent(directory string) error
 }
 
 // NewImageValidator is a constructor that returns an ImageValidator

pkg/lib/bundle/testdata/validate/invalid_manifests_bundle/invalid_crd/etcdbackups.etcd.database.coreos.com.crd.yaml

@@ -0,0 +1,13 @@
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: etcdbackups.etcd.database.coreos.com
+spec:
+  group: etcd.database.coreos.com
+  names:
+    kind: EtcdBackup
+    listKind: EtcdBackupList
+    plural: etcdbackups
+    singular: etcdbackup
+  scope: Namespaced
+  version: v1beta2

pkg/lib/bundle/testdata/validate/invalid_manifests_bundle/invalid_crd/etcdclusters.etcd.database.coreos.com.crd.yaml

@@ -0,0 +1,16 @@
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: etcdclusters.etcd.database.coreos.com
+spec:
+  group: etcd.database.coreos.com
+  names:
+    kind: EtcdCluster
+    listKind: EtcdClusterList
+    plural: etcdclusters
+    shortNames:
+    - etcdclus
+    - etcd
+    singular: etcdcluster
+  scope: Namespaced
+  version: v1beta2