Add ClusterServiceVersion bundling

[lmm]

Description

This PR adds plumbing to generate ClusterServiceVersion (CSV) bundles which we use to validate and publish the operator on the RH catalog.

Two new make targets are added:

• gen-csv creates a CSV for every operator version we want to publish;
• gen-bundle gathers all generated CSVs, adds them to the ones already committed, and creates a zip file

This PR also removes operator manifests from the deploy directory since they are not used for CSV generation. Instead, users should use the operator manifests from our docs.

Testing

• The CSV bundling was tested by generating them and uploading them for validation testing. I also locally installed these bundles in my OCP cluster to verify the operator metadata in the OpenShift Console.

For PR author

• Tests for change.
• If changing pkg/apis/, run make gen-files
• If changing versions, run make gen-versions

For PR reviewers

A note for code reviewers - all pull requests must have the following:

• Milestone set according to targeted release.
• Appropriate labels:
  • kind/bug if this is a bugfix.
  • kind/enhancement if this is a a new feature.
  • enterprise if this PR applies to Calico Enterprise only.

[tmjd]

Some high level thoughts/questions:

• Should we be committing these bundles to this repo? Did you consider any other options?
• I'm not sure if bundling the CRDs in this repo is right, I'm wondering if instead of that we should be pulling from the docs. The ones here are more for testing purposes than anything and the 'real' ones should be coming from the docs. Or maybe part of the process needs to be pulling the latest Calico CRDs from the docs.

[lmm]

Should we be committing these bundles to this repo? Did you consider any other options?

Pre-generating the CSV's as needed was an option that was considered but the trade off is that we'd need some other way to keep track of the operator versions published on RH. The other thing is that the many operators in the operator-sdk ecosystem already check-in their CSVs, and since we already follow a lot of the operator-sdk conventions already, the proposed process in this PR is consistent.

I'm not sure if bundling the CRDs in this repo is right, I'm wondering if instead of that we should be pulling from the docs. The ones here are more for testing purposes than anything and the 'real' ones should be coming from the docs. Or maybe part of the process needs to be pulling the latest Calico CRDs from the docs.

I had a brief chat with @caseydavenport about this and the gist of that was that maybe we want the manifests in deploy/ to be the "real" ones that the docs follow. I think that simplifies the process for CSV bundling but makes it slightly awkward if contributors submit changes to operator manifests in the calico repo. Maybe Casey had some more thoughts on this.

[caseydavenport]

Should we be committing these bundles to this repo? Did you consider any other options?

Yeah, this strongly weirds me out, but it seems to be the pattern that other RH operators are according to my discussion with @lmm.

The main difference I think, based on my conversation with @lmm was that the other operators have a single release stream (e.g., the only maintain the master branch, not additional release branches).

[tmjd]

I'm not sure if bundling the CRDs in this repo is right, I'm wondering if instead of that we should be pulling from the docs. The ones here are more for testing purposes than anything and the 'real' ones should be coming from the docs. Or maybe part of the process needs to be pulling the latest Calico CRDs from the docs.

I had a brief chat with @caseydavenport about this and the gist of that was that maybe we want the manifests in deploy/ to be the "real" ones that the docs follow. I think that simplifies the process for CSV bundling but makes it slightly awkward if contributors submit changes to operator manifests in the calico repo. Maybe Casey had some more thoughts on this.

I think I'd be worried about moving the CRDs into the operator repo without some automation around copying them to the docs and a better understanding of what the process around that would be. Mainly because a lot of integration testing uses the docs so if they get out of sync that would not be good.

I feel like it would be simple enough to download the CRDs as part of the CSV generation step, I guess the caveat with that would be that we should wait until the Calico release is done and the CRDs are 'officially' available.

@caseydavenport WDYT?

[lmm]

I feel like it would be simple enough to download the CRDs as part of the CSV generation step

Thanks @tmjd, I've added this with 9d8ca79 and 2d0fea3

[tmjd]

LGTM, once the tests finish (and pass of course)