OTA-559: pkg/manifest: Add 'overrides' handling

[wking]

Centralize this aspect of manifest filtering, so it can be shared between the cluster-version operator (where it currently lives) and oc (where we want manifest extraction to understand this sort of condition).

[openshift-ci-robot]

@wking: This pull request references OTA-559 which is a valid jira issue.

In response to this:

Centralize this aspect of manifest filtering, so it can be shared between the cluster-version operator (where it currently lives) and oc (where we want manifest extraction to understand this sort of condition).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[petr-muller]

/cc

[petr-muller]

LGTM with a nit, address if you want
/hold

[petr-muller]

nit: we could surface what is being overriden?

[wking]

4.13.0-ec.4 CI:

$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.13-e2e-aws-sdn-serial/1631695148700143616/artifacts/e2e-aws-sdn-serial/gather-extra/artifacts/pods/openshift-cluster-version_cluster-version-operator-6f5d958c6b-k8t5l_cluster-version-operator.log | grep excluding | head -n2
I0303 16:57:25.646863       1 payload.go:210] excluding 0000_10_config-operator_01_infrastructure-TechPreviewNoUpgrade.crd.yaml group=apiextensions.k8s.io kind=CustomResourceDefinition namespace= name=infrastructures.config.openshift.io: "Default" is required, and release.openshift.io/feature-set=TechPreviewNoUpgrade
I0303 16:57:25.676524       1 payload.go:210] excluding 0000_30_cluster-api_00_credentials-request.yaml group=cloudcredential.openshift.io kind=CredentialsRequest namespace=openshift-cloud-credential-operator name=openshift-cluster-api-aws: "Default" is required, and release.openshift.io/feature-set=TechPreviewNoUpgrade

And that excluding ... context gets wrapped around in:

cluster-verison-operator $ git --no-pager grep 'excluding ' pkg
pkg/payload/payload.go:                                 klog.V(2).Infof("excluding %s group=%s kind=%s namespace=%s name=%s: %v\n", manifest.OriginalFilename, manifest.GVK.Group, manifest.GVK.Kind, manifest.Obj.GetNamespace(), manifest.Obj.GetName(), err)

That could probably be cleaned up by leaning on (*Manifest).String(), but it's there. Do we have similar wrapping around each call site?

cluster-version-operator $ git --no-pager grep -A1 '[.]Include('
pkg/cvo/sync_worker.go:                 if err := task.Manifest.Include(nil, nil, nil, &capabilities, work.Overrides); err != nil {
pkg/cvo/sync_worker.go-                         klog.V(2).Infof("Skipping precreation of %s: %s", task, err)
--
pkg/cvo/sync_worker.go:                 if err := task.Manifest.Include(nil, nil, nil, &capabilities, work.Overrides); err != nil {
pkg/cvo/sync_worker.go-                         klog.V(2).Infof("Skipping %s: %s", task, err)
--
pkg/payload/payload.go:                         if err := manifest.Include(&excludeIdentifier, &requiredFeatureSet, &profile, onlyKnownCaps, nil); err != nil {
pkg/payload/payload.go-                                 klog.V(2).Infof("excluding %s group=%s kind=%s namespace=%s name=%s: %v\n", manifest.OriginalFilename, manifest.GVK.Group, manifest.GVK.Kind, manifest.Obj.GetNamespace(), manifest.Obj.GetName(), err)

So Include is covered. What about IncludeAllowUnknownCapabilities?

$ git --no-pager grep -A4 '[.]IncludeAllowUnknownCapabilities(' | grep -v vendor/
pkg/payload/payload.go:         updateManErr := updateManifest.IncludeAllowUnknownCapabilities(nil, nil, nil, &clusterCaps, nil, true)
pkg/payload/payload.go-
pkg/payload/payload.go-         // update manifest is enabled, no need to check
pkg/payload/payload.go-         if updateManErr == nil {
pkg/payload/payload.go-                 continue
--
pkg/payload/payload.go:                 if err := currentManifest.IncludeAllowUnknownCapabilities(nil, nil, nil, &clusterCaps, nil, true); err != nil {
pkg/payload/payload.go-                         continue
pkg/payload/payload.go-                 }
pkg/payload/payload.go-                 caps := capability.GetImplicitlyEnabledCapabilities(currentManifest.GetManifestCapabilities(),
pkg/payload/payload.go-                         updateManifest.GetManifestCapabilities(), capabilities)
--
$ git --no-pager grep '^func .*\|[.]IncludeAllowUnknownCapabilities(' pkg/payload/payload.go | grep -B1 IncludeAllowUnknownCapabilities
pkg/payload/payload.go:func GetImplicitlyEnabledCapabilities(updatePayloadManifests []manifest.Manifest, currentPayloadManifests []manifest.Manifest,
pkg/payload/payload.go:         updateManErr := updateManifest.IncludeAllowUnknownCapabilities(nil, nil, nil, &clusterCaps, nil, true)
pkg/payload/payload.go:                 if err := currentManifest.IncludeAllowUnknownCapabilities(nil, nil, nil, &clusterCaps, nil, true); err != nil {

So unless we wanted context logging in GetImplicitlyEnabledCapabilities, I think we're good. And if we do want that context logging in GetImplicitlyEnabledCapabilities, we can handle that on the CVO side without adding "here's the manifest I am" context on this side.

/hold cancel

[petr-muller]

grumbles about repos that do not respect gh reviews
/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: petr-muller, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/manifest/OWNERS [petr-muller,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@wking: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.