Skip to content

Commit

Permalink
pkg/start: Do not .Run() when we fail to initialize from payload
Browse files Browse the repository at this point in the history
7480a5f (Get cluster version object earlier in startup, 2022-02-14,
leaderelection.RunOrDie.  Before that pivot, failing the load would
quickly shut down the process.  Since that pivot, until this commit,
failures such as [1]:

  $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/746/pull-ci-openshift-cluster-version-operator-master-e2e-agnostic-ovn/1777506846186147840/artifacts/e2e-agnostic-ovn/ipi-install-install/artifacts/log-bundle-20240409022023.tar | tar -xOz log-bundle-20240409022023/bootstrap/containers/cluster-version-operator-d757dec43012ea3097967701b53f4c31cdf7255212882a43327491402da189d4.log | grep -A1 'Failed to initialize from payload'
  I0409 02:13:08.579037       1 start.go:250] Failed to initialize from payload; shutting down: the local release contents are invalid - no current version can be determined from disk: Error loading manifests from /: preprocess 0000_90_cluster-version-operator_02_servicemonitor.yaml: failed to parse manifest: template: manifest:99: undefined variable "$labels"
  I0409 02:13:08.579062       1 start.go:254] Launching CVO.Run gorountine

would cause InitializeFromPayload to error out before setting up
configSync with NewSyncWorkerWithPreconditions, but still call
Operator.Run, which would panic trying to call configSync.StatusCh()
to invoke runThrottledStatusNotifier.

With this commit, I'm returning to failing fast, and no longer calling
Operator.Run or later when we have trouble loading the payload.

[1]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/746/pull-ci-openshift-cluster-version-operator-master-e2e-agnostic-ovn/1777506846186147840
  • Loading branch information
wking committed Apr 9, 2024
1 parent 2396053 commit 20ec957
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions pkg/start/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,7 @@ func (o *Options) run(ctx context.Context, controllerCtx *Context, lock resource
}
klog.Infof("Failed to initialize from payload; shutting down: %v", err)
resultChannel <- asyncResult{name: "payload initialization", error: firstError}
return
}

klog.Infof("Launching CVO.Run gorountine")
Expand Down

0 comments on commit 20ec957

Please sign in to comment.