csi: add hooks in both Node and Controller Services #968

bertinatto · 2020-12-09T14:03:03Z

CC @openshift/storage

bertinatto · 2020-12-09T14:03:21Z

/hold

This is a WIP design. This code is being tested on openshift/aws-ebs-csi-driver-operator#104.

jsafrane · 2020-12-10T14:28:45Z

Should there be a similar hook for DaemonSet?

bertinatto · 2020-12-10T19:31:13Z

Should there be a similar hook for DaemonSet?

Added, but no operator is using it ATM

bertinatto · 2020-12-10T19:32:25Z

/hold cancel

I've tested this with openshift/aws-ebs-csi-driver-operator#104 and it works.

bertinatto · 2020-12-10T19:32:45Z

@deads2k could you take a look, please?

jsafrane · 2020-12-14T09:27:45Z

...perator/csi/csidrivercontrollerservicecontroller/csi_driver_controller_service_controller.go

@@ -89,14 +92,16 @@ func NewCSIDriverControllerServiceController(
 	deployInformer appsinformersv1.DeploymentInformer,
 	optionalConfigInformer configinformers.SharedInformerFactory,
 	recorder events.Recorder,
+	optionalDeploymentHooks ...func(*appsv1.Deployment) error,


Would it make sense to either give owner of the controller a way how to poke the controller for resync (i.e. the owner has informer on secrets / config maps / whatnot and is able to enqueue "fake" event into CSIDriverControllerServiceController' queue) or pass informers to watch into the NewCSIDriverControllerServiceController?

I can see there is ResyncEvery(time.Minute) below, so this PR will work even without it.

Would it make sense to address this in another PR?

Also, could you clarify a use case for poking the controller for resync?

bertinatto · 2021-01-05T20:30:15Z

/assign @deads2k

jsafrane · 2021-01-05T20:49:26Z

/lgtm

deads2k · 2021-01-06T14:54:04Z

...or/csi/csidrivercontrollerservicecontroller/csi_driver_controller_service_controller_test.go

@@ -422,33 +469,6 @@ func TestSync(t *testing.T) {
 					withFalseConditions(conditionProgressing)),
 			},
 		},
-		{
-			// Deployment is fully deployed and its status is synced to CR


this rubs me the wrong way. Why did it need to be removed?

It was a duplicate of the test case right above, probably copied by mistake.

deads2k · 2021-01-06T14:55:42Z

pkg/operator/csi/csidrivernodeservicecontroller/csi_driver_node_service_controller.go

@@ -63,6 +64,8 @@ type CSIDriverNodeServiceController struct {
 	operatorClient v1helpers.OperatorClient
 	kubeClient     kubernetes.Interface
 	dsInformer     appsinformersv1.DaemonSetInformer
+	// Optional hook functions to modify the DaemonSet.
+	optionalDaemonSetHooks []func(*appsv1.DaemonSet) error


typed functions please. Also, make clear what happens on errors and where responsibility for identifying which function is failing lies in the error.

deads2k · 2021-01-06T14:56:01Z

pkg/operator/csi/csidrivernodeservicecontroller/csi_driver_node_service_controller.go

+	for _, fn := range c.optionalDaemonSetHooks {
+		err := fn(required)
+		if err != nil {
+			return fmt.Errorf("error running hook function: %w", err)


you can at least give me an ordinal (i)

deads2k · 2021-01-06T14:56:54Z

I'm curious what you're trying to replace/mutate. Some sorts of mutations are easier while you still have bytes (this is why we have the replacePlaceholders).

bertinatto

I'm curious what you're trying to replace/mutate. Some sorts of mutations are easier while you still have bytes (this is why we have the replacePlaceholders).

As more CSI operators use these control loops, we want to give them freedom to mutate the deployment/daemonset without adding placeholders to library-go, specially if the mutation is specific to that operator, like this one: openshift/aws-ebs-csi-driver-operator#102

Most operators won't need this, and the default mutation approach should be the string replacements, but I figured for some situations mutating the object is better.

Does that make sense?

bertinatto · 2021-01-06T15:13:41Z

...or/csi/csidrivercontrollerservicecontroller/csi_driver_controller_service_controller_test.go

@@ -422,33 +469,6 @@ func TestSync(t *testing.T) {
 					withFalseConditions(conditionProgressing)),
 			},
 		},
-		{
-			// Deployment is fully deployed and its status is synced to CR


It was a duplicate of the test case right above, probably copied by mistake.

bertinatto · 2021-01-06T18:24:45Z

pkg/operator/csi/csidrivernodeservicecontroller/csi_driver_node_service_controller.go

+	for _, fn := range c.optionalDaemonSetHooks {
+		err := fn(required)
+		if err != nil {
+			return fmt.Errorf("error running hook function: %w", err)


bertinatto · 2021-01-06T18:24:52Z

pkg/operator/csi/csidrivernodeservicecontroller/csi_driver_node_service_controller.go

@@ -63,6 +64,8 @@ type CSIDriverNodeServiceController struct {
 	operatorClient v1helpers.OperatorClient
 	kubeClient     kubernetes.Interface
 	dsInformer     appsinformersv1.DaemonSetInformer
+	// Optional hook functions to modify the DaemonSet.
+	optionalDaemonSetHooks []func(*appsv1.DaemonSet) error


deads2k · 2021-01-06T21:29:14Z

Does that make sense?

I understand the desire. I'd be somewhat interested in what you're combining because I've regretted it in the past. But I'm ok adding the option

deads2k · 2021-01-06T21:30:43Z

/lgtm

openshift-ci-robot · 2021-01-06T21:31:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bertinatto, deads2k, jsafrane

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [deads2k]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 9, 2020

openshift-ci-robot requested review from deads2k and sttts December 9, 2020 14:03

bertinatto mentioned this pull request Dec 9, 2020

Bug 1898045: redeploy CSI Controller Deployment when secret changes openshift/aws-ebs-csi-driver-operator#104

Merged

bertinatto force-pushed the add-hook branch from 05b2876 to dce8ec3 Compare December 9, 2020 18:11

bertinatto force-pushed the add-hook branch 3 times, most recently from ef266f6 to e7e6303 Compare December 10, 2020 19:24

bertinatto changed the title ~~csi: add controller deployment hook~~ csi: add hooks in both Node and Controller Services Dec 10, 2020

bertinatto force-pushed the add-hook branch from e7e6303 to a495f91 Compare December 10, 2020 19:31

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 10, 2020

bertinatto force-pushed the add-hook branch 3 times, most recently from 6ea769d to ecd6418 Compare December 11, 2020 12:30

jsafrane reviewed Dec 14, 2020

View reviewed changes

openshift-ci-robot assigned deads2k Jan 5, 2021

openshift-ci-robot assigned jsafrane Jan 5, 2021

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 5, 2021

deads2k reviewed Jan 6, 2021

View reviewed changes

csi: add Controller Service (Deployment) hooks

ffb186b

csi: add Node Service (DaemonSet) hooks

a423d6f

bertinatto commented Jan 6, 2021

View reviewed changes

bertinatto force-pushed the add-hook branch from ecd6418 to a423d6f Compare January 6, 2021 19:34

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jan 6, 2021

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 6, 2021

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 6, 2021

openshift-merge-robot merged commit c4d0b9c into openshift:master Jan 6, 2021

bertinatto mentioned this pull request Jan 7, 2021

Bug 1910459: redeploy CSI Controller Deployment when secret changes openshift/gcp-pd-csi-driver-operator#9

Merged

bertinatto mentioned this pull request Jan 28, 2021

Bug 1915467: cherry-pick hooks helpers and config observer #986

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

csi: add hooks in both Node and Controller Services #968

csi: add hooks in both Node and Controller Services #968

bertinatto commented Dec 9, 2020

bertinatto commented Dec 9, 2020 •

edited

jsafrane commented Dec 10, 2020

bertinatto commented Dec 10, 2020

bertinatto commented Dec 10, 2020

bertinatto commented Dec 10, 2020

jsafrane Dec 14, 2020

bertinatto Jan 5, 2021

jsafrane Jan 5, 2021

bertinatto commented Jan 5, 2021

jsafrane commented Jan 5, 2021

deads2k Jan 6, 2021

bertinatto Jan 6, 2021

deads2k Jan 6, 2021

bertinatto Jan 6, 2021

deads2k Jan 6, 2021

bertinatto Jan 6, 2021

deads2k commented Jan 6, 2021

bertinatto left a comment •

edited

bertinatto Jan 6, 2021

bertinatto Jan 6, 2021

bertinatto Jan 6, 2021

deads2k commented Jan 6, 2021

deads2k commented Jan 6, 2021

openshift-ci-robot commented Jan 6, 2021

csi: add hooks in both Node and Controller Services #968

csi: add hooks in both Node and Controller Services #968

Conversation

bertinatto commented Dec 9, 2020

bertinatto commented Dec 9, 2020 • edited

jsafrane commented Dec 10, 2020

bertinatto commented Dec 10, 2020

bertinatto commented Dec 10, 2020

bertinatto commented Dec 10, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bertinatto commented Jan 5, 2021

jsafrane commented Jan 5, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deads2k commented Jan 6, 2021

bertinatto left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deads2k commented Jan 6, 2021

deads2k commented Jan 6, 2021

openshift-ci-robot commented Jan 6, 2021

bertinatto commented Dec 9, 2020 •

edited

bertinatto left a comment •

edited