New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add AWS EBS CSI operator startup #59
Add AWS EBS CSI operator startup #59
Conversation
a0d3515
to
5c31422
Compare
/retest |
b8743fe
to
699f8ea
Compare
/retest |
7fd3516
to
e52266c
Compare
For new library-go.
It's needed by the new library-go
ee9b913
to
43faa51
Compare
return err | ||
} | ||
operatorInformers := opinformers.NewSharedInformerFactoryWithOptions(operatorClientSet, resync, | ||
opinformers.WithTweakListOptions(singleNameListOptions(operatorclient.GlobalConfigName)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's still one in csoclients.go:
$ ag singleNameListOptions
pkg/csoclients/csoclients.go
107:func singleNameListOptions(name string) func(opts *metav1.ListOptions) {
52b2f56
to
89473a5
Compare
for i := range c.controllers { | ||
if c.controllers[i].operatorConfig.Platform == platform && !c.controllers[i].running { | ||
go c.controllers[i].mgr.Start(ctx) | ||
c.controllers[i].running = true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't ever going to be false anymore, do we care?
When the ctx
above is cancelled, all controllers from c.controllers[i].mgr
will shutdown (which is what we want). However, running
will still be true, and nothing sets it back to false. If for some reason we reach line 93 above again, the CSI driver operator will never run again.
I don't know if this can ever happen, but I wish we could get rid of .running
and have the ControllerManager tell us if it's running or not (maybe we can).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we really need to solve this. The context gets cancelled at the operator shutdown and we won't ever want to statt the CSI driver operators again.
Fixed remaining issues and squashed a lot. |
The operator RBAC rules needs to include all rules collected from CSI driver operators and for OLM, which is complicated. Using cluster-admin for now to make the operator working, however, with goal to rework the CSI driver operator (and CSI driver) to use common roles in the future.
The configuration should encapsulate all that CSO needs to know about a CSI driver operator to be able to start it.
CSO has many clients and passing them around individually is cumbersome.
CSIDriverObserver checks Infracstructure CRD for the underlying cloud and starts corresponding CSIDriverOperatorController(s). CSIDriverOperatorController installs one CSI driver operator.
// This CSIDriverOperatorCRController installs and syncs CSI driver operator CR. It monitors the | ||
// CR status and merges all its conditions to the CSO CR. | ||
// It produces following Conditions: | ||
// <CSI driver name>CSIDriverOperatorControllerDegraded on error |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This controller actually produces 2 types of degraded conditions:
-
WithSyncDegradedOnError()
will produce ac.name + "Degraded"
condition -
Sync()
will produce ac.name + csiDriverControllerName + "Degraded"
condition
IMO ideally we should have only 1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The controller should produce two Degraded conditions because of WithSyncDegradedOnError
- it clears its condition when sync() returns nil. And we still want copied Degraded conditions from the CR.
I added an explicit comment about that + separated their prefixes
// <CSI driver name>CSIDriverOperatorDegraded on error
// <CSI driver name>CSIDriverOperatorCRDegraded - copied from *Degraded conditions from CR.
// <CSI driver name>CSIDriverOperatorCRAvailable - copied from *Available conditions from CR.
// <CSI driver name>CSIDriverOperatorCRProgressing - copied from *Progressing conditions from CR.
(as a separate commit, because this one is a bigger change)
default: | ||
recorder.Eventf(fmt.Sprintf("%sUpdated", gvk.Kind), "Updated %s:\n%s", resourcehelper.FormatResourceForCLIWithNamespace(obj), strings.Join(details, "\n")) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: these report*Event()
are generic enough to live in util.go
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved (and squashed)
|
||
var platform configv1.PlatformType | ||
if infrastructure.Status.PlatformStatus != nil { | ||
platform = infrastructure.Status.PlatformStatus.Type |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you intend to use csiDriverControllerManager.platform
instead?
I don't think you did, if so we can remove it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed csiDriverControllerManager.platform
Use underscore in manifest names
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bertinatto, jsafrane The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Three new controllers are introduced in this PR:
CSIDriverStarterController
: observesInfrastructure
and starts CSI driver operators that are registered for each platform. For each CSI driver operator it starts:CSIDriverOperatorDeploymentController: manages Deployment of the operator and fills
<CSI driver name>CSIDriverOperatorDeployment
conditions (e.g.AWSEBSCSIDriverOperatorDeploymentAvailable
/Progressing
/Degraded
).CSIDriverOperatorCRController
: manages CR of the operator. It syncs all its conditions back tostorage
CR.