Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The command line option to disable leader election does not function as intended #201

Closed
courtneypacheco opened this issue Aug 16, 2021 · 6 comments
Labels
lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.

Comments

@courtneypacheco
Copy link
Contributor

In the main.go file in the main directory, a command line option exists for disabling leader election, and the default value indicates that leader election should be turned off. However, leader election is still enabled anyways:

# oc logs pod/nfd-controller-manager-7cd8d54686-nllqc manager
I0816 17:01:22.272019       1 main.go:55] Operator Version: 783214b0-dirty
I0816 17:01:23.326238       1 request.go:655] Throttling request took 1.021121017s, request: GET:https://172.30.0.1:443/apis/security.openshift.io/v1?timeout=32s
2021-08-16T17:01:25.093Z	INFO	controller-runtime.metrics	metrics server is starting to listen	{"addr": "127.0.0.1:8080"}
2021-08-16T17:01:25.094Z	INFO	setup	starting manager
I0816 17:01:25.094661       1 leaderelection.go:243] attempting to acquire leader lease openshift-nfd/39f5e5c3.nodefeaturediscoveries.nfd.kubernetes.io...
2021-08-16T17:01:25.095Z	INFO	controller-runtime.manager	starting metrics server	{"path": "/metrics"}
I0816 17:01:25.121966       1 leaderelection.go:253] successfully acquired lease openshift-nfd/39f5e5c3.nodefeaturediscoveries.nfd.kubernetes.io
2021-08-16T17:01:25.122Z	INFO	controller-runtime.manager.controller.nodefeaturediscovery	Starting EventSource	{"reconciler group": "nfd.openshift.io", "reconciler kind": "NodeFeatureDiscovery", "source": "kind source: /, Kind="}
2021-08-16T17:01:25.122Z	DEBUG	controller-runtime.manager.events	Normal	{"object": {"kind":"ConfigMap","namespace":"openshift-nfd","name":"39f5e5c3.nodefeaturediscoveries.nfd.kubernetes.io","uid":"e8ebfd1a-d7d7-44cc-9038-70bd5f1ae380","apiVersion":"v1","resourceVersion":"53680499"}, "reason": "LeaderElection", "message": "nfd-controller-manager-7cd8d54686-nllqc_c3d12674-228d-4747-9d07-f025b8aaf51e became leader"}
2021-08-16T17:01:25.122Z	DEBUG	controller-runtime.manager.events	Normal	{"object": {"kind":"Lease","namespace":"openshift-nfd","name":"39f5e5c3.nodefeaturediscoveries.nfd.kubernetes.io","uid":"10ebb14c-84f8-42be-ae4c-9eef1316e853","apiVersion":"coordination.k8s.io/v1","resourceVersion":"53680500"}, "reason": "LeaderElection", "message": "nfd-controller-manager-7cd8d54686-nllqc_c3d12674-228d-4747-9d07-f025b8aaf51e became leader"}
2021-08-16T17:01:25.223Z	INFO	controller-runtime.manager.controller.nodefeaturediscovery	Starting EventSource	{"reconciler group": "nfd.openshift.io", "reconciler kind": "NodeFeatureDiscovery", "source": "kind source: /, Kind="}

I think this problem occurs because LeaderElectionID is provided in the manager creation step:

        mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
                Scheme:                 scheme,
                MetricsBindAddress:     metricsAddr,
                Port:                   9443,
                LeaderElection:         enableLeaderElection,
                LeaderElectionID:       "39f5e5c3.nodefeaturediscoveries.nfd.kubernetes.io",
                HealthProbeBindAddress: probeAddr,
                Namespace:              watchNamespace, // namespaced-scope when the value is not an empty string
        })

If we remove LeaderElectionID and LeaderElection, then everything works as intended. If we remove just LeaderElectionID and leave LeaderElection there, an error occurs saying that LeaderElectionID must be defined, even though LeaderElection is set to false by default.

A better approach to this problem is to write an if-else conditional that says "If leader election is enabled, then [...]. Else, [...]", rather than providing a Leader election ID and relying on LeaderElection to disable Leader Election (since the true/false value of LeaderElection is ignored when LeaderElectionID is provided).

Will provide a PR fix for this soon.

@courtneypacheco
Copy link
Contributor Author

I want to add that even modifying the relevant manifests to set LeaderElection to false does not seem to disable leader election. We still need to have an 'if-else' conditional to fix this.

@courtneypacheco
Copy link
Contributor Author

Should be fixed with PR #202

@openshift-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 14, 2021
@openshift-bot
Copy link
Contributor

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@openshift-ci openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 14, 2021
@openshift-bot
Copy link
Contributor

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 13, 2022

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot closed this as completed Jan 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.
Projects
None yet
Development

No branches or pull requests

2 participants