operator: LeaderElectionReleaseOnCancel #556

zeeke · 2023-12-05T16:51:50Z

When manually restarting the operator, the leader election
takes 5+ minutes to acquire the lease on startup:

I1205 16:06:02.101302       1 leaderelection.go:245] attempting to acquire leader lease openshift-sriov-network-operator/a56def2a.openshift.io...
...
I1205 16:08:40.133558       1 leaderelection.go:255] successfully acquired lease openshift-sriov-network-operator/a56def2a.openshift.io

This PR makes sure the lease is released when the operator shutdown

github-actions · 2023-12-05T16:52:03Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

coveralls · 2023-12-05T16:58:41Z

Pull Request Test Coverage Report for Build 8325705141

Details

0 of 0 changed or added relevant lines in 0 files are covered.
3 unchanged lines in 1 file lost coverage.
Overall coverage remained the same at 37.718%

Files with Coverage Reduction	New Missed Lines	%
controllers/drain_controller.go	3	70.68%

Totals
Change from base Build 8324682841:	0.0%
Covered Lines:	4837
Relevant Lines:	12824

💛 - Coveralls

github-actions · 2023-12-06T08:55:33Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2023-12-06T08:55:41Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2023-12-06T10:07:55Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2023-12-06T11:43:15Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

zeeke · 2023-12-06T11:48:29Z

I validated this change with a test commit. This comes from the related CI logs:

// operator logs
I1206 10:27:04.430837       1 leaderelection.go:250] attempting to acquire leader lease sriov-network-operator/a56def2a.openshift.io...
I1206 10:27:04.435885       1 leaderelection.go:260] successfully acquired lease sriov-network-operator/a56def2a.openshift.io

// events.json
{
            "metadata": {
                "name": "a56def2a.openshift.io.179e37328a9a0420",
                "namespace": "sriov-network-operator",
                ...
            },
            "involvedObject": {
                "kind": "Lease",
                "namespace": "sriov-network-operator",
                "name": "a56def2a.openshift.io",
                ...
            },
            "reason": "LeaderElection",
            "message": "sriov-network-operator-787ddd7794-xl66c_d5427070-808c-4c52-ae03-c53bf37c6e9f became leader",
            "source": {
                "component": "sriov-network-operator-787ddd7794-xl66c_d5427070-808c-4c52-ae03-c53bf37c6e9f"
            },
            "firstTimestamp": "2023-12-06T10:26:39Z",
            ...
            "reportingComponent": "sriov-network-operator-787ddd7794-xl66c_d5427070-808c-4c52-ae03-c53bf37c6e9f",
            "reportingInstance": ""
        },
        {
            "metadata": {
                "name": "a56def2a.openshift.io.179e37382af3762a",
                "namespace": "sriov-network-operator",
                ...
            },
            "involvedObject": {
                "kind": "Lease",
                "namespace": "sriov-network-operator",
                "name": "a56def2a.openshift.io",
                ...
            },
            "reason": "LeaderElection",
            "message": "sriov-network-operator-787ddd7794-xl66c_d5427070-808c-4c52-ae03-c53bf37c6e9f stopped leading",
            "source": {
                "component": "sriov-network-operator-787ddd7794-xl66c_d5427070-808c-4c52-ae03-c53bf37c6e9f"
            },
            "firstTimestamp": "2023-12-06T10:27:03Z",
            ...
            "reportingComponent": "sriov-network-operator-787ddd7794-xl66c_d5427070-808c-4c52-ae03-c53bf37c6e9f",
            "reportingInstance": ""
        },
        {
            "metadata": {
                "name": "a56def2a.openshift.io.179e37385e817eda",
                "namespace": "sriov-network-operator",
                ...
            },
            "involvedObject": {
                "kind": "Lease",
                "namespace": "sriov-network-operator",
                "name": "a56def2a.openshift.io",
                ...
            },
            "reason": "LeaderElection",
            "message": "sriov-network-operator-787ddd7794-vmsck_a6f02db2-11d5-4dc2-b4b6-8c615ba7f452 became leader",
            "source": {
                "component": "sriov-network-operator-787ddd7794-vmsck_a6f02db2-11d5-4dc2-b4b6-8c615ba7f452"
            },
            "firstTimestamp": "2023-12-06T10:27:04Z",
            ...
            "reportingComponent": "sriov-network-operator-787ddd7794-vmsck_a6f02db2-11d5-4dc2-b4b6-8c615ba7f452",
        },

github-actions · 2023-12-14T12:26:16Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

adrianchiris · 2023-12-18T18:00:47Z

controller runtime defaults seem much shorter.

i see we override with openshift inspired defaults.

lease is 137s and renew 26s, so it should take no longer than 2.7min
in your logs its 2.6min

given its not 5+ min, do we really need this ?

adrianchiris · 2023-12-18T18:13:09Z

main.go

+		setupLog.Info("starting leader election manager")
+		if err := leaderElectionMgr.Start(leaderElectionContext); err != nil {
+			setupLog.Error(err, "Leader Election Manager exited non-zero")
+			os.Exit(1)


something seems off to me with the use of os.Exit() here and below

maybe just need to use waitgroup to wait for these managers to complete.

what if we os.Exit() before any defer statements are executed. it may not really be an issue IDK.

TBH, I don't know: if one manager returns an error while the others are running, it should signal a stop and wait. But it could become a little bit complicated and there is a risk of getting deadlock as far as I can see.
I can explore a little more if you think it's worth

+1 on looking how to streamline this

main.go

mlguerrero12 · 2023-12-19T14:56:11Z

main.go

-		setupLog.Error(err, "unable to set up ready check")
-		os.Exit(1)
-	}
+	leaderElectionContext, cancelLeaderElection := context.WithCancel(context.Background())


I think it's better to create this WithCancel context using the stopCh context (created below) instead of a context.Background

That way leaderElectionContext would stop as soon as the stop context is Done. Which is what I'm trying to achieve with this PR.

I tried using nested contexts here
https://go.dev/play/p/6dFfBQyXlW1

by using the stop context as parent of this one, the leaderElectionMgr.Start function will be cancelled if a singint or sigterm is received.

Yes, that's what I don't want to happen. Here's the ending sequence

sigint arrive

stopCh is Done

mgr and mgrGlobals finish their work and return

as mgr is not on a go routing, when it returns it complete the function, triggering all the defers

utils.Shutdown() defer goes first. do the cleanup (the leader election is still running here, so we have the lock lease)

cancelLeaderElection() defer goes second, stopping the manager which in turn release the lock

Am I missing something here?

Right, it would be the same as using the ReleaseOnCancel option.

My concern here is that after calling cancelLeaderElection() (with the defer), the program exits, so nothing ensures that leaderElectionMgr really finishes. There is a race condition.

The internal implementation of the ReleaseOnCancel option uses a channel to ensure that the routine has stopped before continuing with the shutdown. Perhaps you can do the same here.

My concern here is that after calling cancelLeaderElection() (with the defer), the program exits, so nothing ensures that leaderElectionMgr really finishes. There is a race condition.

Added a wait group to ensure leaderElectionMgr is correctly stopped

it seems to me that this never exits. The leader election manager is stopped with a defer but before that, you are waiting for this to stop, so, it won't happen?

main.go

zeeke · 2023-12-19T15:17:09Z

controller runtime defaults seem much shorter.

i see we override with openshift inspired defaults.

lease is 137s and renew 26s, so it should take no longer than 2.7min in your logs its 2.6min

given its not 5+ min, do we really need this ?

You're right: regular clusters may get a delay of 2.7min at most. With Single Node Openshift (SNO in this context) it can grow to 5 minutes.

sriov-network-operator/pkg/leaderelection/leaderelection.go

Line 45 in 572dc27

    
           func leaderElectionSingleNodeConfig(config leaderelection.LeaderElectionConfig) leaderelection.LeaderElectionConfig {

openshift/library-go@2612981#diff-61dd95c7fd45fa18038e825205fbfab8a803f1970068157608b6b1e9e6c27248R127

I'm taking that math for granted, and one of our customers is experiencing that 5 minutes restart during upgrades, on SNO.

I1123 13:20:44.792821       1 leaderelection.go:245] attempting to acquire leader lease openshift-sriov-network-operator/a56def2a.openshift.io...
I1123 13:25:56.033843       1 leaderelection.go:255] successfully acquired lease openshift-sriov-network-operator/a56def2a.openshift.io

github-actions · 2023-12-19T16:58:41Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

SchSeba · 2023-12-20T13:12:36Z

Hi @zeeke @adrianchiris not related to this PR directly but question do you know why we have two manges?? and now 3

zeeke · 2023-12-20T13:52:28Z

Hi @zeeke @adrianchiris not related to this PR directly but question do you know why we have two manges?? and now 3

AFAIU, mgr watches resource in the operator's namespace. mgrGlobal watches every namespace, as it runs Sriov[IB}Network reconcilers which react on NetworkAttachmentDefinitions.

I guess we can unify those managers with one that watches all namespaces, but maybe we get some performance degradation.

Also, I'm investigating about the shutdown logic to see if we can get rid of some bits and pieces, as I'm not very happy with the current election-proxy-manager proposal

github-actions · 2023-12-20T17:15:27Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2023-12-21T09:15:38Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-02-27T17:23:37Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-02-28T11:08:04Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-02-28T13:05:16Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-02-28T14:16:32Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-02-28T17:03:16Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

zeeke · 2024-03-05T16:13:23Z

Shutdown logic for the long run will be discussed in

Operator's shutdown logic redsign #608

@SchSeba, @e0ne , @adrianchiris Please take another look at this.

I added an end-to-end test to verify the restart of the operator is fast enough

SchSeba

I left some small comments nice work :)

deploy/operator.yaml

hack/run-e2e-conformance-virtual-ocp.sh

SchSeba · 2024-03-14T12:51:12Z

test/conformance/tests/test_sriov_operator.go

+				g.Expect(err).ToNot(HaveOccurred())
+
+				g.Expect(newLease.Spec.HolderIdentity).ToNot(Equal(oldLease.Spec.HolderIdentity))
+			}, 45*time.Second, 5*time.Second).Should(Succeed())


This one I a complicated test. if the operator starts on another master for example we need to add the time that takes for the image to get pulled into the node.

maybe we wait for the new pod to be running and then we do the leader elector check?

good point! This will save us a lot of flakes.
Fixing

github-actions · 2024-03-14T15:46:04Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

SchSeba

LGTM

SchSeba · 2024-03-18T10:25:40Z

please rebase this one an open the issue so we don't forget :)

When manually restarting the operator, the leader election may take 5+ minutest to acquire the lease on startup: ``` I1205 16:06:02.101302 1 leaderelection.go:245] attempting to acquire leader lease openshift-sriov-network-operator/a56def2a.openshift.io... ... I1205 16:08:40.133558 1 leaderelection.go:255] successfully acquired lease openshift-sriov-network-operator/a56def2a.openshift.io ``` The manager's option `LeaderElectionReleaseOnCancel` would solve this problem, but it's not safe as the shutdown cleanup procedures (inhibiting webhooks and removing finalizers) would run without any leader guard. This commit moves the LeaderElection mechanism from the namespaced manager to a dedicated, no-op controller manager. This approach has been preferred to directly dealing with the LeaderElection API as: - It leverages library code that has been proved to be stable - It includes recording k8s Events about the Lease process - The election process must come after setting up the health probe. Doing it manually would involve handling the healthz endpoint as well. Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

Add CoordinationV1 to `test/util/clients.go` to make assertions on `coordination.k8s.io/Lease` objects. Add `OPERATOR_LEADER_ELECTION_ENABLE` environment variable to `deploy/operator.yaml` to let user enable leader election on the operator. Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

github-actions · 2024-03-18T11:23:35Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

zeeke · 2024-03-18T11:23:58Z

please rebase this one an open the issue so we don't forget :)

rebased! The discussion about shutdown is here:

Operator's shutdown logic redsign #608

SchSeba · 2024-03-21T14:19:54Z

@e0ne @ykulazhenkov if you have time please take a look on this PR please

ykulazhenkov

Hi @zeeke @adrianchiris not related to this PR directly but question do you know why we have two manges?? and now 3

AFAIU, mgr watches resource in the operator's namespace. mgrGlobal watches every namespace, as it runs Sriov[IB}Network reconcilers which react on NetworkAttachmentDefinitions.

I guess we can unify those managers with one that watches all namespaces, but maybe we get some performance degradation.

Also, I'm investigating about the shutdown logic to see if we can get rid of some bits and pieces, as I'm not very happy with the current election-proxy-manager proposal

@zeeke

TBH, I think we should try to use the single manager and rely on cancellation logic from the controller-runtime package.
Recent versions of the controller-runtime package provides a way to configure cache/watches in a very granular way. It is possible to watch some resources in all namespace and other only in specific namespace.

Here is example:
https://github.com/Mellanox/nvidia-k8s-ipam/blob/a34b4a2547815b7c016df7d563a2e04000f0add3/cmd/ipam-node/app/app.go#L146

WDYT?

zeeke · 2024-03-25T07:58:26Z

Hi @zeeke @adrianchiris not related to this PR directly but question do you know why we have two manges?? and now 3

AFAIU, mgr watches resource in the operator's namespace. mgrGlobal watches every namespace, as it runs Sriov[IB}Network reconcilers which react on NetworkAttachmentDefinitions.
I guess we can unify those managers with one that watches all namespaces, but maybe we get some performance degradation.
Also, I'm investigating about the shutdown logic to see if we can get rid of some bits and pieces, as I'm not very happy with the current election-proxy-manager proposal

@zeeke

TBH, I think we should try to use the single manager and rely on cancellation logic from the controller-runtime package. Recent versions of the controller-runtime package provides a way to configure cache/watches in a very granular way. It is possible to watch some resources in all namespace and other only in specific namespace.

Here is example: https://github.com/Mellanox/nvidia-k8s-ipam/blob/a34b4a2547815b7c016df7d563a2e04000f0add3/cmd/ipam-node/app/app.go#L146

WDYT?

Hi @ykulazhenkov , Thank you for your feedback.

I agree with you we should simplify as much as we can. With your suggestion, we can merge the namespace and the non-namespaced manager to a single one, but it's not the problem this PR is trying to solve.

The problem here is that the operator has a shutdown logic with two constraints:
a. it must run with the controllers turned off
b. it must run as a cluster leader, i.e. no other sriov-network-operators are supposed to do any logic during that shutdown logic

The cleaner solution I found (apart from getting rid of the shutdown logic, see discussions/608) is to add a new manager that controls only the leader election.

I admit it's probably not the best solution, but it solves an issue found in a production environment (OCPBUGS-23795)

ykulazhenkov · 2024-03-25T08:34:50Z

Hi @ykulazhenkov , Thank you for your feedback.

I agree with you we should simplify as much as we can. With your suggestion, we can merge the namespace and the non-namespaced manager to a single one, but it's not the problem this PR is trying to solve.

The problem here is that the operator has a shutdown logic with two constraints: a. it must run with the controllers turned off b. it must run as a cluster leader, i.e. no other sriov-network-operators are supposed to do any logic during that shutdown logic

The cleaner solution I found (apart from getting rid of the shutdown logic, see discussions/608) is to add a new manager that controls only the leader election.

I admit it's probably not the best solution, but it solves an issue found in a production environment (OCPBUGS-23795)

thx, for the clarification

ykulazhenkov

LGTM

SchSeba

LGTM

Thanks!

zeeke force-pushed the fast-leader-election branch 2 times, most recently from 9472a4c to 361c81b Compare December 6, 2023 08:55

zeeke force-pushed the fast-leader-election branch from 361c81b to cca761b Compare December 6, 2023 10:07

zeeke force-pushed the fast-leader-election branch from cca761b to f8fba94 Compare December 6, 2023 11:43

zeeke marked this pull request as draft December 6, 2023 17:05

zeeke force-pushed the fast-leader-election branch from f8fba94 to 9894ab3 Compare December 14, 2023 12:26

adrianchiris reviewed Dec 18, 2023

View reviewed changes

main.go Outdated Show resolved Hide resolved

zeeke marked this pull request as ready for review December 19, 2023 13:47

mlguerrero12 reviewed Dec 19, 2023

View reviewed changes

main.go Outdated Show resolved Hide resolved

mlguerrero12 reviewed Dec 19, 2023

View reviewed changes

main.go Outdated Show resolved Hide resolved

zeeke force-pushed the fast-leader-election branch from 9894ab3 to 1328f48 Compare December 19, 2023 16:58

zeeke force-pushed the fast-leader-election branch from 1328f48 to 28cff08 Compare December 20, 2023 17:15

zeeke force-pushed the fast-leader-election branch from 28cff08 to 1dac637 Compare December 21, 2023 09:15

zeeke force-pushed the fast-leader-election branch from d3643ba to a045409 Compare February 28, 2024 13:05

zeeke force-pushed the fast-leader-election branch from a045409 to 77eea87 Compare February 28, 2024 14:16

zeeke force-pushed the fast-leader-election branch from 77eea87 to ca913c1 Compare February 28, 2024 17:03

SchSeba reviewed Mar 14, 2024

View reviewed changes

zeeke force-pushed the fast-leader-election branch from ca913c1 to 88f560d Compare March 14, 2024 15:45

SchSeba approved these changes Mar 18, 2024

View reviewed changes

zeeke added 3 commits March 18, 2024 12:21

e2e: dump info on deploy-wait.sh error

8431672

Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

zeeke force-pushed the fast-leader-election branch from 88f560d to 0ef001c Compare March 18, 2024 11:23

ykulazhenkov reviewed Mar 25, 2024

View reviewed changes

ykulazhenkov approved these changes Mar 25, 2024

View reviewed changes

zeeke requested review from e0ne, adrianchiris and Eoghan1232 March 29, 2024 11:49

SchSeba approved these changes May 2, 2024

View reviewed changes

SchSeba merged commit c2d9e32 into k8snetworkplumbingwg:master May 2, 2024
12 checks passed

operator: LeaderElectionReleaseOnCancel #556

operator: LeaderElectionReleaseOnCancel #556

Conversation

zeeke commented Dec 5, 2023

github-actions bot commented Dec 5, 2023

coveralls commented Dec 5, 2023 • edited Loading

Pull Request Test Coverage Report for Build 8325705141

Details

💛 - Coveralls

github-actions bot commented Dec 6, 2023

github-actions bot commented Dec 6, 2023

github-actions bot commented Dec 6, 2023

github-actions bot commented Dec 6, 2023

zeeke commented Dec 6, 2023

github-actions bot commented Dec 14, 2023

adrianchiris commented Dec 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeeke commented Dec 19, 2023

github-actions bot commented Dec 19, 2023

SchSeba commented Dec 20, 2023

zeeke commented Dec 20, 2023 • edited Loading

github-actions bot commented Dec 20, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Feb 27, 2024

github-actions bot commented Feb 28, 2024

github-actions bot commented Feb 28, 2024

github-actions bot commented Feb 28, 2024

github-actions bot commented Feb 28, 2024

zeeke commented Mar 5, 2024

SchSeba left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Mar 14, 2024

SchSeba left a comment

Choose a reason for hiding this comment

SchSeba commented Mar 18, 2024 • edited Loading

github-actions bot commented Mar 18, 2024

zeeke commented Mar 18, 2024

SchSeba commented Mar 21, 2024

ykulazhenkov left a comment

Choose a reason for hiding this comment

zeeke commented Mar 25, 2024

ykulazhenkov commented Mar 25, 2024

ykulazhenkov left a comment

Choose a reason for hiding this comment

SchSeba left a comment

Choose a reason for hiding this comment

coveralls commented Dec 5, 2023 •

edited

Loading

zeeke commented Dec 20, 2023 •

edited

Loading

SchSeba commented Mar 18, 2024 •

edited

Loading