Revert the revert of #107456 - "apf: change controller to use SSA for patches" #108383

tkashem · 2022-02-28T02:17:11Z

What type of PR is this?

/kind bug

What this PR does / why we need it:

This brings back #107456 to see if it introduces flakes/timeouts in the integration tests

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

This reverts commit 83ca745, reversing changes made to 1c216c6.

tkashem · 2022-02-28T02:17:26Z

cc @MikeSpreitzer

MadhavJivrajani · 2022-02-28T09:07:04Z

/retest

tkashem · 2022-02-28T13:07:29Z

/test pull-kubernetes-integration

tkashem · 2022-02-28T15:19:04Z

/test pull-kubernetes-integration

tkashem · 2022-02-28T16:39:44Z

/test pull-kubernetes-integration

tkashem · 2022-02-28T18:30:00Z

/test pull-kubernetes-integration

tkashem · 2022-02-28T20:47:38Z

/test pull-kubernetes-integration

tkashem · 2022-02-28T21:32:45Z

/test pull-kubernetes-integration

tkashem · 2022-03-01T14:12:33Z

/test pull-kubernetes-integration

jpbetz · 2022-03-01T21:10:38Z

/triage accepted
/cc @wojtek-t @MikeSpreitzer

tkashem · 2022-03-01T21:24:41Z

/test pull-kubernetes-integration

tkashem · 2022-03-02T14:56:03Z

8 tries for pull-kubernetes-integration, one failure https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/108383/pull-kubernetes-integration/1498771235025195008

/test pull-kubernetes-integration

aojea · 2022-03-02T17:20:35Z

8 tries for pull-kubernetes-integration, one failure https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/108383/pull-kubernetes-integration/1498771235025195008

/test pull-kubernetes-integration

the good thing is that we store now the number of leaked goroutines

grep goroutines junit_20220301-214450.stdout                                                                                                                                                                                                                                                                                            
{"Time":"2022-03-01T21:47:35.765303573Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/apiserver/tracing","Output":"I0301 21:47:35.765240  118920 etcd.go:198] unexpected number of goroutines: before: 3 after 1541\n"}
{"Time":"2022-03-01T21:47:36.042225892Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/apiserver/podlogs","Output":"I0301 21:47:36.042152  118909 etcd.go:198] unexpected number of goroutines: before: 3 after 987\n"}
{"Time":"2022-03-01T21:48:23.455810868Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/apimachinery","Output":"I0301 21:48:23.455757  118727 etcd.go:198] unexpected number of goroutines: before: 3 after 2999\n"}
{"Time":"2022-03-01T21:49:24.644900783Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/apiserver/certreload","Output":"I0301 21:49:24.644831  118887 etcd.go:198] unexpected number of goroutines: before: 3 after 4879\n"}
{"Time":"2022-03-01T21:49:34.396372627Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/certificates","Output":"I0301 21:49:34.396148  119243 etcd.go:198] unexpected number of goroutines: before: 3 after 3556\n"}
{"Time":"2022-03-01T21:49:39.848027866Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/configmap","Output":"I0301 21:49:39.840470  119384 etcd.go:198] unexpected number of goroutines: before: 3 after 1007\n"}
{"Time":"2022-03-01T21:49:46.230610039Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/apiserver/flowcontrol","Output":"I0301 21:49:46.230537  118898 etcd.go:198] unexpected number of goroutines: before: 3 after 3101\n"}
{"Time":"2022-03-01T21:50:01.919001956Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/cronjob","Output":"I0301 21:50:01.918935  119624 etcd.go:198] unexpected number of goroutines: before: 3 after 1024\n"}
{"Time":"2022-03-01T21:50:27.469369553Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/defaulttolerationseconds","Output":"I0301 21:50:27.469289  120470 etcd.go:198] unexpected number of goroutines: before: 3 after 1007\n"}
{"Time":"2022-03-01T21:51:04.646134767Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/apiserver/admissionwebhook","Output":"I0301 21:51:04.644256  118934 etcd.go:198] unexpected number of goroutines: before: 3 after 3059\n"}
{"Time":"2022-03-01T21:59:12.016035665Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/endpoints","Output":"I0301 21:59:12.015900  121066 etcd.go:198] unexpected number of goroutines: before: 3 after 2009\n"}
{"Time":"2022-03-01T21:59:16.159152794Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/disruption","Output":"I0301 21:59:16.158942  120640 etcd.go:198] unexpected number of goroutines: before: 3 after 1132\n"}
{"Time":"2022-03-01T21:59:21.79111696Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/dryrun","Output":"I0301 21:59:21.791056  121004 etcd.go:198] unexpected number of goroutines: before: 3 after 249\n"}
{"Time":"2022-03-01T21:59:26.547260113Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/events","Output":"I0301 21:59:26.547192  121184 etcd.go:198] unexpected number of goroutines: before: 3 after 250\n"}
{"Time":"2022-03-01T21:59:50.580302255Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/evictions","Output":"I0301 21:59:50.580239  121294 etcd.go:198] unexpected number of goroutines: before: 3 after 3014\n"}
{"Time":"2022-03-01T21:59:54.337125792Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/examples","Output":"I0301 21:59:54.337064  122051 etcd.go:198] unexpected number of goroutines: before: 3 after 1311\n"}
{"Time":"2022-03-01T21:59:54.535763682Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/endpointslice","Output":"I0301 21:59:54.535359  121085 etcd.go:198] unexpected number of goroutines: before: 3 after 7010\n"}
{"Time":"2022-03-01T22:00:08.67585337Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/etcd","Output":"I0301 22:00:08.675757  121107 etcd.go:198] unexpected number of goroutines: before: 3 after 4501\n"}
{"Time":"2022-03-01T22:00:12.982432894Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/kubelet","Output":"I0301 22:00:12.982373  123146 etcd.go:198] unexpected number of goroutines: before: 3 after 1799\n"}
{"Time":"2022-03-01T22:00:26.31239131Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/dualstack","Output":"I0301 22:00:26.312325  121056 etcd.go:198] unexpected number of goroutines: before: 3 after 13975\n"}
{"Time":"2022-03-01T22:00:27.118015838Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/namespace","Output":"I0301 22:00:27.117949  123196 etcd.go:198] unexpected number of goroutines: before: 3 after 2005\n"}
{"Time":"2022-03-01T22:00:32.315697862Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/network","Output":"I0301 22:00:32.315633  123279 etcd.go:198] unexpected number of goroutines: before: 3 after 1007\n"}
{"Time":"2022-03-01T22:00:36.265988295Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/metrics","Output":"I0301 22:00:36.265897  123156 etcd.go:198] unexpected number of goroutines: before: 3 after 4991\n"}
{"Time":"2022-03-01T22:00:39.954476249Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/objectmeta","Output":"I0301 22:00:39.951263  123394 etcd.go:198] unexpected number of goroutines: before: 3 after 1007\n"}
{"Time":"2022-03-01T22:00:56.681304173Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/scale","Output":"I0301 22:00:56.681230  123663 etcd.go:198] unexpected number of goroutines: before: 3 after 243\n"}
{"Time":"2022-03-01T22:01:08.592488254Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/quota","Output":"I0301 22:01:08.592421  123620 etcd.go:198] unexpected number of goroutines: before: 3 after 3029\n"}
{"Time":"2022-03-01T22:01:13.412467178Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/node","Output":"I0301 22:01:13.412396  123300 etcd.go:198] unexpected number of goroutines: before: 3 after 4072\n"}
{"Time":"2022-03-01T22:01:28.189577469Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/secrets","Output":"I0301 22:01:28.189512  123910 etcd.go:198] unexpected number of goroutines: before: 3 after 1007\n"}
{"Time":"2022-03-01T22:01:29.149558437Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/pods","Output":"I0301 22:01:29.149495  123592 etcd.go:198] unexpected number of goroutines: before: 3 after 5987\n"}
{"Time":"2022-03-01T22:02:22.638666535Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/service","Output":"I0301 22:02:22.637945  123951 etcd.go:198] unexpected number of goroutines: before: 3 after 9202\n"}
{"Time":"2022-03-01T22:02:23.662929701Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/serviceaccount","Output":"I0301 22:02:23.662866  124182 etcd.go:198] unexpected number of goroutines: before: 3 after 3995\n"}
{"Time":"2022-03-01T22:02:28.565856594Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/replicationcontroller","Output":"I0301 22:02:28.565782  123643 etcd.go:198] unexpected number of goroutines: before: 3 after 14109\n"}
{"Time":"2022-03-01T22:02:40.556993448Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/garbagecollector","Output":"I0301 22:02:40.556910  122677 etcd.go:198] unexpected number of goroutines: before: 3 after 3089\n"}
{"Time":"2022-03-01T22:02:41.715607952Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/replicaset","Output":"I0301 22:02:41.715575  123632 etcd.go:198] unexpected number of goroutines: before: 3 after 19133\n"}
{"Time":"2022-03-01T22:02:44.831681394Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/storageclasses","Output":"I0301 22:02:44.831625  126124 etcd.go:198] unexpected number of goroutines: before: 3 after 1007\n"}
{"Time":"2022-03-01T22:02:55.018113641Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/tls","Output":"I0301 22:02:55.018041  126293 etcd.go:198] unexpected number of goroutines: before: 3 after 124\n"}
{"Time":"2022-03-01T22:03:03.992507601Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/job","Output":"I0301 22:03:03.992386  123134 etcd.go:198] unexpected number of goroutines: before: 3 after 21023\n"}
{"Time":"2022-03-01T22:03:05.208920923Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/ttlcontroller","Output":"I0301 22:03:05.208626  126304 etcd.go:198] unexpected number of goroutines: before: 3 after 1007\n"}
{"Time":"2022-03-01T22:03:12.927648568Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/storageversion","Output":"I0301 22:03:12.927576  126146 etcd.go:198] unexpected number of goroutines: before: 3 after 729\n"}
{"Time":"2022-03-01T22:03:13.391092536Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/statefulset","Output":"I0301 22:03:13.391011  126114 etcd.go:198] unexpected number of goroutines: before: 3 after 4256\n"}
{"Time":"2022-03-01T22:03:23.847942205Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/serving","Output":"I0301 22:03:23.847813  126083 etcd.go:198] unexpected number of goroutines: before: 3 after 799\n"}
{"Time":"2022-03-01T22:05:17.670386047Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/volume","Output":"I0301 22:05:17.670298  126367 etcd.go:198] unexpected number of goroutines: before: 3 after 15019\n"}
{"Time":"2022-03-01T22:07:15.35259352Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/volumescheduling","Output":"I0301 22:07:15.352351  126410 etcd.go:198] unexpected number of goroutines: before: 3 after 14290\n"}
{"Time":"2022-03-01T22:10:04.216211003Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/scheduler","Output":"I0301 22:10:04.216110  123754 etcd.go:198] unexpected number of goroutines: before: 3 after 68597\n"}

after that no more tests are scheduled

tail -2 junit_20220301-214450.stdout                                                                                                                                                                                                                                                                                                         SIGINT 
{"Time":"2022-03-01T22:10:04.504166796Z","Action":"output","Package":"k8s.io/kubernetes/test/integration/scheduler","Output":"ok  \tk8s.io/kubernetes/test/integration/scheduler\t537.040s\n"}
{"Time":"2022-03-01T22:10:04.504214527Z","Action":"pass","Package":"k8s.io/kubernetes/test/integration/scheduler","Elapsed":537.108}

some tests timeout, but they run until the end

tail -25 build-log.txt 
W0301 21:58:37.319462  120640 reflector.go:442] k8s.io/client-go/informers/factory.go:134: watch of *v1.RuntimeClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received
W0301 21:58:37.319494  120640 reflector.go:442] k8s.io/client-go/informers/factory.go:134: watch of *v1.StorageClass ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received
W0301 21:58:37.320235  120640 cacher.go:150] Terminating all watchers from cacher *admissionregistration.MutatingWebhookConfiguration
W0301 21:58:37.320676  120640 reflector.go:442] k8s.io/client-go/informers/factory.go:134: watch of *v1.MutatingWebhookConfiguration ended with: very short watch: k8s.io/client-go/informers/factory.go:134: Unexpected watch close - watch lasted less than a second and no items received
    testserver.go:331: failed to launch server: failed to wait for default namespace to be created: timed out waiting for the condition
    --- FAIL: TestEmptySelector/v1_should_target_all_pods (423.64s)

=== FAIL: test/integration/disruption TestEmptySelector (431.70s)

DONE 3872 tests, 53 skipped, 11 failures in 83.247s
+++ [0301 22:11:29] Saved JUnit XML test report to /logs/artifacts/junit_20220301-214450.xml
make[1]: *** [Makefile:185: test] Error 1
!!! [0301 22:11:29] Call tree:
!!! [0301 22:11:29]  1: hack/make-rules/test-integration.sh:96 runTests(...)
+++ [0301 22:11:29] Cleaning up etcd
+++ [0301 22:11:29] Integration test cleanup complete
make: *** [Makefile:204: test-integration] Error 1
+ EXIT_VALUE=2
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up after docker
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
================================================================================
Done cleaning up after docker in docker.

The job may not be able to parse the size output to reports the results

-rw-r--r--   1 aojea aojea 2,9G mar  2 16:59  junit_20220301-214450.stdout

All the test failures show problems related to grpc connecting to etcd 🤷 , but it is possible that a lot of those orphan goroutines are etcd connections lingering

tkashem · 2022-03-14T12:23:08Z

/assign @wojtek-t @MikeSpreitzer

MikeSpreitzer

/LGTM

wojtek-t · 2022-03-21T09:16:00Z

/lgtm
/approve

k8s-ci-robot · 2022-03-21T09:16:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MikeSpreitzer, tkashem, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~staging/src/k8s.io/apiserver/pkg/util/flowcontrol/OWNERS~~ [MikeSpreitzer,wojtek-t]
~~test/integration/apiserver/OWNERS~~ [wojtek-t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-triage-robot · 2022-03-21T12:33:24Z

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

The PR does have any do-not-merge/* labels
The PR does not have the needs-ok-to-test label
The PR is mergeable (does not have a needs-rebase label)
The PR is approved (has cncf-cla: yes, lgtm, approved labels)
The PR is failing tests required for merge

You can:

Review the full test history for this PR
Prevent this bot from retesting with /lgtm cancel or /hold
Help make our tests less flaky by following our Flaky Tests Guide

/retest

MikeSpreitzer · 2022-03-21T14:42:09Z

/test pull-kubernetes-dependencies

MikeSpreitzer · 2022-03-21T17:11:40Z

/retest

k8s-triage-robot · 2022-03-21T19:53:43Z

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

The PR does have any do-not-merge/* labels
The PR does not have the needs-ok-to-test label
The PR is mergeable (does not have a needs-rebase label)
The PR is approved (has cncf-cla: yes, lgtm, approved labels)
The PR is failing tests required for merge

You can:

Review the full test history for this PR
Prevent this bot from retesting with /lgtm cancel or /hold
Help make our tests less flaky by following our Flaky Tests Guide

/retest

k8s-triage-robot · 2022-03-22T00:55:12Z

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

The PR does have any do-not-merge/* labels
The PR does not have the needs-ok-to-test label
The PR is mergeable (does not have a needs-rebase label)
The PR is approved (has cncf-cla: yes, lgtm, approved labels)
The PR is failing tests required for merge

You can:

Review the full test history for this PR
Prevent this bot from retesting with /lgtm cancel or /hold
Help make our tests less flaky by following our Flaky Tests Guide

/retest

k8s-triage-robot · 2022-03-22T03:55:12Z

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

The PR does have any do-not-merge/* labels
The PR does not have the needs-ok-to-test label
The PR is mergeable (does not have a needs-rebase label)
The PR is approved (has cncf-cla: yes, lgtm, approved labels)
The PR is failing tests required for merge

You can:

Review the full test history for this PR
Prevent this bot from retesting with /lgtm cancel or /hold
Help make our tests less flaky by following our Flaky Tests Guide

/retest

MadhavJivrajani · 2022-03-22T04:31:59Z

/retest

liggitt · 2022-03-25T18:24:13Z

This reintroduced panics in integration tests:

#109026 (comment)

liggitt · 2022-03-25T18:27:07Z

opened a revert. please don't reintroduce it until there's a clean integration test run without panics like "Observed a panic: FieldManager must be installed to run apply" in the log.

integration tests have been very flaky since this merged (and failed 3/18 runs on this PR = ~16% failure rate)... it looks like this is causing 40k+ panics per run, and a log size of 3gb

tkashem · 2022-03-25T18:34:44Z

okay, reverting it.

Revert "Merge pull request kubernetes#107797 from tkashem/revert-107456"

b0b4609

This reverts commit 83ca745, reversing changes made to 1c216c6.

k8s-ci-robot requested a review from lavalamp February 28, 2022 02:17

k8s-ci-robot added the area/apiserver label Feb 28, 2022

k8s-ci-robot requested a review from SataQiu February 28, 2022 02:17

k8s-ci-robot added area/test sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Feb 28, 2022

k8s-ci-robot requested review from MikeSpreitzer and wojtek-t March 1, 2022 21:10

k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Mar 1, 2022

tkashem changed the title ~~Revert "Merge pull request #107797 from tkashem/revert-107456"~~ Revert the revert of #107456 - "apf: change controller to use SSA for patches" Mar 7, 2022

k8s-ci-robot assigned MikeSpreitzer and wojtek-t Mar 14, 2022

MikeSpreitzer approved these changes Mar 18, 2022

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 18, 2022

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 21, 2022

k8s-ci-robot merged commit 9a8defd into kubernetes:master Mar 22, 2022

k8s-ci-robot added this to the v1.24 milestone Mar 22, 2022

liggitt mentioned this pull request Mar 25, 2022

apiserver routines failing in integration tests #109026

Closed

liggitt mentioned this pull request Mar 25, 2022

Revert "Revert "Merge pull request #107797 from tkashem/revert-107456"" #109030

Merged

liggitt mentioned this pull request Apr 1, 2022

pull-kubernetes-integration failing ~60-90% of runs - updated 2022-03-31 #109038

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revert the revert of #107456 - "apf: change controller to use SSA for patches" #108383

Revert the revert of #107456 - "apf: change controller to use SSA for patches" #108383

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

MadhavJivrajani commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Mar 1, 2022

jpbetz commented Mar 1, 2022

tkashem commented Mar 1, 2022

tkashem commented Mar 2, 2022

aojea commented Mar 2, 2022 •

edited

tkashem commented Mar 14, 2022

MikeSpreitzer left a comment

wojtek-t commented Mar 21, 2022

k8s-ci-robot commented Mar 21, 2022

k8s-triage-robot commented Mar 21, 2022

MikeSpreitzer commented Mar 21, 2022

MikeSpreitzer commented Mar 21, 2022

k8s-triage-robot commented Mar 21, 2022

k8s-triage-robot commented Mar 22, 2022

k8s-triage-robot commented Mar 22, 2022

MadhavJivrajani commented Mar 22, 2022

liggitt commented Mar 25, 2022

liggitt commented Mar 25, 2022 •

edited

tkashem commented Mar 25, 2022

Revert the revert of #107456 - "apf: change controller to use SSA for patches" #108383

Revert the revert of #107456 - "apf: change controller to use SSA for patches" #108383

Conversation

tkashem commented Feb 28, 2022

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

tkashem commented Feb 28, 2022

MadhavJivrajani commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Feb 28, 2022

tkashem commented Mar 1, 2022

jpbetz commented Mar 1, 2022

tkashem commented Mar 1, 2022

tkashem commented Mar 2, 2022

aojea commented Mar 2, 2022 • edited

tkashem commented Mar 14, 2022

MikeSpreitzer left a comment

Choose a reason for hiding this comment

wojtek-t commented Mar 21, 2022

k8s-ci-robot commented Mar 21, 2022

k8s-triage-robot commented Mar 21, 2022

MikeSpreitzer commented Mar 21, 2022

MikeSpreitzer commented Mar 21, 2022

k8s-triage-robot commented Mar 21, 2022

k8s-triage-robot commented Mar 22, 2022

k8s-triage-robot commented Mar 22, 2022

MadhavJivrajani commented Mar 22, 2022

liggitt commented Mar 25, 2022

liggitt commented Mar 25, 2022 • edited

tkashem commented Mar 25, 2022

aojea commented Mar 2, 2022 •

edited

liggitt commented Mar 25, 2022 •

edited