kubelet: fix flaky unit test with TestRotateLogs due to improper handling of goroutine in the tests

[harshanarayana]

What type of PR is this?

/kind failing-test
/kind flake

What this PR does / why we need it:

Fix the TestRotateLogs function in kubelet's UTs to handle the invocation of processQueueItems properly. This was added by changes introduced in #114301.

The test was flaking due to improper handling of the routines leading to flaky test failures when running under stress.

Which issue(s) this PR fixes:

Fixes #123414

Special notes for your reviewer:

Additional details on what lead to the issue can be found in the comment #123414 (comment)

❯ stress ./logs.test -test.run TestRotateLogs -test.v
5s: 0 runs so far, 0 failures
10s: 36 runs so far, 0 failures
15s: 84 runs so far, 0 failures
20s: 127 runs so far, 0 failures
25s: 168 runs so far, 0 failures
30s: 207 runs so far, 0 failures
35s: 254 runs so far, 0 failures
40s: 297 runs so far, 0 failures
45s: 340 runs so far, 0 failures
50s: 380 runs so far, 0 failures
55s: 420 runs so far, 0 failures
1m0s: 448 runs so far, 0 failures
1m5s: 473 runs so far, 0 failures
1m10s: 503 runs so far, 0 failures
1m15s: 533 runs so far, 0 failures
1m20s: 564 runs so far, 0 failures
1m25s: 599 runs so far, 0 failures
1m30s: 635 runs so far, 0 failures
1m35s: 672 runs so far, 0 failures
1m40s: 698 runs so far, 0 failures
1m45s: 731 runs so far, 0 failures
1m50s: 770 runs so far, 0 failures
1m55s: 796 runs so far, 0 failures
2m0s: 824 runs so far, 0 failures
2m5s: 851 runs so far, 0 failures
2m10s: 885 runs so far, 0 failures
2m15s: 909 runs so far, 0 failures
2m20s: 948 runs so far, 0 failures
2m25s: 993 runs so far, 0 failures
2m30s: 1037 runs so far, 0 failures
2m35s: 1084 runs so far, 0 failures
2m40s: 1132 runs so far, 0 failures
2m45s: 1180 runs so far, 0 failures
2m50s: 1223 runs so far, 0 failures
2m55s: 1264 runs so far, 0 failures
3m0s: 1310 runs so far, 0 failures
3m5s: 1358 runs so far, 0 failures
3m10s: 1403 runs so far, 0 failures
3m15s: 1448 runs so far, 0 failures
3m20s: 1496 runs so far, 0 failures
3m25s: 1539 runs so far, 0 failures
3m30s: 1586 runs so far, 0 failures
3m35s: 1634 runs so far, 0 failures
3m40s: 1683 runs so far, 0 failures
3m45s: 1730 runs so far, 0 failures
3m50s: 1776 runs so far, 0 failures
3m55s: 1820 runs so far, 0 failures
4m0s: 1863 runs so far, 0 failures

Does this PR introduce a user-facing change?

NA

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

NA

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[harshanarayana]

/cc @kannon92 @dims @Rajalakshmi-Girish

PTAL

[dims]

/approve
/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 7184e887a34742899a056167c41a54be900b7f99

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims, harshanarayana

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/kubelet/OWNERS [dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment