[Flaking Test] TestLog/stateful_set_logs_with_all_pods

[Rajalakshmi-Girish]

Which jobs are flaking?

https://prow.k8s.io/job-history/gs/ppc64le-kubernetes/logs/periodic-kubernetes-unit-test-ppc64le

Which tests are flaking?

TestLog/stateful_set_logs_with_all_pods in staging/src/k8s.io/kubectl/pkg/cmd/logs/logs_test.go

Since when has it been flaking?

#124732
This flaking test case was newly added as part of the above PR

Testgrid link

https://testgrid.k8s.io/ibm-k8s-unit-tests-ppc64le#periodic-k8s-unit-tests-ppc64le

Reason for failure (if possible)

{Failed;Failed;  === RUN   TestLog/stateful_set_logs_with_all_pods
    logs_test.go:557: stateful set logs with all pods: expected to contain "[pod/test-sts-0/test-container] test log content for pod test-sts-0\n[pod/test-sts-1/test-container] test log content for pod test-sts-1\n". Output: "[pod/test-sts-1/test-container] test log content for pod test-sts-1\n[pod/test-sts-0/test-container] test log content for pod test-sts-0\n"
    --- FAIL: TestLog/stateful_set_logs_with_all_pods (0.08s)
;=== RUN   TestLog
--- FAIL: TestLog (1.55s)
;}

Anything else we need to know?

Flake was reproducible with stress tool:
[root@centos-8-small-9f46c kubernetes]# go version
go version go1.22.2 linux/ppc64le

[root@centos-8-small-9f46c kubernetes]# stress ./logs.test -test.run ^TestLog

/tmp/go-stress-20240522T102036-3748936792
--- FAIL: TestLog (0.35s)
    --- FAIL: TestLog/stateful_set_logs_with_all_pods (0.02s)
        logs_test.go:557: stateful set logs with all pods: expected to contain "[pod/test-sts-0/test-container] test log content for pod test-sts-0\n[pod/test-sts-1/test-container] test log content for pod test-sts-1\n". Output: "[pod/test-sts-1/test-container] test log content for pod test-sts-1\n[pod/test-sts-0/test-container] test log content for pod test-sts-0\n"
FAIL


ERROR: exit status 1


/tmp/go-stress-20240522T102036-1557253734
--- FAIL: TestLog (0.35s)
    --- FAIL: TestLog/stateful_set_logs_with_all_pods (0.01s)
        logs_test.go:557: stateful set logs with all pods: expected to contain "[pod/test-sts-0/test-container] test log content for pod test-sts-0\n[pod/test-sts-1/test-container] test log content for pod test-sts-1\n". Output: "[pod/test-sts-1/test-container] test log content for pod test-sts-1\n[pod/test-sts-0/test-container] test log content for pod test-sts-0\n"
FAIL

Relevant SIG(s)

/sig testing

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[srivastav-abhishek]

/assign

[srivastav-abhishek]

Looks like expectedOutSubstrings in this test case is a concatenation of logs from Pod0 and Pod1, hence it is failing in some cases. I will add a PR to split it.

[eddiezane]

https://storage.googleapis.com/k8s-triage/index.html?test=k8s.io%2Fkubectl%2Fpkg%2Fcmd%2Flogs.logs