Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Failing Test] CSI Mock volume storage capacity storage capacity unlimited #125547

Closed
AdminTurnedDevOps opened this issue Jun 17, 2024 · 3 comments
Labels
kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability.

Comments

@AdminTurnedDevOps
Copy link

AdminTurnedDevOps commented Jun 17, 2024

Which jobs are failing?

master-informing
gce-master-scale-correctness

CSI Mock volume storage capacity storage capacity unlimited

Which tests are failing?

Kubernetes e2e suite: [It] [sig-storage] CSI Mock volume storage capacity storage capacity unlimited expand_less | 40s
-- | --
{ failed [FAILED] while waiting for all CSI calls: error waiting for expected CSI calls: Unexpected CSI call 0: expected CreateVolume (0), got DeleteVolume (0) In [It] at: k8s.io/kubernetes/test/e2e/storage/csimock/csi_storage_capacity.go:196 @ 06/16/24 17:45:57.515 }

Looks like there's an extra Worker Node running when it's not supposed to, or at least not scheduled to.

Detected 5000 ready nodes, found 5001 nodes out of expected 5001. Your cluster may not be fully functional.

Since when has it been failing?

It looks like failures started on 4/18/2024, but they may have been deemed flakes until now as it's now happening every day.

Testgrid link

https://testgrid.k8s.io/sig-release-master-informing#gce-master-scale-correctness

Reason for failure (if possible)

No response

Anything else we need to know?

On k8s-triage, I'm also seeing this error tied to a storage issue. It may be because there's an extra Worker Node running when there's not supposed to be, so there isn't enough storage availab.e

[TIMEDOUT] A node timeout occurred
In [It] at: k8s.io/kubernetes/test/e2e/storage/csimock/csi_storage_capacity.go:323 @ 05/31/24 19:00:20.242

This is the Progress Report generated when the node timeout occurred:
  [sig-storage] CSI Mock volume storage capacity CSIStorageCapacity CSIStorageCapacity used, no capacity (Spec Runtime: 5m0.092s)
    k8s.io/kubernetes/test/e2e/storage/csimock/csi_storage_capacity.go:323
    In [It] (Node Runtime: 5m0.001s)
      k8s.io/kubernetes/test/e2e/storage/csimock/csi_storage_capacity.go:323
      At [By Step] deploying csi mock driver (Step Runtime: 4m59.937s)
        k8s.io/kubernetes/test/e2e/storage/drivers/csi.go:558

Relevant SIG(s)

/sig scalability

@AdminTurnedDevOps AdminTurnedDevOps added the kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. label Jun 17, 2024
@k8s-ci-robot k8s-ci-robot added sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jun 17, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@Vyom-Yadav
Copy link
Member

Duplicate of #125539

/close

@k8s-ci-robot
Copy link
Contributor

@Vyom-Yadav: Closing this issue.

In response to this:

Duplicate of #125539

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability.
Projects
Development

No branches or pull requests

3 participants