[Flaky] Creating a Job In a Twostepadmission Queue [It] Should unsuspend a job only after all checks are cleared

[alculquicondor]

What happened:

Kueue when Creating a Job In a Twostepadmission Queue [It] Should unsuspend a job only after all checks are cleared
/home/prow/go/src/sigs.k8s.io/kueue/test/e2e/e2e_test.go:215
  [FAILED] Timed out after 30.001s.
  Expected
      <[]interface {} | len:2, cap:2>: [
          <bool>true,
          <map[string]string | len:0>nil,
      ]
  to equal
      <[]interface {} | len:2, cap:2>: [
          <bool>false,
          <map[string]string | len:1>{
              "instance-type": "on-demand",
          },
      ]
  In [It] at: /home/prow/go/src/sigs.k8s.io/kueue/test/e2e/e2e_test.go:270 @ 08/30/23 19:13:45.757

How to reproduce it (as minimally and precisely as possible):

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/1031/pull-kueue-test-e2e-main-1-24/1696962388068143104

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• Kueue version (use git describe --tags --dirty --always):
• Cloud provider or hardware configuration:
• OS (e.g: cat /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[tenzen-y]

Same here: https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/1094/pull-kueue-test-e2e-main-1-24/1698634053189636096

[tenzen-y]

Same: https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/1096/pull-kueue-test-e2e-main-1-24/1698634117425401856

[alculquicondor]

@achernevskii could you take a look?

[achernevskii]

/assign

[alculquicondor]

/unassign @achernevskii
/assign @trasc

This is flakying a lot, can you PTAL Traian?

[trasc]

Sure

[trasc]

#1127 should fix this particular case , however the main problem looks to be the fact the the eviction is not longer working as expected.

[alculquicondor]

Can you elaborate?

[tenzen-y]

however the main problem looks to be the fact the the eviction is not longer working as expected.

@trasc Could you clarify the non-expected behavior in the eviction?

[trasc]

Even with the "false" admission, the flow wold have been ,

1. Admit(before first reconcile)
2. Evict (during the first reconcile when the ACs are added)
3. Readmit

However, at least in the configured timeout, the eviction is not ending and the readmission will not take place.

If remember correctly, "Kueue when Creating a Job With Queueing Should readmit preempted job with workloadPriorityClass into a separate flavor" was flaky as well , but since we no longer test 1.24 and we no longer have access to the tests grid I'n not sure.

[tenzen-y]

Even with the "false" admission, the flow wold have been ,

1. Admit(before first reconcile)
2. Evict (during the first reconcile when the ACs are added)
3. Readmit

However, at least in the configured timeout, the eviction is not ending and the readmission will not take place.

If remember correctly, "Kueue when Creating a Job With Queueing Should readmit preempted job with workloadPriorityClass into a separate flavor" was flaky as well , but since we no longer test 1.24 and we no longer have access to the tests grid I'n not sure.

I see. Thanks for clarifying.