E2E run serial tests in parallel suite

[aojea]

GinkgoV1 doesn't support to run Serial and Parallel tests in the same suite, this caused us to set up different jobs for Serial tests.

@onsi implemented this long standing request in V2 , so users can run Serial tests at the same time, Ginkgo will do the magic

Serial allows the user to mark specs and containers of specs as only eligible to run in serial. Ginkgo will guarantee that these specs never run in parallel with other specs.

https://onsi.github.io/ginkgo/#the-serial-decorator

This can simplify a lot the maintenance of current test-infra jobs and help Conformance users

/sig testing

[aojea]

/cc @pohly @chendave @BenTheElder @johnSchnake

@chendave since you already do the heavylifting for ginkgov2 check if this is something you are interested and feel free to assign it to yourself

[chendave]

@aojea thanks Antonio! pls let me do some investigation first.

[BenTheElder]

Since "serial" means "these tests are disruptive and can be the only test running at the time" it is inherently slow to run these ...

We also exclude these in some CI jobs because of the time it takes to run them ... 🤔

[aojea]

Since "serial" means "these tests are disruptive and can be the only test running at the time" it is inherently slow to run these ...

We also exclude these in some CI jobs because of the time it takes to run them ... thinking

oh, yeah, I forget about that, I was thinking in this open issue for Conformance #76344

so, these means we need to handle Dirsuptive and Slow too XD , or just do it only for Conformance, last time I've checked there were only 6 Serial tests in Conformance

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[pohly]

/remove-lifecycle stale
/triage accepted
/assign

[pohly]

So what is the conclusion? Is it worthwhile to move serial, non-slow, non-disruptive tests into the "normal" Prow jobs? It feels conceptually right to me.

But do we have any such test at all? Yes, it seems so:

$ git grep '\[Serial\]' test/e2e/ | grep -v -e '\[Slow\']  -e '\[Disruptive\]' | wc -l
46

Looking at some of those, several seem to be about checking metrics. That's not disruptive. They just cannot run in parallel because it needs "exclusive" access to the cluster.

[pohly]

/remove-lifecycle stale

[BenTheElder]

We'll need to be careful about this, since test times will still go up adding serial tests to jobs that were only doing parallel before.

FWIW I think most of the Serial tests have previously been poor candidates for PR jobs and in post-submit it's relatively fine to have two jobs.