Decouple yaml based integration test from legacy test

[chendave]

• Move utilities and constants out so that both of them should be able
  to run independently.
• Rename the legacy test so that it can eventually be deleted when the
  perf dash changes is done

There are two approaches to run scheduler_perf testcases with different ways to define the test spec, both of them could be able to run independently.

• test spec is defined in the yaml

- template:
    desc: SchedulingPodAntiAffinity
    nodes:
      uniqueNodeLabelStrategy:
        labelKey: kubernetes.io/hostname
    initPods:
      podTemplatePath: config/pod-with-pod-anti-affinity.yaml
    podsToSchedule:
      podTemplatePath: config/pod-with-pod-anti-affinity.yaml
  params:
    - numNodes: 500
      numInitPods: 100
      numPodsToSchedule: 400
    - numNodes: 5000
      numInitPods: 1000
      numPodsToSchedule: 1000

• test spec is defined in the golang source.

func BenchmarkSchedulingPodAntiAffinity(b *testing.B) {
	// Since the pods has anti affinity to each other, the number of pods to schedule
	// can't exceed the number of nodes (the topology used in the test)
	tests := []struct{ nodes, existingPods, minPods int }{
		{nodes: 500, existingPods: 100, minPods: 400},
		{nodes: 5000, existingPods: 1000, minPods: 1000},
	}
       ...
}

The second one becomes duplicated along with the merge of defining the test spec in the yaml file. There is no need to maintain both of them in the code base, this change refactor the code to decouple one from the other and drop the support of the legacy way to run the scheduler_perf.

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:

/kind cleanup

What this PR does / why we need it:

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[chendave]

/test pull-kubernetes-conformance-kind-ipv6-parallel
/test pull-kubernetes-integration
/test pull-kubernetes-e2e-gce
/test pull-kubernetes-bazel-build

[chendave]

/test pull-kubernetes-verify
/test pull-kubernetes-e2e-kind-ipv6
/test pull-kubernetes-e2e-gce

[chendave]

/test pull-kubernetes-verify

[chendave]

/test pull-kubernetes-e2e-kind-ipv6

[chendave]

/test pull-kubernetes-kubemark-e2e-gce-big

[chendave]

I think we can remove the legacy way to run scheduler perf directly, or mark the legacy way as deprecated?
Will push another PR later.

[hex108]

@chendave It seems that what the PR does is different from its description? It only moves a function and some variables to the util file.

[chendave]

@chendave It seems that what the PR does is different from its description? It only moves a function and some variables to the util file.

I think there are two approaches to run those benchmarks, one is based on the yaml template, and another one is based on inline test spec, they are peer-to-peer instead of one approach depends on another one.

But current code base has some method and consts defined in the legacy inline test spec, this doesn't make sense in my opinion, simply because I should be able to run the benchmark without any dependencies in the legacy code (which eventually should be removed in the future).

[chendave]

@hex108 , thank you for the review and comment! hope you get my points.

[chendave]

remove this file: test/integration/scheduler_perf/scheduler_bench_test.go (eventually, this should be deleted in the future).

and then run the scheduler perf with the new way,
make test-integration WHAT=./test/integration/scheduler_perf KUBE_TEST_VMODULE="''" KUBE_TEST_ARGS="-alsologtostderr=false -logtostderr=false -run=^$$ -benchtime=1ns -bench=BenchmarkPerfScheduling/SchedulingBasic/5000Nodes/5000InitPods/1000PodsToSchedule"

you will find it will fail with some constants / method missing.

[chendave]

/test pull-kubernetes-verify

[chendave]

/test pull-kubernetes-verify

[chendave]

/test pull-kubernetes-verify

[chendave]

@hex108 how about deprecate those test suit? at least the new one should not depends on the old one (see my first PR).

[Huang-Wei]

@ahg-g @liu-cong @ingvagabund, is this PR resolve part of what you discussed in #88195 to replace legacy BenchmarkScheduling suite with new BenchmarkPerfScheduling suite? I suppose there is some change needed for test-infra/dashboard as well?

[chendave]

/test pull-kubernetes-e2e-kind

[ingvagabund]

It makes sense to do scheduler_bench_test.go → scheduler_perf_legacy_test.go as noted in #88195 (comment). Though, it no longer make sense to change anything in scheduler_bench_test.go unless it's a bug fix. We will keep both suites running in parallel for few releases to compare the measurements. The new test suite is already running here: http://perf-dash.k8s.io/#/?jobname=scheduler-perf-benchmark&metriccategoryname=Scheduler&metricname=BenchmarkPerfResults&Metric=scheduler_scheduling_algorithm_predicate_evaluation_seconds&Name=BenchmarkPerfScheduling%2FSchedulingBasic%2F500Nodes%2F500InitPods%2F1000PodsToSchedule

[chendave]

Though, it no longer make sense to change anything in scheduler_bench_test.go unless it's a bug fix.

The refactor makes it possible to run those integration test without the code dependencies in the legacy approach. This is somehow align with what you said below.

We will keep both suites running in parallel for few releases to compare the measurements.

[ingvagabund]

The refactor makes it possible to run those integration test without the code dependencies in the legacy approach. This is somehow align with what you said below.

@chendave oh, I see what you mean. The code you moved from scheduler_bench_test.go to util.go is used in both scheduler_bench_test.go and scheduler_perf_test.go. Yeah, good catch. Changes look good. Can you rebase the PR?

[ingvagabund]

/lgtm

[alculquicondor]

/assign

[alculquicondor]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alculquicondor, chendave

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• test/integration/scheduler_perf/OWNERS [alculquicondor]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ingvagabund]

/test pull-kubernetes-e2e-gce

[Huang-Wei]

/retest

[chendave]

/sig scheduling