Refactor the Load Generator and Support SteadyState Types #7265

rthallisey · 2022-02-22T15:11:39Z

Refactor the load generator code.

Define workload types as an interface so it can be extended to accommodate each job type
Add support for steady-state job
Simplify the config (rely on outside scripts to handle ns creation, iterations, ect...)

Test below shows a steady-state workload with churn: 2 count: 5. This will create 5 VMIs and delete 2 VMIs frequently over a 5m period.

name: kubevirt-steady-state-test
timeout: 5m
count: 5
churn: 2
minChurnSleep: 20s
type: "steady-state"
object:
  templateFile: vmi-ephemeral.yaml
  inputVars:
    containerPrefix: quay.io/kubevirt
    containerImg: cirros-container-disk-demo
    containerTag: ""
    namespace: default

Release note:

Support steady-state job types in the load-generator tool

Future work:

The SteadyState CreateWorkload is really the same as Burst CreateWorkload. Call Burst create from SteadyState.
Refacore the Watcher package

rthallisey · 2022-02-22T15:11:54Z

/cc @marceloamaral @davidvossel

marceloamaral

This PR will affect two prow jobs. So before merging this, you also need another PR to fix those jobs.

periodic-kubevirt-performance-cluster-100-density-test

periodic-kubevirt-performance-cluster-scale-density-test

Both jobs execute the automation/perfscale-test.sh, which run the perfscale-load-generator tool. You also need to update this script accordingly to the new logic.

/hold

marceloamaral · 2022-02-24T06:57:15Z

tools/perfscale-load-generator/config/config.go

-	config.QPS = float32(t.Global.QPS)
-	config.Burst = t.Global.Burst
+	config.QPS = QPS
+	config.Burst = Burst


Why the client QPS and Burst are not configurable anymore?
In fact, by default I was disabling rate limiting for the client library (i.e., 0 values)

I wanted to reduce config options. I default to 20 which should be plenty for the time being.

Not sure if we should hard-code this.... Actually, we should avoid hard-coded parameters....

Changed both to unlimited. We can come back and change this as the tool matures.

tools/perfscale-load-generator/object/unstructured.go

rthallisey · 2022-03-07T19:39:43Z

/retest

rthallisey · 2022-03-07T20:14:44Z

created kubevirt/project-infra#1946. I think this change can go in first and 1946 can quickly follow.

rthallisey · 2022-03-07T20:14:54Z

/unhold

rthallisey · 2022-03-10T14:54:21Z

/retest

marceloamaral · 2022-04-07T12:41:40Z

/retest
/lgtm

rthallisey · 2022-04-14T14:09:23Z

/retest

davidvossel

/approve

kubevirt-bot · 2022-04-20T16:15:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: davidvossel

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [davidvossel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

rthallisey · 2022-04-20T17:28:49Z

rebased. @marceloamaral need another lgtm from you

rthallisey · 2022-04-21T12:48:59Z

/retest

marceloamaral · 2022-04-22T01:43:14Z

/lgtm

kubevirt-commenter-bot · 2022-04-22T06:31:45Z

/retest-required
This bot automatically retries required jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

marceloamaral suggested changes Mar 3, 2022

View reviewed changes

kubevirt-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 3, 2022

kubevirt-bot assigned marceloamaral Mar 3, 2022

kubevirt-bot added dco-signoff: no Indicates the PR's author has not DCO signed all their commits. size/XXL labels Mar 5, 2022

rthallisey force-pushed the refactor-lg branch 2 times, most recently from db0867a to 6c50d82 Compare March 7, 2022 18:46

rthallisey force-pushed the refactor-lg branch from 6c50d82 to 11bcd66 Compare March 7, 2022 18:47

kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 7, 2022

kubevirt-bot added dco-signoff: no Indicates the PR's author has not DCO signed all their commits. and removed dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Mar 7, 2022

rthallisey force-pushed the refactor-lg branch from 085584a to eda3880 Compare March 7, 2022 20:09

kubevirt-bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. and removed dco-signoff: no Indicates the PR's author has not DCO signed all their commits. labels Mar 7, 2022

rthallisey mentioned this pull request Mar 7, 2022

Use the load generators refactored config kubevirt/project-infra#1946

Merged

kubevirt-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 7, 2022

kubevirt-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Mar 10, 2022

rthallisey force-pushed the refactor-lg branch from eda3880 to ef1a579 Compare March 10, 2022 16:15

kubevirt-bot added dco-signoff: no Indicates the PR's author has not DCO signed all their commits. and removed dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Mar 10, 2022

rthallisey force-pushed the refactor-lg branch from ef1a579 to 6cc3353 Compare March 10, 2022 16:15

kubevirt-bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. and removed dco-signoff: no Indicates the PR's author has not DCO signed all their commits. labels Mar 10, 2022

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Apr 7, 2022

davidvossel reviewed Apr 20, 2022

View reviewed changes

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 20, 2022

rthallisey added 10 commits April 20, 2022 13:24

Refactor loadgenerator

68246af

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Add SteadyState test

970c3c7

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Update examples

6cafc41

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Update README

4f53200

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Add another abstraction layer

fa81bfa

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Add Create helper function

0281e49

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Update path perfscale-tests.sh

775ee56

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

MinChurnSleep will sleep a configurable amount of time between churn

a74c8a4

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Add 400 and 600 density tests

20b46c7

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

Set unlimited QPS and Burst

0a2631e

Signed-off-by: Ryan Hallisey <rhallisey@nvidia.com>

rthallisey force-pushed the refactor-lg branch from 6cc3353 to 0a2631e Compare April 20, 2022 17:28

kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Apr 20, 2022

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Apr 22, 2022

kubevirt-bot merged commit 52e6e9b into kubevirt:main Apr 22, 2022

rthallisey deleted the refactor-lg branch April 22, 2022 14:33

marceloamaral mentioned this pull request May 11, 2022

fix perf test removing a deprecated parameter #7724

Merged

rthallisey mentioned this pull request May 12, 2022

KubeVirt Performance test framework #5878

Closed

17 tasks

marceloamaral mentioned this pull request Jun 1, 2022

Fix the updated version of perfscale load generator, VMIs are not being created #7839

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor the Load Generator and Support SteadyState Types #7265

Refactor the Load Generator and Support SteadyState Types #7265

rthallisey commented Feb 22, 2022 •

edited

rthallisey commented Feb 22, 2022

marceloamaral left a comment

marceloamaral Feb 24, 2022

rthallisey Mar 7, 2022

marceloamaral Mar 10, 2022

rthallisey Mar 10, 2022

rthallisey commented Mar 7, 2022

rthallisey commented Mar 7, 2022

rthallisey commented Mar 7, 2022

rthallisey commented Mar 10, 2022

marceloamaral commented Apr 7, 2022

rthallisey commented Apr 14, 2022

davidvossel left a comment

kubevirt-bot commented Apr 20, 2022

rthallisey commented Apr 20, 2022

rthallisey commented Apr 21, 2022

marceloamaral commented Apr 22, 2022

kubevirt-commenter-bot commented Apr 22, 2022

Refactor the Load Generator and Support SteadyState Types #7265

Refactor the Load Generator and Support SteadyState Types #7265

Conversation

rthallisey commented Feb 22, 2022 • edited

rthallisey commented Feb 22, 2022

marceloamaral left a comment

Choose a reason for hiding this comment

marceloamaral Feb 24, 2022

Choose a reason for hiding this comment

rthallisey Mar 7, 2022

Choose a reason for hiding this comment

marceloamaral Mar 10, 2022

Choose a reason for hiding this comment

rthallisey Mar 10, 2022

Choose a reason for hiding this comment

rthallisey commented Mar 7, 2022

rthallisey commented Mar 7, 2022

rthallisey commented Mar 7, 2022

rthallisey commented Mar 10, 2022

marceloamaral commented Apr 7, 2022

rthallisey commented Apr 14, 2022

davidvossel left a comment

Choose a reason for hiding this comment

kubevirt-bot commented Apr 20, 2022

rthallisey commented Apr 20, 2022

rthallisey commented Apr 21, 2022

marceloamaral commented Apr 22, 2022

kubevirt-commenter-bot commented Apr 22, 2022

rthallisey commented Feb 22, 2022 •

edited