Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changes benchmarks in preparation of value caching #38

Merged
merged 1 commit into from
Jun 14, 2023

Conversation

codefromthecrypt
Copy link
Contributor

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

The pod being scheduled is a parameter to all Scheduler framework callbacks. Passing this for each callback has overhead, because the main communication channel in wasm is its memory. The host must marshal the pod to proto and copy to to the guest's memory. Much more expensive is having the guest de-marshal it.

For example, a simple filter that unmarshals the pod takes roughly 65us. Every callback that uses pod metadata would cost something similar, possibly 4 or more times.

This change makes overhead more visible by having benchmarks access all parameters.

Which issue(s) this PR fixes:

NONE

Special notes for your reviewer:

This change is missing tests and RATIONALE. I'm raising it for early review in case the approach isn't accepted.

Does this PR introduce a user-facing change?

NONE

What are the benchmark results of this change?

Below you can see higher overhead than if we only use our examples, because more data is in use.

This makes any future optimizations visible, as we know what the base case is when all parameters are evaluated.

goos: darwin
goarch: arm64
pkg: sigs.k8s.io/kube-scheduler-wasm-extension/internal/e2e
BenchmarkPluginFilter
BenchmarkPluginFilter/noop-wat
BenchmarkPluginFilter/noop-wat/params:_small
BenchmarkPluginFilter/noop-wat/params:_small-12         	 9274990	       122.5 ns/op
BenchmarkPluginFilter/noop-wat/params:_real
BenchmarkPluginFilter/noop-wat/params:_real-12          	 9589492	       121.4 ns/op
BenchmarkPluginFilter/noop
BenchmarkPluginFilter/noop/params:_small
BenchmarkPluginFilter/noop/params:_small-12             	 6338865	       188.2 ns/op
BenchmarkPluginFilter/noop/params:_real
BenchmarkPluginFilter/noop/params:_real-12              	 6309466	       185.5 ns/op
BenchmarkPluginFilter/test
BenchmarkPluginFilter/test/params:_small
BenchmarkPluginFilter/test/params:_small-12             	  166362	      6822 ns/op
BenchmarkPluginFilter/test/params:_real
BenchmarkPluginFilter/test/params:_real-12              	    7332	    162329 ns/op
BenchmarkPluginScore
BenchmarkPluginScore/noop-wat
BenchmarkPluginScore/noop-wat/params:_small
BenchmarkPluginScore/noop-wat/params:_small-12          	 9267765	       127.4 ns/op
BenchmarkPluginScore/noop-wat/params:_real
BenchmarkPluginScore/noop-wat/params:_real-12           	 9341564	       125.7 ns/op
BenchmarkPluginScore/noop
BenchmarkPluginScore/noop/params:_small
BenchmarkPluginScore/noop/params:_small-12              	 4750544	       250.0 ns/op
BenchmarkPluginScore/noop/params:_real
BenchmarkPluginScore/noop/params:_real-12               	 5480874	       212.5 ns/op
BenchmarkPluginScore/test
BenchmarkPluginScore/test/params:_small
BenchmarkPluginScore/test/params:_small-12              	  300349	      3995 ns/op
BenchmarkPluginScore/test/params:_real
BenchmarkPluginScore/test/params:_real-12               	   21944	     52739 ns/op
BenchmarkPluginFilterAndScore
BenchmarkPluginFilterAndScore/noop-wat
BenchmarkPluginFilterAndScore/noop-wat/params:_small
BenchmarkPluginFilterAndScore/noop-wat/params:_small-12 	 4651711	       255.6 ns/op
BenchmarkPluginFilterAndScore/noop-wat/params:_real
BenchmarkPluginFilterAndScore/noop-wat/params:_real-12  	 4755004	       244.0 ns/op
BenchmarkPluginFilterAndScore/noop
BenchmarkPluginFilterAndScore/noop/params:_small
BenchmarkPluginFilterAndScore/noop/params:_small-12     	 2694627	       450.9 ns/op
BenchmarkPluginFilterAndScore/noop/params:_real
BenchmarkPluginFilterAndScore/noop/params:_real-12      	 2976104	       396.5 ns/op
BenchmarkPluginFilterAndScore/test
BenchmarkPluginFilterAndScore/test/params:_small
BenchmarkPluginFilterAndScore/test/params:_small-12     	  109630	     10638 ns/op
BenchmarkPluginFilterAndScore/test/params:_real
BenchmarkPluginFilterAndScore/test/params:_real-12      	    6324	    191027 ns/op

Signed-off-by: Adrian Cole <adrian@tetrate.io>
@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 14, 2023
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 14, 2023
Copy link
Member

@sanposhiho sanposhiho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍
/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 14, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: codefromthecrypt, sanposhiho

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 14, 2023
@k8s-ci-robot k8s-ci-robot merged commit 4adffbb into kubernetes-sigs:main Jun 14, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants