Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of eventfilter lookups #5288

Merged
merged 2 commits into from Apr 20, 2021
Merged

Improve performance of eventfilter lookups #5288

merged 2 commits into from Apr 20, 2021

Conversation

benmoss
Copy link
Member

@benmoss benmoss commented Apr 20, 2021

Only do event field lookups as needed, don't pre-emptively cache them into a map

Fixes #5277

Previous:

$ go test -benchtime 10s -bench=. ./pkg/eventfilter/benchmarks/
goos: linux
goarch: amd64
pkg: knative.dev/eventing/pkg/eventfilter/benchmarks
cpu: Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz
BenchmarkAttributesFilter/Creation:_Pass_with_exact_match_of_id-12              1000000000               2.699 ns/op
BenchmarkAttributesFilter/Run:_Pass_with_exact_match_of_id-12                    1675917              7419 ns/op
BenchmarkAttributesFilter/Creation:_Pass_with_exact_match_of_all_context_attributes_(except_time)-12            1000000000               2.697 ns/op
BenchmarkAttributesFilter/Run:_Pass_with_exact_match_of_all_context_attributes_(except_time)-12                  1697353              7299 ns/op
BenchmarkAttributesFilter/Creation:_No_pass_with_exact_match_of_id_and_source-12                                1000000000               2.683 ns/op
BenchmarkAttributesFilter/Run:_No_pass_with_exact_match_of_id_and_source-12                                      1570285              6987 ns/op

Now:

$ go test -benchtime 10s -bench=. ./pkg/eventfilter/benchmarks/
goos: linux
goarch: amd64
pkg: knative.dev/eventing/pkg/eventfilter/benchmarks
cpu: Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz
BenchmarkAttributesFilter/Creation:_Pass_with_exact_match_of_id-12              1000000000               2.686 ns/op
BenchmarkAttributesFilter/Run:_Pass_with_exact_match_of_id-12                   143855318               82.71 ns/op
BenchmarkAttributesFilter/Creation:_Pass_with_exact_match_of_all_context_attributes_(except_time)-12            1000000000               2.689 ns/op
BenchmarkAttributesFilter/Run:_Pass_with_exact_match_of_all_context_attributes_(except_time)-12                 14394333               999.9 ns/op
BenchmarkAttributesFilter/Creation:_No_pass_with_exact_match_of_id_and_source-12                                1000000000               2.733 ns/op
BenchmarkAttributesFilter/Run:_No_pass_with_exact_match_of_id_and_source-12                                     39759967               563.0 ns/op

Release Note

:page_facing_up: MTChannelBasedBrokers will see improved filter performance for triggers with few or no trigger filters

Only do event field lookups as needed, don't pre-emptively cache them
into a map
@knative-prow-robot knative-prow-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 20, 2021
@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Apr 20, 2021
@n3wscott
Copy link
Contributor

Am I reading those numbers right? this is a 10x improvement?

@knative-metrics-robot
Copy link

The following is the coverage report on the affected files.
Say /test pull-knative-eventing-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/eventfilter/attributes/filter.go 100.0% 66.7% -33.3

@codecov
Copy link

codecov bot commented Apr 20, 2021

Codecov Report

Merging #5288 (bcdec63) into main (60bc1cf) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #5288      +/-   ##
==========================================
+ Coverage   83.63%   83.65%   +0.02%     
==========================================
  Files         243      243              
  Lines        6879     6891      +12     
==========================================
+ Hits         5753     5765      +12     
  Misses        783      783              
  Partials      343      343              
Impacted Files Coverage Δ
pkg/eventfilter/attributes/filter.go 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60bc1cf...bcdec63. Read the comment docs.

@benmoss
Copy link
Member Author

benmoss commented Apr 20, 2021

Am I reading those numbers right? this is a 10x improvement?

Avoiding unnecessary function calls and using a switch instead of a map, who knew!

@benmoss
Copy link
Member Author

benmoss commented Apr 20, 2021

/test pull-knative-eventing-go-coverage

Copy link
Contributor

@antoineco antoineco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM
Amazing finding!

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 20, 2021
Copy link
Contributor

@n3wscott n3wscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking care of that so fast!!

@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: antoineco, n3wscott

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow-robot knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 20, 2021
@knative-prow-robot knative-prow-robot merged commit f1909db into knative:main Apr 20, 2021
@benmoss benmoss deleted the eventfilter-perf branch April 21, 2021 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Indicates the PR's author has signed the CLA. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[pkg/eventfilter/attributes/filter.go] Filter method should be rewritten to me more performant.
5 participants