Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logql benchmark and performance improvement. #1371

Merged
merged 4 commits into from
Dec 9, 2019

Conversation

cyriltovena
Copy link
Contributor

Hello !

This PR adds a benchmark for the LogQL engine and also improve allocations and therefore cpu usage when doing metrics queries. Improvement is around 40% in term of CPU and 80% for allocation/memory usage.

I can see there is still ways to improve metrics queries, but I didn't wanted to change the implementation too much, since we might add more to the language in the future.

Here is the result:

❯ benchcmp old.txt new.txt
benchmark                         old ns/op      new ns/op      delta
BenchmarkRangeQuery100000-16      3969200        1980841        -50.09%
BenchmarkRangeQuery200000-16      25067162       5204838        -79.24%
BenchmarkRangeQuery500000-16      2036930499     1394141266     -31.56%
BenchmarkRangeQuery1000000-16     3927444457     2844597499     -27.57%

benchmark                         old allocs     new allocs     delta
BenchmarkRangeQuery100000-16      49956          3518           -92.96%
BenchmarkRangeQuery200000-16      204431         9820           -95.20%
BenchmarkRangeQuery500000-16      13413890       4135290        -69.17%
BenchmarkRangeQuery1000000-16     26830056       8268585        -69.18%

benchmark                         old bytes      new bytes     delta
BenchmarkRangeQuery100000-16      1808878        321027        -82.25%
BenchmarkRangeQuery200000-16      9968806        719492        -92.78%
BenchmarkRangeQuery500000-16      822888736      262896816     -68.05%
BenchmarkRangeQuery1000000-16     1646616496     526540984     -68.02%

I'm also fixing the storage benchmark that is currently crashing because of a change in the limits.

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
Copy link
Collaborator

@slim-bean slim-bean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice LGTM!

@cyriltovena cyriltovena merged commit 1fc0ffd into grafana:master Dec 9, 2019
@cyriltovena cyriltovena deleted the logql-benchmark branch December 9, 2019 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants