perf: do not match each token in aggregation - #474
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #474 +/- ##
==========================================
+ Coverage 71.16% 71.20% +0.04%
==========================================
Files 232 232
Lines 18466 18492 +26
==========================================
+ Hits 13141 13168 +27
+ Misses 4343 4340 -3
- Partials 982 984 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
🔴 Performance DegradationSome benchmarks have degraded compared to the previous run. Show table
|
1651f58 to
81cd8b1
Compare
8dadbde to
c431f4e
Compare
|
@seqbenchbot start search-aggregation 0-skip-matching count client_ip --query 'k8s_pod:payment-backend-eu-0' --duration 5m |
|
Nice, @dkharms Started benchmark Show details
I'll post the statistics here once it finishes. Here is a list of helpful links: Have a great time! |
🔴 Performance DegradationSome benchmarks have degraded compared to the previous run. Show table
|
|
Nice, @dkharms Benchmark Show summary
Compare it against another run with |
|
@seqbenchbot start search-aggregation main count client_ip --query 'k8s_pod:payment-backend-eu-0' --duration 5m |
|
Nice, @dkharms Started benchmark Show details
I'll post the statistics here once it finishes. Here is a list of helpful links: Have a great time! |
|
Oh-oh, @dkharms Something went wrong and I couldn't process your request. |
Description
When getting tokens for aggregation, we scan each token and checking it against
*-- so basically, useless computations and IO (we have to load+unpack token blocks). I suggest to return range[firstTID; lastTID].Obviously, this optimization will have more impact on high-cardinality fields that span over several token blocks (like
remote_addror something).mean (ms)stddev (ms)p(50) (ms)p(95) (ms)p(99) (ms)iterations304.33287.94-5.39%7.484.93-34.12%303.76287.90-5.22%317.10305.11-3.78%317.98312.73-1.65%52.0053.00+1.92%168.43163.55-2.90%4.154.27+3.03%168.14163.17-2.96%176.48173.26-1.82%181.75180.69-0.58%711.00731.00+2.81%132.04129.33-2.05%5.334.21-21.03%131.31128.26-2.32%143.63137.18-4.49%2637.98142.37-94.60%54.0057.00+5.56%28.0527.68-1.31%1.871.50-19.56%27.6527.42-0.84%33.6632.99-2.02%38.0136.31-4.49%4208.004258.00+1.19%If you have used LLM/AI assistance please provide model name and full prompt: