Commit 7429039
authored
perf(aw-transform): cache categorize results per event data to fix month-view timeout (#657)
* perf(aw-transform): cache categorize results per event data to avoid redundant regex matching
For a typical month with 50k+ events (from aw-watcher-window) and 20 category
rules, categorize() was performing ~1M regex evaluations because every event was
matched against every rule individually.
Most events in a heartbeat-based watcher share identical data (same app+title).
This adds an in-function HashMap cache keyed on the serialized event data JSON so
that only the first occurrence of each distinct data fingerprint is matched against
the rule set; subsequent identical events reuse the cached category.
Expected speedup for a month-view query with 50k events and O(100) distinct
app/title pairs: >99% reduction in regex work, turning a 30+ second query into
a sub-second one.
serde_json::Map preserves insertion order, so events produced by the same watcher
in the same session produce a consistent JSON key without normalization.
Adds test_categorize_cache_correctness: cache hits on identical data, correct
distinct categories for differing data, and no false cache collisions.
Fixes #629
* style: run cargo fmt to fix CI format check1 parent 2de9533 commit 7429039
2 files changed
Lines changed: 151 additions & 18 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| |||
117 | 118 | | |
118 | 119 | | |
119 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
120 | 126 | | |
121 | | - | |
122 | | - | |
123 | | - | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
124 | 151 | | |
125 | 152 | | |
126 | 153 | | |
127 | 154 | | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | 155 | | |
142 | 156 | | |
143 | 157 | | |
| |||
290 | 304 | | |
291 | 305 | | |
292 | 306 | | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
293 | 364 | | |
294 | 365 | | |
295 | 366 | | |
| |||
0 commit comments