You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Push range counter aggregations down to ClickHouse (#9)
* Push range counter aggregations down to ClickHouse
Range queries of the shape `agg by (...) (F(selector[w]))` over the PostHog
layout now run in one ClickHouse query when F is increase, delta, rate, irate,
idelta or a bare selector. ClickHouse sorts each series into arrays, computes
the MetricsQL rollup at every step and aggregates across series, so one row per
group and step returns to Go instead of every raw sample.
The engine path reads PostHog float samples as one RowBinary payload per
series and iterates float series in place. The query log records the
ClickHouse, evaluation and encoding time of every request.
Generated-By: PostHog Desktop
Task-Id: 8ac0a9e0-3731-4a96-8538-5537a08cbefc
* Rewrite range pushdown comments
Generated-By: PostHog Desktop
Task-Id: e87db87e-6854-4575-9551-c11cc5ecd4d1
* feat(metrics): push offset range aggregations down to ClickHouse
A selector offset shifts the steps ClickHouse evaluates and the output
timestamps stay unshifted. An offset on the aggregate, which the MetricsQL
rewrite turns into a default_rollup over a subquery, evaluates the aggregate
on the epoch-aligned step grid that covers the shifted range; Go then reads
the latest grid point within the MetricsQL window before each shifted step,
as the engine does for the subquery.
Generated-By: PostHog Desktop
Task-Id: 212188a2-7ee4-44ee-b2ae-581e72136474
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(metrics): read PostHog samples as typed delta arrays
The engine path read one RowBinary string per series. An opaque string of
raw 8-byte integers and doubles compresses poorly: 12 MiB with LZ4 for a
24-hour read of 2.2M samples. The same samples as typed arrays of timestamp
deltas and values move 1.5 MiB, and the ClickHouse statement no longer waits
on the socket.
Generated-By: PostHog Desktop
Task-Id: 212188a2-7ee4-44ee-b2ae-581e72136474
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* perf(metrics): compress ClickHouse traffic with ZSTD
CH_COMPRESSION selects the codec for native protocol blocks in both
directions and defaults to zstd. On a 24-hour read of 1974 series the
sample arrays moved 0.8 MiB instead of 1.5 MiB with LZ4 at the same
ClickHouse CPU; lz4 and none stay available.
Generated-By: PostHog Desktop
Task-Id: 212188a2-7ee4-44ee-b2ae-581e72136474
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>