chore(clickhouse): bring the ai_events satellite under declarative HCL#67281
Merged
Conversation
Contributor
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
orian
force-pushed
the
pawel/chore/ai-events-hcl-schema
branch
from
8eb6b88 to
1ef4e3b
Compare
rorylshanks
approved these changes
Jul 1, 2026
orian
force-pushed
the
pawel/chore/ai-events-hcl-schema
branch
4 times, most recently
from
9c79675 to
33b6eac
Compare
Contributor
|
Reviews (2): Last reviewed commit: "chore(clickhouse): bring the AI_EVENTS s..." | Re-trigger Greptile |
Model the AI_EVENTS role in posthog/clickhouse/hcl/ (the third managed role after OPS and LOGS) and wire it into both the offline guard and the live gate. The ai_events column set is identical across every table. Factor it into two abstract bases in roles/ai_events/shared and extend them: - _ai_events_columns (columns only) -> the distributed ai_events reader (local); - _ai_events_data (extends _ai_events_columns, adds order_by/partition/ttl + indexes) -> the data tables sharded_ai_events (local) and ai_events (US/EU), which then only set their own engine (zoo_path differs). Env layering follows the actual topology: - local/hobby run the MSK ingest variant (kafka_ai_events_json + ai_events_json_mv) with sharded_ai_events + a distributed ai_events reader; - US/EU run WarpStream (kafka_ai_events_json_ws + ai_events_json_ws_mv, MSK dropped by migration 0248) writing into a single ai_events data table (US and EU identical; the earlier per-EU sharded_ai_events has been dropped in prod); - person / person_distinct_id2 are env-uniform Distributed shims (0240), also in roles/ai_events/shared; dev has only the top-level shared objects. - nodes: local/dev/prod-us/prod-eu ai_events lines. - dump-live.sh / check-live.sh: add ai_events (:9100) so the convergence gate introspects and diffs it too. - check.sh: skip validation of the person / cross-cluster Distributed proxies. Goldens are generated from the layers (abstract bases resolve away, so the flat schema is unchanged). Verified against a live multinode ai_events node: dump-live + check-live report no drift for local/ai_events; prod goldens are authored from the captured prod dumps.
orian
force-pushed
the
pawel/chore/ai-events-hcl-schema
branch
from
33b6eac to
99871b3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The declarative ClickHouse HCL (
posthog/clickhouse/hcl/) manages the OPS and LOGS satellite roles, and the live convergence gate verifies migrations reproduce them.ai_events(LLM analytics) is the next satellite — this brings it under HCL and under the gate.Changes
ai_eventsrole:roles/ai_events/{shared,local,prod}layers,nodeslines, per-env goldens + build SQL.zoo_pathdiffer), so it's anabstract_ai_events_database thatsharded_ai_events(local) andai_events(prod)extend.ai_events(:9100) intodump-live.sh/check-live.shso the multinode convergence gate introspects and diffs it too.Detail:
kafka_ai_events_json+ai_events_json_mv) with a sharded data table + distributed reader; US/EU run WarpStream (kafka_ai_events_json_ws+ai_events_json_ws_mv, MSK dropped by migration0248) writing into a singleai_eventsdata table (US and EU identical — the mid-migration per-EUsharded_ai_eventshas since been dropped in prod).person/person_distinct_id2are env-uniform Distributed shims (0240), modeled inroles/ai_events/shared;check.shskips validation of these cross-cluster proxies.devhas only the top-level shared objects.How did you test this code?
I'm an agent. Automated checks I ran:
check.shgreen — every ai_events golden reproduces from its layers, cross-object refs validate,sql/fresh (11 goldens: ops 4 + logs 3 + ai_events 4).dump-live.sh+check-live.sh(enforcing) —local/ai_events: no differencesagainst the live node (person shims and MSK ingest matched).git diffongolden/+sql/is empty.Automatic notifications
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Authored with Claude Code, following the OPS/LOGS pilot pattern. Built on the merged convergence gate (#67134). Key decisions surfaced during review: model the real current prod (EU dropped its transitional
sharded_ai_events, so prod-us == prod-eu now) rather than the mid-migration dump; keepperson/person_distinct_id2modeled rather than excluded; and DRY the data table via anabstractbase +extend. Layers were derived from a live local introspection + the prod dumps.