Skip to content

fix(charts): route property-MV charts to anon-inclusive v2 (Jul-1 cutoff), else events - #412

Open
ayushjhanwar-png wants to merge 1 commit into
mainfrom
feat/property-mv-v2-routing
Open

fix(charts): route property-MV charts to anon-inclusive v2 (Jul-1 cutoff), else events#412
ayushjhanwar-png wants to merge 1 commit into
mainfrom
feat/property-mv-v2-routing

Conversation

@ayushjhanwar-png

@ayushjhanwar-png ayushjhanwar-png commented Aug 8, 2026

Copy link
Copy Markdown

Problem

Property-filter/breakdown charts (the canUsePropertyMVgetChartSqlFromPropertyMV path) read v1 profile_event_property_summary_mv, which carries a profile_id != device_id filter — so it counts identified users only and silently drops anonymous events.

Validated on prod (appOpen + $referring_domain=www.google.com, Aug 7):

source count
v1 property MV / dashboard ~131
raw events (all users) 821
v2 property MV 821 ✅ exact

→ v1 undercounts ~5–7× for anonymous-heavy web traffic.

Why v2 property MV (not events_v2)

profile_event_property_summary_v2:

  • No anon filter → matches raw events exactly.
  • Holds every property (property_key/property_value for all of them) — no materialized column needed, unlike events_v2.
  • Pre-aggregated → appOpen breakdown by $referring_domain over the full 5-week window (Jul 1→Aug 8, top-50/day) = 0.21s / 758K rows.
  • Data from 2026-07-01.

Change

  • getPropertyMvTableForRange(startDate) (mirrors getEventsTableForRange): returns v2 when startDate >= PROPERTY_MV_V2_MIN_DATE, else v1. Used for both FROMs in getChartSqlFromPropertyMV.
  • canUsePropertyMV gate: when PROPERTY_MV_V2_MIN_DATE is set, the cutoff becomes the v2 window — ranges before it return false and fall through to the events_v2/events path (correct) instead of the undercounting v1 MV. Env unset = legacy v1 behavior, byte-identical.
  • Logs [property-mv-routing] -> <table>.

Routing result (with PROPERTY_MV_V2_MIN_DATE=2026-07-01)

range start table correctness
≥ Jul 1 v2 property MV anon-inclusive ✅
< Jul 1 events_v2 (in window) → events v1 anon-inclusive ✅ (slower)

Follow-up (not here)

To make pre-Jul-1 ranges correct+fast too, backfill the v2 property MV (migration 22) toward June/May and lower PROPERTY_MV_V2_MIN_DATE.

Enable on prod via openpanel-config PROPERTY_MV_V2_MIN_DATE="2026-07-01 00:00:00" after this deploys.

Summary by CodeRabbit

  • New Features
    • Added support for routing property summary queries to the newer data source for eligible date ranges.
    • Preserved fallback to the existing data source when the configured cutoff is not met or unavailable.
    • Chart queries now automatically use the appropriate data source based on the selected date range.

…lse events

The property-MV chart path read v1 profile_event_property_summary_mv, which has a
profile_id != device_id filter -> counts identified users ONLY, silently dropping
anonymous events. For web appOpen by $referring_domain this undercounts ~5-7x
(validated: Aug 7 appOpen+google v1=131 vs raw=821).

v2 profile_event_property_summary_v2 has NO such filter (anon-inclusive), holds ALL
properties (not just materialized ones), is pre-aggregated (appOpen breakdown over
5 weeks = 0.21s / 758K rows), and has data from 2026-07-01.

- getPropertyMvTableForRange(startDate): v2 when startDate >= PROPERTY_MV_V2_MIN_DATE
  (mirrors getEventsTableForRange). Both FROMs in getChartSqlFromPropertyMV use it.
- canUsePropertyMV gate: when the env is set, use the v2 cutoff -> ranges before it
  return false and fall THROUGH to the events_v2/events path (correct) instead of the
  undercounting v1 MV. Env unset = legacy (v1) unchanged.

Logs [property-mv-routing] -> <table>. Extend correctness before Jul 1 by backfilling
the v2 property MV further and lowering the cutoff.
@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a59ba294-78a9-4500-a82a-85da7973be82

📥 Commits

Reviewing files that changed from the base of the PR and between edd60a6 and 47c3905.

📒 Files selected for processing (2)
  • packages/db/src/clickhouse/client.ts
  • packages/db/src/services/chart.service.ts

📝 Walkthrough

Walkthrough

The ClickHouse client now routes property-summary materialized-view queries between v1 and v2 tables by date. Chart SQL uses this routing for the top-values CTE and main query while preserving legacy cutoff behavior when no v2 cutoff is configured.

Changes

Property MV routing

Layer / File(s) Summary
Property MV table selector
packages/db/src/clickhouse/client.ts
Adds the v2 property-summary table name and exports a selector that chooses v1 or v2 from PROPERTY_MV_V2_MIN_DATE and the query start date.
Chart query integration
packages/db/src/services/chart.service.ts
Uses the configured v2 cutoff for eligibility and selects the routed table in both property-MV query paths.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ChartService
  participant PropertyMVRouter
  participant ClickHouse
  ChartService->>PropertyMVRouter: Select table for query start date
  PropertyMVRouter-->>ChartService: Return v1 or v2 table
  ChartService->>ClickHouse: Run top-values and main queries
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: routing charts to the anonymous-inclusive v2 property MV after the July 1 cutoff and using events otherwise.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/property-mv-v2-routing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant