feat(insights): customize trends compare periods as one base series - #70406
Conversation
Result customization keys for trends and stickiness included the compare_label, so color and hidden state applied to each compare period independently. Customizations now key on the base series: current and previous share one entry, previous keeps its derived 50% dim, and customizations survive toggling compare on/off. Schema migration 0003 collapses stored per-period entries into base entries. Generated-By: PostHog Code Task-Id: 9b780076-301c-453d-876d-db0be4ace588
…-base-series-customizations
🤖 CI report
|
| File | Size | Δ vs base |
|---|---|---|
posthog-app/_parent/products/feature_flags/frontend/staff/FeatureFlagsStaffToolsScene.js |
12.1 KiB | 🔺 +1.6 KiB (+14.7%) |
render-query/src/render-query/render-query.js |
21.21 MiB | 🔺 +1.5 KiB (+0.0%) |
Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report
✅ Eager graph — within budget
How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.21 MiB · 22 files | 🔺 +43 B (+0.0%) | ███░░░░░░░ 28.1% of 4.29 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
8.11 MiB · 2,973 files | 🔺 +2.0 KiB (+0.0%) | █████████░ 87.6% of 9.25 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 789 B | src/scenes/ChunkLoadErrorBoundary.tsx |
| 668 B | src/index.tsx |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 278.6 KiB | ../node_modules/.pnpm/posthog-js@1.399.4/node_modules/posthog-js/dist/rrweb.js |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 234.9 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 221.8 KiB | ../node_modules/.pnpm/posthog-js@1.399.4/node_modules/posthog-js/dist/module.js |
| 164.0 KiB | src/queries/validators.js |
| 154.3 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 105.9 KiB | src/lib/api.ts |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 90.6 KiB | ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
⚠️ Dist folder size — 🔺 +107.0 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1278.78 MiB · 🔺 +107.0 KiB (+0.0%)
⚠️ Backend coverage — 99.0% of changed backend lines covered — 1 uncovered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ████████████████████ 99.0% (164 / 165)
| File | Patch | Uncovered changed lines |
|---|---|---|
posthog/schema_migrations/0003_trends_result_customizations_compare.py |
99.0% | 85 |
🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 29281258210 -n patch-coverage), or the coverage-data block at the end of this comment.
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
platform_features |
██░░░░░░░░░░░░░░░░░░ 12.1% |
7 / 58 |
batch_exports |
████████░░░░░░░░░░░░ 39.5% |
8,345 / 21,104 |
demo |
███████████░░░░░░░░░ 55.2% |
1,436 / 2,601 |
warehouse_sources_queue |
████████████░░░░░░░░ 59.2% |
148 / 250 |
tasks |
█████████████░░░░░░░ 66.7% |
24,376 / 36,570 |
data_tools |
██████████████░░░░░░ 70.0% |
63 / 90 |
ai_gateway |
███████████████░░░░░ 75.0% |
9 / 12 |
data_modeling |
████████████████░░░░ 78.4% |
4,696 / 5,987 |
signals |
████████████████░░░░ 78.5% |
18,070 / 23,028 |
cdp |
████████████████░░░░ 80.6% |
3,105 / 3,851 |
wizard |
████████████████░░░░ 82.5% |
772 / 936 |
cohorts |
█████████████████░░░ 82.9% |
3,048 / 3,675 |
notebooks |
█████████████████░░░ 83.8% |
6,086 / 7,259 |
agent_platform |
█████████████████░░░ 84.1% |
3,095 / 3,678 |
actions |
█████████████████░░░ 86.6% |
717 / 828 |
engineering_analytics |
█████████████████░░░ 87.2% |
4,203 / 4,821 |
product_tours |
█████████████████░░░ 87.5% |
1,266 / 1,447 |
exports |
██████████████████░░ 88.3% |
6,857 / 7,763 |
visual_review |
██████████████████░░ 88.5% |
5,565 / 6,287 |
business_knowledge |
██████████████████░░ 88.5% |
4,400 / 4,969 |
conversations |
██████████████████░░ 88.9% |
15,924 / 17,921 |
mcp_analytics |
██████████████████░░ 89.1% |
2,485 / 2,790 |
dashboards |
██████████████████░░ 89.1% |
5,650 / 6,342 |
error_tracking |
██████████████████░░ 89.2% |
9,473 / 10,615 |
streamlit_apps |
██████████████████░░ 90.4% |
2,499 / 2,764 |
slack_app |
██████████████████░░ 90.6% |
9,460 / 10,444 |
links |
██████████████████░░ 90.6% |
183 / 202 |
marketing_analytics |
██████████████████░░ 90.7% |
11,476 / 12,646 |
product_analytics |
██████████████████░░ 91.2% |
5,621 / 6,164 |
managed_migrations |
██████████████████░░ 91.9% |
908 / 988 |
workflows |
██████████████████░░ 92.0% |
4,795 / 5,210 |
mcp_store |
██████████████████░░ 92.1% |
3,665 / 3,981 |
data_warehouse |
██████████████████░░ 92.1% |
17,282 / 18,765 |
alerts |
██████████████████░░ 92.1% |
3,389 / 3,678 |
web_analytics |
███████████████████░ 92.7% |
13,702 / 14,787 |
notifications |
███████████████████░ 92.7% |
1,026 / 1,107 |
ai_observability |
███████████████████░ 92.7% |
14,670 / 15,822 |
surveys |
███████████████████░ 92.9% |
5,660 / 6,094 |
posthog_ai |
███████████████████░ 93.2% |
1,311 / 1,407 |
tracing |
███████████████████░ 93.2% |
2,423 / 2,599 |
approvals |
███████████████████░ 93.3% |
3,395 / 3,640 |
reminders |
███████████████████░ 93.4% |
468 / 501 |
early_access_features |
███████████████████░ 93.8% |
848 / 904 |
legal_documents |
███████████████████░ 94.1% |
1,568 / 1,667 |
endpoints |
███████████████████░ 94.1% |
8,606 / 9,143 |
messaging |
███████████████████░ 94.3% |
2,366 / 2,508 |
skills |
███████████████████░ 94.4% |
2,819 / 2,987 |
revenue_analytics |
███████████████████░ 94.4% |
3,586 / 3,797 |
data_catalog |
███████████████████░ 94.6% |
895 / 946 |
growth |
███████████████████░ 94.9% |
2,393 / 2,522 |
logs |
███████████████████░ 95.3% |
9,516 / 9,983 |
experiments |
███████████████████░ 95.6% |
24,055 / 25,164 |
replay_vision |
███████████████████░ 95.6% |
12,690 / 13,272 |
feature_flags |
███████████████████░ 95.9% |
14,486 / 15,099 |
warehouse_sources |
███████████████████░ 96.1% |
209,734 / 218,320 |
annotations |
███████████████████░ 96.2% |
732 / 761 |
user_interviews |
███████████████████░ 96.4% |
2,242 / 2,325 |
access_control |
███████████████████░ 96.8% |
849 / 877 |
customer_analytics |
███████████████████░ 97.3% |
7,277 / 7,481 |
analytics_platform |
████████████████████ 98.2% |
2,098 / 2,137 |
metrics |
████████████████████ 98.3% |
2,363 / 2,403 |
live_debugger |
████████████████████ 99.2% |
613 / 618 |
field_notes |
████████████████████ 99.4% |
158 / 159 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
🦔 Hogbox preview · ✅ ready▶ Open the preview
commit |
👀 Auto-assigned reviewersThese soft owners were skipped because they only have minor changes here. Nothing blocks merge, so self-assign if you'd like a look:
Soft owners come from |
|
Reviews (1): Last reviewed commit: "Merge branch 'master' into posthog-code/..." | Re-trigger Greptile |
|
Note 🤖 stamphog reviewed Gates denied this PR (deny-listed migration path, T2-never tier) — it ships a schema migration that rewrites stored insight customization data, which is risky territory requiring human sign-off regardless of the thorough test coverage and addressed review comment.
Gate mechanics and policy version
Updated in place — this replaces 1 earlier stamphog review(s) on this PR. |
Replace direct LATEST_VERSIONS[NodeKind.TrendsQuery] version assignments with the setLatestVersionsOnQuery wrapper, which is the established convention for hardcoded queries and future-proofs nested nodes. Kept explicit version fields where wrapping can't express the expectation: sources whose kind has no version entry (the version leaks from the initial trends source), and a toDispatchActions payload where kea-test-utils compares actions via JSON.stringify, making key order significant. Generated-By: PostHog Code Task-Id: 5ca3d704-7964-4d8b-a945-0eced3f60945
…ion test compare_to without compare: true never enables comparison (both query runners and all frontend surfaces gate on compare being truthy), so migration 0003's value-mode merge must let the unlabelled entry win in that state. Locks in the semantics questioned in review. Generated-By: PostHog Code Task-Id: fccc88d5-7043-45c6-bd62-062ce6cdf44f
|
✅ Visual changes approved by @thmsobrmlr — baseline updated in 2 changed. Changed
|
2 updated Run: 07226b75-89ff-479a-9f37-976265122ab6 Co-authored-by: thmsobrmlr <1851359+thmsobrmlr@users.noreply.github.com>
Bumping TrendsQuery/StickinessQuery to schema version 3 made the v2 query fixtures in these tests route through the mocked `query/upgrade` endpoint, which returned 500 for anything but retention, failing 6 tests. Mirror the backend here: migration 0003 is a no-op on fixtures without result customizations, so the mock now echoes trends/stickiness queries back with the version bumped to latest, and the expected outputs track `LATEST_VERSIONS`. Generated-By: PostHog Code Task-Id: 4b5ef8c3-42db-441f-9a05-f179a9d61290
…odes (#70799) ## TL;DR (eli5) Some saved insights have chart series stored without the `kind` label that says what type of thing each series is (an event, an action, a warehouse table). The code used to guess the type silently, but we recently made it require the label ([#70552](#70552), [#70688](#70688)) - so those insights would start erroring when viewed. This PR adds a migration that writes the correct label back onto those stored insights, both when they're read and via the background backfill job. ## Problem `FunnelsQuery.series` was made a discriminated union on `kind` in #70552, and #70688 extends that to the remaining series unions (Trends, Stickiness, Lifecycle, CalendarHeatmap) and funnel exclusions. Discriminated unions require the tag in the raw input, but stored insights contain series items saved without `kind`: the insight API validated queries with the old smart union (which coerced tag-less dicts to `EventsNode` via its default) and then persisted the raw request dict, not the validated dump. A small number of saved insights across multiple teams carry such items, some recently created and viewed - they would fail with `union_tag_not_found` on view once the discriminators are live. ## Changes - New query schema migration `0004_entity_node_kind`: stamps `kind` onto dict entity nodes missing it in `series` (Trends, Stickiness, Lifecycle, CalendarHeatmap, Funnels) and `funnelsFilter.exclusions` (Funnels), mirroring the old smart-union coercion: `table_name`-shaped items get the data warehouse node kind, id-only items get `ActionsNode`, everything else (event-shaped, empty) gets `EventsNode`. Non-dict items (e.g. stray template placeholder strings) are left untouched - they were invalid before and stay invalid. - This runs at read time via `upgrade()` (so affected insights render correctly immediately on deploy) and is backfilled into stored insights by the existing `upgrade-queries` Temporal schedule. - Regenerated `frontend/src/queries/latest-versions.json` (FunnelsQuery/LifecycleQuery/CalendarHeatmapQuery now versioned at 2, TrendsQuery/StickinessQuery bumped to 4). - Updated two jest suites that assert exact version stamps (`insightNavLogic.test.ts`, notebook `migrate.test.ts`), following the pattern from #70406. > [!NOTE] > The version bump means the Temporal backfill will rewrite all stored Funnels/Lifecycle/CalendarHeatmap/Trends/Stickiness insights below the new latest version (same mechanics as previous schema migrations, just a wider sweep). ## How did you test this code? - New `posthog/schema_migrations/test/test_0004_entity_node_kind.py`: parameterized stamping cases per query kind and item shape, pass-through cases (tagged items, string placeholders, null/missing series), exclusions stamping, and end-to-end `upgrade()` -> `model_validate` round-trips for stored-shape Funnels and Trends queries. These catch the regression where a stored kind-less insight fails validation - no existing test covered it. - Full `posthog/schema_migrations/test/` suite passes locally (75 tests), including discovery linearity and the frontend versions-file sync check. - Affected jest suites pass locally: `src/queries`, `src/scenes/insights`, `src/scenes/funnels`, notebook migrations, plus every other suite referencing the newly versioned query kinds (experiments, dashboards, trends, sharing, alerts, customer analytics). - Not run locally: DB-backed suites (no database in the sandbox) - covered by CI. ## Automatic notifications - [ ] Publish to changelog? - [ ] Alert Sales and Marketing teams? ## Docs update Not needed - internal storage migration, no user-facing behavior or API change. ## 🤖 Agent context **Autonomy:** Human-driven (agent-assisted) Authored with PostHog Code (Claude). Skills invoked: /writing-tests. Follow-up to #70688: querying the warehouse-synced insights tables confirmed stored insights with kind-less series exist (a small number, some recently created and viewed), because the insight API persisted the raw request dict after validating with the pre-discriminator smart union. Chose the existing schema-migrations system over a one-off Django data migration since it applies at read time and backfills via the established Temporal sweep. Considered and rejected re-running `migrate_mock_insights.py` - its output fights prettier formatting and nothing asserts fixture versions. --- *Created with [PostHog Code](https://posthog.com/code?ref=pr)*
TL;DR (eli5)
When a trends chart shows "this week vs last week", each line used to have its own color picker and show/hide checkbox — so you could paint or hide "last week" separately from "this week". That was confusing and inconsistent: now each series has one setting, and both of its lines follow it (the previous period stays automatically faded). Old insights with per-period settings are converted over by a schema migration.
Problem
With compare against previous period enabled, trends and stickiness result customizations were keyed per compare period (
compare_labelwas part of the customization key). That meant:Changes
getTrendDatasetKeyno longer includescompare_label— the base key is identical to the compare-off format, so customizations survive toggling compare.colorIndex) instead of the rawseriesIndex, which is offset for previous-period rows.getTrendDatasetPositionpreferscolorIndex(shared by current/previous pairs) overseriesIndex, making by-rank customizations base-series-level too. The value-mode special case ingetTrendResultCustomizationColorTokencollapses as a result.0003(targetsTrendsQuery/StickinessQueryv2) collapses stored per-period entries:compare_labelfrom keys. On conflict, the entry matching the stored compare state wins ("current" when compare is on, the unlabelled one otherwise). "previous"-only entries are dropped, since carrying e.g.hiddenover would hide the current period too.frontend/src/queries/latest-versions.jsonregenerated (TrendsQuery/StickinessQuery→ 3); test fixtures that hardcodedversion: 2now referenceLATEST_VERSIONS.Note
Per-period hiding is intentionally no longer possible: hiding a series hides both periods, matching funnels. Rollout has no visible gap — insight serializers upgrade queries on read (#69926), and the 6-hourly backfill persists the rewrite.
How did you test this code?
posthog/schema_migrations/test/test_0003_trends_result_customizations_compare.py): value-mode merge matrix (compare on/off × base/current/previous conflicts — catches wrong-winner regressions), key byte-preservation incl. non-ASCII breakdown values (catchesensure_asciimismatches with JSJSON.stringify), position remap with conflict/formula-count/breakdown-skip cases, stickiness coverage, and no-op guards for new-shape queries.getTrendDatasetKeyjest cases: current/previous now assert the shared base key (catches reintroduction ofcompare_label), plus a formula-with-compare case locking the phantom series-name fix (formula3for formula 1's previous period).posthog/schema_migrations/suite (56 passed), temporal upgrade-workflow tests (4 passed), jest overscenes/insights,scenes/trends,scenes/dashboard,queries,products/product_analytics,scenes/data-management(2400+ tests passed), frontend typecheck (only pre-existing failures in untouchedlayout//products/workflowsfiles),hogli ci:preflight --fix.Automatic notifications
Docs update
Not needed — behavior change is in-product UI semantics without documented workflow references.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
posthog/schema_migrations/migrate_mock_insights.py— it produced a 53-file pure-reformat diff (its oxfmt step is broken); reverted, verified no fixture contains affected query nodes, and reported the script bug via devex feedback.Created with PostHog Code