chore(web_analytics): move models to product app#59223
Merged
Merged
Conversation
Move WebAnalyticsFilterPreset, SavedHeatmap, and HeatmapSnapshot from posthog/models/ into products/web_analytics/backend/models/. State-only Django migrations with SeparateDatabaseAndState — no DB changes, tables stay in place via explicit db_table declarations.
Contributor
|
🎭 Playwright didn't run on this PR — your changes touch code that could affect E2E behavior, but Playwright is opt-in via label now to keep CI cost down. Add the Most PRs don't need this. Real regressions still get caught on master and fix-forward. |
Contributor
Migration SQL ChangesHey 👋, we've detected some migrations on this PR. Here's the SQL output for each migration, make sure they make sense:
|
Contributor
🔍 Migration Risk AnalysisWe've analyzed your migrations for potential risks. Summary: 1 Safe | 1 Needs Review | 0 Blocked
|
Co-locate web_analytics product code into the product backend: - API: web_analytics_filter_preset.py, heatmaps_api.py, heatmaps_utils.py - HogQL query runners: entire posthog/hogql_queries/web_analytics/ tree - Celery task: heatmap_screenshot.py and its test - Heatmaps API tests + snapshots posthog/heatmaps/sql.py stays — shared ClickHouse schema used by CH migrations. Imports rewritten with the product-model-migration skill's import_rewriter across 58 files. Added web_analytics/backend/** to the ANN exemption in products/ruff.toml to match other migrated products.
…ytics-models-to-product # Conflicts: # posthog/migrations/max_migration.txt
Contributor
|
⏭️ Skipped snapshot commit because branch advanced to The new commit will trigger its own snapshot update workflow. If you expected this workflow to succeed: This can happen due to concurrent commits. To get a fresh workflow run, either:
|
The import_rewriter only handles import statements — it doesn't see string paths in unittest.mock.patch() decorators or docstring/comment references. These caused 140 test failures in CI: "module 'posthog.hogql_queries' has no attribute 'web_analytics'". Updated all string references from posthog.hogql_queries.web_analytics, posthog.heatmaps.heatmaps_api, and posthog.tasks.heatmap_screenshot to their new products.web_analytics.backend.* paths.
…ytics-models-to-product # Conflicts: # posthog/migrations/max_migration.txt
…to-product' into chore/move-web-analytics-models-to-product
Contributor
MCP UI Apps size report
|
Contributor
|
Size Change: +15.5 kB (+0.01%) Total Size: 119 MB 📦 View Changed
ℹ️ View Unchanged
|
Contributor
Query snapshots: Backend query snapshots updatedChanges: 2 snapshots (2 modified, 0 added, 0 deleted) What this means:
Next steps:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b86c26c776
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
11
to
14
| exporter, | ||
| feature_flags, | ||
| health_checks, | ||
| heatmap_screenshot, | ||
| hog_flows, |
There was a problem hiding this comment.
Removing heatmap_screenshot from the posthog.tasks import list without adding an equivalent re-export under products.web_analytics.backend.tasks leaves generate_heatmap_screenshot undiscovered by workers: app.autodiscover_tasks() imports <app>.tasks, but the new tasks/__init__.py is empty and never imports tasks.heatmap_screenshot. In production, calls to .delay() will enqueue products.web_analytics.backend.tasks.heatmap_screenshot.generate_heatmap_screenshot, and workers that haven't imported that module will treat it as an unregistered task, so screenshot jobs can stay stuck in processing.
Useful? React with 👍 / 👎.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR continues PostHog’s product-isolation work by moving Web Analytics Django models (and a sizable chunk of related web analytics query/API/task code) out of monolithic posthog/* into products/web_analytics/backend/*, while keeping existing DB tables in place via state-only migrations and a ContentType relabel migration.
Changes:
- Introduces the
web_analyticsDjango app (WebAnalyticsConfig), adds state-only migrations to “move” models, and migrates ContentTypes fromposthog→web_analytics. - Relocates Web Analytics models, heatmaps API/task code, and updates imports/routes across the backend (plus generated frontend API clients).
- Moves/duplicates a large set of web analytics HogQL query runners/utilities and adds/updates extensive test coverage and snapshots.
Reviewed changes
Copilot reviewed 79 out of 116 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| products/web_analytics/mcp/tools.yaml | Adds (disabled) MCP tool definitions for heatmaps/saved/presets endpoints. |
| products/web_analytics/frontend/generated/api.zod.ts | Adds Zod schemas for web analytics filter presets + heatmap saved endpoints. |
| products/web_analytics/frontend/generated/api.ts | Adds generated TS client functions for new web analytics endpoints. |
| products/web_analytics/frontend/generated/api.schemas.ts | Adds generated TS types for presets/heatmaps saved endpoints. |
| products/web_analytics/dags/web_preaggregated_asset_checks.py | Updates WebOverviewQueryRunner import path into product app. |
| products/web_analytics/backend/weekly_digest.py | Updates HogQL query runner imports into product app. |
| products/web_analytics/backend/test/test_heatmaps_api.py | Adds tests for legacy heatmap endpoint behavior. |
| products/web_analytics/backend/test/test_heatmap_screenshots.py | Updates model imports + patch targets to product app modules. |
| products/web_analytics/backend/tasks/test/test_heatmap_screenshot.py | Updates imports/patch targets for moved heatmap screenshot task + models. |
| products/web_analytics/backend/tasks/test/init.py | Test package init. |
| products/web_analytics/backend/tasks/heatmap_screenshot.py | Moves/updates heatmap screenshot task to product app, updates imports. |
| products/web_analytics/backend/tasks/init.py | Adds tasks package for the product app. |
| products/web_analytics/backend/models/web_analytics_filter_preset.py | Moves preset model, updates FK string refs, sets db_table. |
| products/web_analytics/backend/models/heatmap_saved.py | Moves heatmap models, updates FK string refs, sets db_table. |
| products/web_analytics/backend/models/init.py | Adds product-app model barrel exports. |
| products/web_analytics/backend/migrations/max_migration.txt | Adds product-app max migration marker. |
| products/web_analytics/backend/migrations/0001_initial.py | Adds state-only initial migration for moved models. |
| products/web_analytics/backend/migrations/init.py | Migration package init. |
| products/web_analytics/backend/hogql_queries/web_vitals_path_breakdown.py | Updates imports to product-app query runner base. |
| products/web_analytics/backend/hogql_queries/web_trends_query_runner.py | Updates imports to product-app query runner base/builders. |
| products/web_analytics/backend/hogql_queries/web_overview.py | Updates imports to product-app query runner base/builders. |
| products/web_analytics/backend/hogql_queries/web_overview_pre_aggregated.py | Updates imports to product-app pre-aggregated builder/properties. |
| products/web_analytics/backend/hogql_queries/web_goals.py | Updates imports to product-app query runner base. |
| products/web_analytics/backend/hogql_queries/web_analytics_query_runner.py | Updates imports for metrics + traffic type helpers into product app. |
| products/web_analytics/backend/hogql_queries/trends_pre_aggregated_query_builder.py | Updates imports to product-app pre-aggregated modules. |
| products/web_analytics/backend/hogql_queries/traffic_type.py | Updates BOT_DEFINITIONS import to product app. |
| products/web_analytics/backend/hogql_queries/test/web_preaggregated_test_base.py | Adds shared test base for pre-aggregated web analytics tests. |
| products/web_analytics/backend/hogql_queries/test/test_web_vitals_path_breakdown.py | Updates test imports for moved query runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_trends_query_runner.py | Updates test imports for moved query runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_traffic_type.py | Updates test imports for moved traffic_type helpers. |
| products/web_analytics/backend/hogql_queries/test/test_web_stats_table.py | Updates test imports + patch targets for moved stats table. |
| products/web_analytics/backend/hogql_queries/test/test_web_stats_table_pre_aggregated_conversions.py | Updates test imports for moved modules/builders. |
| products/web_analytics/backend/hogql_queries/test/test_web_stats_pre_aggregated.py | Updates test imports for moved modules/builders. |
| products/web_analytics/backend/hogql_queries/test/test_web_stats_pre_aggregated_channel_types.py | Updates test imports for moved modules/builders. |
| products/web_analytics/backend/hogql_queries/test/test_web_overview.py | Updates test imports for moved modules/builders. |
| products/web_analytics/backend/hogql_queries/test/test_web_overview_state_aggregations.py | Updates test imports for moved web_overview runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_overview_dashboard_filters.py | Updates test imports for moved web_overview runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_goals.py | Updates test imports for moved web_goals runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_bounces_pre_aggregated.py | Updates test imports for moved modules. |
| products/web_analytics/backend/hogql_queries/test/test_web_analytics_rbac.py | Updates test imports for moved web_overview runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_analytics_query_runner.py | Updates test imports + patch targets for moved query runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_analytics_metrics.py | Updates test imports + patch targets for moved metrics/query runner. |
| products/web_analytics/backend/hogql_queries/test/test_web_analytics_cohort_filter.py | Updates test imports for moved query runners. |
| products/web_analytics/backend/hogql_queries/test/test_trends_pre_aggregated_query_builder.py | Updates test imports for moved builder. |
| products/web_analytics/backend/hogql_queries/test/test_traffic_type_snapshot.py | Adds snapshot tests for traffic type HogQL helpers. |
| products/web_analytics/backend/hogql_queries/test/test_session_attribution_explorer_query_runner.py | Updates test imports for moved session attribution runner. |
| products/web_analytics/backend/hogql_queries/test/test_sample_web_analytics_queries.py | Updates test imports for moved query runners. |
| products/web_analytics/backend/hogql_queries/test/test_page_url_search_query_runner.py | Updates test imports for moved page URL search runner. |
| products/web_analytics/backend/hogql_queries/test/test_page_reports_avg_time.py | Adds tests for p90 time-on-page reporting behavior. |
| products/web_analytics/backend/hogql_queries/test/test_notable_changes.py | Updates test imports for moved notable changes runner. |
| products/web_analytics/backend/hogql_queries/test/test_external_clicks_table.py | Updates test imports for moved external clicks runner. |
| products/web_analytics/backend/hogql_queries/test/test_events_prefilter.py | Adds/updates tests for events prefilter transformer/paginator. |
| products/web_analytics/backend/hogql_queries/test/test_bot_definitions.py | Updates test import for moved bot definitions. |
| products/web_analytics/backend/hogql_queries/test/snapshots/test_web_vitals_path_breakdown.ambr | Adds snapshot baseline for web vitals path breakdown SQL. |
| products/web_analytics/backend/hogql_queries/test/snapshots/test_active_hours_heatmap_query_runner.ambr | Adds snapshot baseline for active hours heatmap SQL. |
| products/web_analytics/backend/hogql_queries/test_web_analytics_tables_version.py | Updates query runner imports for table-version tests. |
| products/web_analytics/backend/hogql_queries/stats_table.py | Updates imports and adds events prefilter paginator wiring. |
| products/web_analytics/backend/hogql_queries/stats_table_strategies.py | Updates imports for stats table query constants/typing. |
| products/web_analytics/backend/hogql_queries/stats_table_pre_aggregated.py | Updates imports for moved pre-aggregated property mapping/builder. |
| products/web_analytics/backend/hogql_queries/session_attribution_explorer_query_runner.py | Adds moved session attribution explorer query runner. |
| products/web_analytics/backend/hogql_queries/query_constants/stats_table_queries.py | Adds SQL templates for stats table strategies. |
| products/web_analytics/backend/hogql_queries/query_constants/init.py | Package init for query constants. |
| products/web_analytics/backend/hogql_queries/pre_aggregated/query_builder.py | Updates imports for moved pre-aggregated property transformer. |
| products/web_analytics/backend/hogql_queries/pre_aggregated/property_transformer.py | Adds moved property transformer + channel type replacer. |
| products/web_analytics/backend/hogql_queries/pre_aggregated/properties.py | Adds moved pre-aggregated property mappings. |
| products/web_analytics/backend/hogql_queries/page_url_search_query_runner.py | Updates imports for moved query runner base. |
| products/web_analytics/backend/hogql_queries/notable_changes.py | Updates imports for moved pre-aggregated modules/query runner base. |
| products/web_analytics/backend/hogql_queries/metrics.py | Adds moved Prometheus metrics for web analytics query runners. |
| products/web_analytics/backend/hogql_queries/external/test/test_summary_query_runner.py | Updates imports/patch targets for moved external summary runner. |
| products/web_analytics/backend/hogql_queries/external/test/init.py | Test package init. |
| products/web_analytics/backend/hogql_queries/external/summary_query_runner.py | Adds external (S3/chdb) web analytics summary query runner. |
| products/web_analytics/backend/hogql_queries/external/init.py | Package init. |
| products/web_analytics/backend/hogql_queries/external_clicks.py | Updates imports for moved query runner base. |
| products/web_analytics/backend/hogql_queries/events_prefilter.py | Adds events prefilter transformer + specialized paginator. |
| products/web_analytics/backend/hogql_queries/ctes.py | Adds SQL CTE constants used by web analytics queries. |
| products/web_analytics/backend/hogql_queries/bot_ua_fixtures.py | Adds UA fixture lists for bot/traffic-type testing + demo data. |
| products/web_analytics/backend/hogql_queries/bot_definitions.py | Adds bot definition mapping for HogQL traffic type functions. |
| products/web_analytics/backend/hogql_queries/init.py | Package init. |
| products/web_analytics/backend/apps.py | Adds WebAnalyticsConfig app config (name/label). |
| products/web_analytics/backend/api/web_analytics_filter_preset.py | Updates model import to product app. |
| products/web_analytics/backend/api/heatmaps_utils.py | Adds DEFAULT_TARGET_WIDTHS constant under product app. |
| products/web_analytics/backend/api/heatmaps_api.py | Updates imports to product models/task + constants. |
| products/web_analytics/backend/api/api.py | Adds WebAnalytics weekly digest endpoint ViewSet. |
| products/web_analytics/backend/api/init.py | Exports WebAnalyticsViewSet. |
| products/ruff.toml | Exempts web_analytics backend from ANN typing rule enforcement. |
| posthog/test/base.py | Updates patching hooks for moved web analytics query modules. |
| posthog/tasks/test/snapshots/test_task_registration.ambr | Updates snapshot (removes moved heatmap screenshot task from posthog.*). |
| posthog/tasks/init.py | Stops importing heatmap_screenshot from monolithic tasks package. |
| posthog/settings/web.py | Registers products.web_analytics.backend app config. |
| posthog/models/web_preaggregated/test_web_preaggregated_sql.py | Updates imports for moved web preaggregated test base. |
| posthog/models/web_preaggregated/test_web_pre_aggregated_timezones.py | Updates imports + patch targets for moved stats table. |
| posthog/models/init.py | Removes re-exports for moved heatmap/preset models. |
| posthog/migrations/max_migration.txt | Bumps max migration pointer to 1168. |
| posthog/migrations/1168_migrate_web_analytics_models.py | Adds state-only model removal + ContentType relabel migration. |
| posthog/management/commands/generate_bot_demo_data.py | Updates UA fixtures import to product app. |
| posthog/hogql/transforms/test/test_preaggregated_table_transformation.py | Updates imports for moved pre-aggregated property mappings. |
| posthog/hogql/transforms/preaggregated_table_transformation.py | Updates imports for moved pre-aggregated property mappings. |
| posthog/hogql/functions/traffic_type.py | Updates bot definitions import path + docstring reference. |
| posthog/hogql/functions/test/test_traffic_type_real_ua.py | Updates imports for moved bot definitions/fixtures. |
| posthog/hogql/functions/test/test_traffic_type_functions.py | Updates import for moved bot definitions. |
| posthog/hogql/database/schema/sessions_v3.py | Updates comment reference to moved session attribution runner. |
| posthog/hogql/database/schema/sessions_v2.py | Updates comment reference to moved session attribution runner. |
| posthog/hogql/database/schema/sessions_v1.py | Updates comment reference to moved session attribution runner. |
| posthog/hogql_queries/query_runner.py | Repoints web analytics query runner imports into product app. |
| posthog/api/init.py | Repoints heatmaps + filter preset ViewSet imports into product app. |
| livestream/bot/generate_definitions.py | Updates bot definitions import path for generator script. |
Comments suppressed due to low confidence (1)
products/web_analytics/backend/tasks/heatmap_screenshot.py:18
- Since this task module moved under
products.web_analytics.backend.tasks.*, the Celery task name will change (default name derives from the module path). That can break execution of any in-flight queued messages still referencing the old name (posthog.tasks.heatmap_screenshot.generate_heatmap_screenshot). Consider explicitly settingname=on the@shared_taskdecorator (or keeping a thin compatibility module at the old import path).
Also ensure this module is imported during Celery autodiscovery: app.autodiscover_tasks() will import products.web_analytics.backend.tasks, but if that package doesn’t import heatmap_screenshot, the task won’t be registered on workers.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
lricoy
approved these changes
May 21, 2026
…ytics-models-to-product # Conflicts: # posthog/migrations/max_migration.txt # posthog/settings/web.py # products/web_analytics/backend/hogql_queries/test/test_web_analytics_cohort_filter.py # products/web_analytics/backend/hogql_queries/test/test_web_goals.py # products/web_analytics/backend/hogql_queries/test/test_web_overview.py # products/web_analytics/backend/hogql_queries/test/test_web_overview_lazy_precompute.py # products/web_analytics/backend/hogql_queries/test/test_web_stats_table.py # products/web_analytics/backend/hogql_queries/test/test_web_stats_table_pre_aggregated_conversions.py # products/web_analytics/backend/hogql_queries/web_analytics_query_runner.py # products/web_analytics/backend/hogql_queries/web_goals.py # products/web_analytics/backend/hogql_queries/web_overview.py # products/web_analytics/backend/hogql_queries/web_overview_lazy_precompute.py
…ytics-models-to-product Renumber migration 1170 -> 1176 to slot in after the new master leaf (1175_drop_alertconfiguration_is_calculating_column). Drop the stale 1168 file that survived a previous renumber and was producing the duplicate migration leaf nodes blowing up CI. Align the lazy precompute test's mock patch path with the new products/web_analytics location and update the livestream bot docstring pointer.
…ytics-models-to-product
Contributor
ClickHouse migration SQL per cloud environmentNo ClickHouse migrations changed in this PR. |
ruff I001 — the file came in via merge with master and the rearranged imports (after the file move from posthog/ to products/) tripped the import sort check.
web_overview_lazy_precompute and its test (came in from master via #59075) import from products.analytics_platform — that was fine while the file lived under posthog/, but moving it into products/web_analytics/ tightens the tach boundary. analytics_platform has no facade, so allow the raw dep here (same pattern posthog uses).
…ytics-models-to-product
ruff format wants single-line ContentType chains; the older multi-line shape carried over when the file was renamed across master merges. Local ruff check passed but ruff format --check (separate CI step) didn't.
…ytics-models-to-product
Contributor
Author
|
@mendral-app is the CI failure from my branch? |
Contributor
|
This failure is very unlikely to be caused by your branch. Here's why:
TL;DR: This looks like a pre-existing flaky test. You should be safe to re-run CI or merge once the other checks pass. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Web analytics models (
WebAnalyticsFilterPreset,SavedHeatmap,HeatmapSnapshot) still live in the monolithicposthog/models/directory. Moving them toproducts/web_analytics/backend/models/continues the ongoing product isolation effort.Changes
WebAnalyticsConfigAppConfig and register inINSTALLED_APPSgit mvthe two model files intoproducts/web_analytics/backend/models/"Team"→"posthog.Team","User"→"posthog.User")db_tabledeclarations so tables stay in placeposthog/models/__init__.pySeparateDatabaseAndState— no DB changesposthog→web_analyticsHow did you test this code?
I'm an agent. Automated validation only:
python manage.py migrate --plan— plan is cleanpython manage.py makemigrations --check— no pending changesrgPublish to changelog?
No
🤖 Agent context
Agent-authored via Claude Code. Followed the
product-model-migrationskill workflow manually (small enough scope that the phased script wasn't needed). Straightforward migration — 3 models, no cross-app FK refs, no mock patches, noget_modelcalls.