feat(replay-vision): show and run the daily digest on demand - #73062
Conversation
|
😎 This pull request was merged. |
🤖 CI report
|
| File | Size | Δ vs base |
|---|---|---|
posthog-app/_parent/products/replay_vision/frontend/replay_scanners/ReplayScanner.js |
95.5 KiB | 🔺 +1.4 KiB (+1.5%) |
Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report
✅ Eager graph — within budget
How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.24 MiB · 22 files | no change | ███░░░░░░░ 27.5% of 4.51 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
8.25 MiB · 3,015 files | no change | ████████░░ 84.9% of 9.71 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 789 B | src/scenes/ChunkLoadErrorBoundary.tsx |
| 762 B | src/index.tsx |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 281.5 KiB | ../node_modules/.pnpm/posthog-js@1.407.2/node_modules/posthog-js/dist/rrweb.js |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 236.0 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 226.1 KiB | ../node_modules/.pnpm/posthog-js@1.407.2/node_modules/posthog-js/dist/module.js |
| 167.1 KiB | src/queries/validators.js |
| 154.3 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 105.8 KiB | src/lib/api.ts |
| 94.0 KiB | ../packages/quill/packages/quill/dist/index.js |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Toolbar bundle — eager 2.18 MiB within budget
What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.
| Metric | Size | Δ vs base | Budget |
|---|---|---|---|
| Eager (shipped) entry + static imports |
2.18 MiB · 17 files | no change | ████░░░░░░ 38.1% of 5.72 MiB |
| Deferred (lazy) | 2.07 MiB · 33 files | no change | n/a — loads on demand |
Loader dist/toolbar.js |
1.1 KiB | no change | █░░░░░░░░░ 5.8% of 19.5 KiB |
Largest eagerly-shipped chunks
| Size | File |
|---|---|
| 714.7 KiB | dist/toolbar/toolbar-app-MKBF64XP.css |
| 545.4 KiB | dist/toolbar/chunk-chunk-RY664CLV.js |
| 484.2 KiB | dist/toolbar/chunk-chunk-QS5AHYGW.js |
| 133.6 KiB | dist/toolbar/chunk-chunk-SD7L5PL4.js |
| 131.8 KiB | dist/toolbar/chunk-chunk-T5KY5WYR.js |
| 71.0 KiB | dist/toolbar/toolbar-app-IAU4H5TY.js |
| 69.0 KiB | dist/toolbar/chunk-chunk-27JL52RE.js |
| 35.6 KiB | dist/toolbar/chunk-chunk-YHJZ5HSV.js |
| 20.9 KiB | dist/toolbar/chunk-chunk-SLYAFD5M.js |
| 12.2 KiB | dist/toolbar/chunk-chunk-PIK3PADE.js |
Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile
✅ Dist folder size — 🔺 +18.0 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1359.35 MiB · 🔺 +18.0 KiB (+0.0%)
ℹ️ MCP UI apps size — 32 app(s), 17065.5 KB JS
Built size of each MCP UI app (main.js + styles.css).
| App | JS | CSS |
|---|---|---|
| debug | 599.5 KB | 187.7 KB |
| action | 457.8 KB | 187.7 KB |
| action-list | 564.3 KB | 187.7 KB |
| cohort | 456.8 KB | 187.7 KB |
| cohort-list | 563.3 KB | 187.7 KB |
| email-template | 456.6 KB | 187.7 KB |
| error-details | 472.4 KB | 187.7 KB |
| error-issue | 457.5 KB | 187.7 KB |
| error-issue-list | 564.2 KB | 187.7 KB |
| experiment | 561.5 KB | 187.7 KB |
| experiment-list | 565.1 KB | 187.7 KB |
| experiment-results | 563.2 KB | 187.7 KB |
| feature-flag | 567.1 KB | 187.7 KB |
| feature-flag-list | 570.9 KB | 187.7 KB |
| feature-flag-testing | 461.0 KB | 187.7 KB |
| insight-actors | 562.1 KB | 187.7 KB |
| invite-email-preview | 456.0 KB | 187.7 KB |
| llm-costs | 559.5 KB | 187.7 KB |
| session-recording | 458.6 KB | 187.7 KB |
| session-summary | 463.9 KB | 187.7 KB |
| survey | 458.4 KB | 187.7 KB |
| survey-global-stats | 562.2 KB | 187.7 KB |
| survey-list | 565.0 KB | 187.7 KB |
| survey-stats | 562.2 KB | 187.7 KB |
| trace-span | 457.2 KB | 187.7 KB |
| trace-span-list | 564.2 KB | 187.7 KB |
| workflow | 457.1 KB | 187.7 KB |
| workflow-list | 563.7 KB | 187.7 KB |
| loops-review | 461.2 KB | 187.7 KB |
| query-results | 745.5 KB | 187.7 KB |
| render-ui | 826.2 KB | 187.7 KB |
| visual-review-snapshots | 461.6 KB | 187.7 KB |
⚠️ Playwright — 1 failed
🎭 Playwright report · View test results →
❌ 1 failed test:
- Split a person with multiple distinct IDs (chromium)
These issues are not necessarily caused by your changes.
Annoyed by this section? Help fix flakies and failures and it will go green!
⚠️ Backend coverage — 91.0% of changed backend lines covered — 7 uncovered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ██████████████████░░ 91.0% (79 / 86)
| File | Patch | Uncovered changed lines |
|---|---|---|
products/replay_vision/backend/api/trigger.py |
70.6% | 149–150, 153–155 |
products/replay_vision/backend/api/vision_actions.py |
90.0% | 637, 646 |
🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 30030336418 -n patch-coverage), or the coverage-data block at the end of this comment.
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
platform_features |
██░░░░░░░░░░░░░░░░░░ 12.1% |
7 / 58 |
batch_exports |
████████░░░░░░░░░░░░ 39.6% |
8,412 / 21,249 |
demo |
███████████░░░░░░░░░ 56.2% |
1,497 / 2,663 |
warehouse_sources_queue |
████████████░░░░░░░░ 59.2% |
148 / 250 |
tasks |
██████████████░░░░░░ 69.2% |
29,650 / 42,817 |
data_tools |
██████████████░░░░░░ 70.0% |
63 / 90 |
ai_gateway |
███████████████░░░░░ 75.0% |
9 / 12 |
signals |
████████████████░░░░ 79.8% |
21,626 / 27,098 |
cdp |
████████████████░░░░ 81.0% |
3,144 / 3,881 |
data_modeling |
█████████████████░░░ 82.7% |
5,547 / 6,708 |
wizard |
█████████████████░░░ 84.8% |
935 / 1,102 |
notebooks |
█████████████████░░░ 85.3% |
7,275 / 8,531 |
agent_platform |
█████████████████░░░ 86.4% |
3,807 / 4,405 |
actions |
█████████████████░░░ 86.6% |
717 / 828 |
cohorts |
██████████████████░░ 87.8% |
4,488 / 5,114 |
product_tours |
██████████████████░░ 87.9% |
1,303 / 1,482 |
exports |
██████████████████░░ 88.4% |
6,949 / 7,861 |
data_warehouse |
██████████████████░░ 88.9% |
11,887 / 13,378 |
business_knowledge |
██████████████████░░ 89.0% |
4,391 / 4,936 |
conversations |
██████████████████░░ 89.3% |
16,834 / 18,848 |
engineering_analytics |
██████████████████░░ 89.4% |
6,371 / 7,130 |
dashboards |
██████████████████░░ 89.4% |
5,983 / 6,693 |
visual_review |
██████████████████░░ 89.4% |
5,818 / 6,505 |
error_tracking |
██████████████████░░ 89.7% |
10,174 / 11,342 |
alerts |
██████████████████░░ 89.9% |
4,054 / 4,508 |
early_access_features |
██████████████████░░ 90.1% |
1,031 / 1,144 |
mcp_analytics |
██████████████████░░ 90.1% |
2,763 / 3,065 |
streamlit_apps |
██████████████████░░ 90.4% |
2,501 / 2,767 |
links |
██████████████████░░ 90.6% |
183 / 202 |
slack_app |
██████████████████░░ 90.7% |
9,028 / 9,951 |
marketing_analytics |
██████████████████░░ 91.0% |
11,792 / 12,964 |
stamphog |
██████████████████░░ 91.1% |
4,056 / 4,450 |
mcp_store |
██████████████████░░ 91.9% |
4,257 / 4,634 |
notifications |
███████████████████░ 92.6% |
1,011 / 1,092 |
product_analytics |
███████████████████░ 92.7% |
5,811 / 6,268 |
ai_observability |
███████████████████░ 92.7% |
14,995 / 16,168 |
surveys |
███████████████████░ 93.0% |
5,730 / 6,163 |
managed_migrations |
███████████████████░ 93.1% |
1,424 / 1,530 |
web_analytics |
███████████████████░ 93.1% |
14,398 / 15,461 |
posthog_ai |
███████████████████░ 93.2% |
1,326 / 1,422 |
approvals |
███████████████████░ 93.3% |
3,437 / 3,682 |
reminders |
███████████████████░ 93.4% |
468 / 501 |
workflows |
███████████████████░ 93.6% |
6,390 / 6,826 |
legal_documents |
███████████████████░ 94.1% |
1,568 / 1,667 |
endpoints |
███████████████████░ 94.1% |
8,640 / 9,177 |
messaging |
███████████████████░ 94.3% |
2,682 / 2,845 |
tracing |
███████████████████░ 94.5% |
2,670 / 2,826 |
revenue_analytics |
███████████████████░ 94.6% |
3,651 / 3,858 |
skills |
███████████████████░ 94.6% |
3,035 / 3,207 |
review_hog |
███████████████████░ 94.7% |
6,814 / 7,199 |
logs |
███████████████████░ 95.4% |
9,935 / 10,416 |
experiments |
███████████████████░ 95.7% |
25,036 / 26,159 |
growth |
███████████████████░ 96.1% |
3,245 / 3,376 |
replay_vision |
███████████████████░ 96.2% |
15,170 / 15,774 |
annotations |
███████████████████░ 96.2% |
732 / 761 |
feature_flags |
███████████████████░ 96.5% |
17,110 / 17,734 |
user_interviews |
███████████████████░ 96.5% |
2,638 / 2,734 |
access_control |
███████████████████░ 96.8% |
851 / 879 |
customer_analytics |
███████████████████░ 97.1% |
9,047 / 9,317 |
warehouse_sources |
███████████████████░ 97.1% |
329,188 / 338,926 |
data_catalog |
███████████████████░ 97.4% |
2,367 / 2,429 |
analytics_platform |
████████████████████ 98.0% |
2,153 / 2,197 |
metrics |
████████████████████ 98.2% |
2,491 / 2,536 |
pulse |
████████████████████ 98.4% |
2,017 / 2,049 |
live_debugger |
████████████████████ 99.2% |
613 / 618 |
field_notes |
████████████████████ 99.4% |
158 / 159 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
🦔 Hogbox preview · ✅ ready▶ Open the preview
commit |
There was a problem hiding this comment.
New on-demand "run now" endpoint and digest-visibility change are well-tested (coalescing, mode rejection, schedule-untouched invariant all covered) and properly scoped/access-controlled; author is on the owning team with STRONG familiarity (92% of lines, 123 recent merged PRs in this path), which supplies the independent assurance this touch of API surface would otherwise need.
- Author wrote 92% of the modified lines and has 123 merged PRs in these paths (familiarity STRONG).
- 👍 on the PR from hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 767L, 11F substantive, 1087L/19F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1d-complex (1087L, 19F, cross-cutting, feat) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ 5c6568c · reviewed head 41e9eb5 |
…3296) ## Problem Replay Vision scanners produce group summaries and alerts (the "Summaries and alerts" tab) by synthesizing up to 100 observations into one report. Those reports cite their source observations inline with `[obs N]` markers. There was no way for an agent to read those reports over the MCP, so there was no way to audit whether a report's citations actually hold up, whether `[obs N]` points at an observation that supports the claim or a bogus one. The underlying REST API (`VisionActionViewSet`, `VisionActionRunViewSet`) already exists and already resolves each `[obs N]` to its observation. It just wasn't exposed as MCP tools. ## Changes Enables four read-only MCP tools that were already scaffolded (disabled) in `products/replay_vision/mcp/tools.yaml`, adds their scopes, annotations, and descriptions, and regenerates the MCP and type artifacts: - `vision-actions-list` / `vision-actions-retrieve`, the "and then…" automations over a scanner's observations (`mode: group_summary` or `alert`). - `vision-actions-runs-list`, a single action's run history (lightweight rows). - `vision-actions-runs-retrieve`, one run in full: `synthesized_markdown` (the report, with its `[obs N]` markers) plus `observations`, the recordings it included in summary order, each carrying the 1-based `index` that `[obs N]` references. So `observations[N-1]` is the observation cited by `[obs N]`, which is exactly what a citation audit needs. No backend changes. The endpoints, serializers, and RBAC already existed on master. Scopes match the viewsets: `vision_action:read` throughout, plus `session_recording:read` on the run tools since runs surface recording-derived summaries. All four are read-only and gated behind the existing `replay-vision` feature flag. ## How did you test this code? I (Claude) ran `hogli build:openapi` to regenerate, `pnpm --filter=@posthog/mcp lint-tool-names` (passes), and `hogli ci:preflight` (0 failures). I verified codegen is idempotent: regenerating on clean master with no change produces zero diff, so the only generated churn here is the four new tools. I did not exercise the tools against a live MCP connection, the deployed MCP doesn't serve them until this ships, and the local MCP wasn't wired up for prod data in this session. ## Automatic notifications - [ ] Publish to changelog? - [ ] Alert Sales and Marketing teams? ## 🤖 Agent context **Autonomy:** Human-driven (agent-assisted) Kim directed this. The goal was to audit whether the citations in Replay Vision group summaries actually support their claims. Investigating that surfaced the gap: the per-session summarizer observations were readable over MCP, but the group summaries (VisionActionRun) were not. I confirmed the REST API already existed and resolved `[obs N]` to observations, so the fix was to enable the already-scaffolded tools rather than build anything. Tools/agent: Claude Code (Opus 4.8). Skills invoked: `/exploring-replay-vision-observations`, `/implementing-mcp-tools`, `/querying-local-postgres`, `/improving-drf-endpoints` (read-only, no serializer changes were needed). One snag worth noting for the reviewer: an earlier version of this change got picked up and committed by a parallel session into #73062. This PR rebuilds it cleanly off master so it stands alone, so expect a `tools.yaml` conflict between the two. Whichever lands second reconciles the generated files by re-running `hogli build:openapi`.
… table #72428 filtered the built-in daily digest out of the Summaries and alerts table (it had only its Observations-tab hero card). That made the digest invisible on the page where every other automation is listed. Show it here too, marked with a "Daily digest" chip, so the page is the one place to see and manage all of a scanner's summaries and alerts. Generated-By: PostHog Code Task-Id: e93a941a-13d2-44c5-a60d-080dd2de854f
Add a "Run now" button so you don't have to wait for the schedule — the
common case is: create a scanner, scan a batch of recordings on demand,
then immediately want a group summary of them.
Backend: POST /vision/actions/{id}/run/ starts the per-action processing
workflow now (group_summary only; alerts check continuously). It reuses
the scheduled workflow's deterministic id, so a manual run coalesces with
an in-flight run rather than double-charging (reported as already_running).
The workflow never advances next_run_at, so the recurring schedule is
untouched; scheduled_at=now just anchors the observation window at the
present (covering everything since the last summary, or the last 24h).
The vision_actions engine import is deferred off the API module-load path.
Frontend: "Run now" on the digest card (both the pending and populated
states) and on every summary's run-history page, which refreshes the run
list so the new run appears immediately.
Generated-By: PostHog Code
Task-Id: e93a941a-13d2-44c5-a60d-080dd2de854f
Spam-clicking Run now was already safe (the deterministic workflow id coalesces repeat clicks server-side — no double-charge), but the UI didn't show it: the button re-enabled the moment the ~1s trigger request returned, while the summary kept generating for another minute or two, and the new run often didn't appear because the refresh raced the workflow creating its run row. Now both surfaces (the digest card and the run-history page) track whether a run is actually processing, from the runs list: - "Run now" shows "Running…" and is disabled while a run is in flight, so it visibly can't be spam-fired. - After triggering, poll through a short grace window (the workflow creates its row asynchronously) so the new run appears, then keep polling until it finishes — at which point the card/list update and the button re-enables. Generated-By: PostHog Code Task-Id: e93a941a-13d2-44c5-a60d-080dd2de854f
41e9eb5 to
85556ed
Compare
New commits pushed (delta classified non_linear_history) — stamphog approval dismissed; re-review running automatically.
There was a problem hiding this comment.
New on-demand run endpoint is properly team-scoped and permission-gated, ships solid tests for idempotency/coalescing and alert rejection, and the author is on the owning team with STRONG, same-day familiarity in this exact code — sufficient assurance even though it touches a new API endpoint.
- Author wrote 93% of the modified lines and has 78 merged PRs in these paths (familiarity STRONG).
- 👍 on the PR from hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 586L, 9F substantive, 758L/14F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1d-complex (758L, 14F, two-areas, feat) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ d35860b · reviewed head 85556ed |
Adds a new section to running-scanners.mdx explaining the on-demand summary trigger introduced in PR PostHog/posthog#73062. Covers: - How to use the Run now button - Summaries only (not alerts) - Schedule unaffected - Duplicate run coalescing
Problem
Two gaps in the daily-digest experience:
Changes
Show the digest (commit 1). The daily digest is listed in the Summaries and alerts table alongside user-created summaries and alerts, marked with a "Daily digest" chip. The page is now the one place to see and manage every automation on a scanner. (It keeps its Observations-tab hero card too.)
Run now (commit 2). New
POST /vision/actions/{id}/run/starts the per-action processing workflow immediately.group_summary) — alerts check continuously on the sweep, so there's no meaningful on-demand run for them (400).already_running).next_run_at, so the recurring schedule is untouched.scheduled_at=nowjust anchors this run's observation window at the present — covering everything since the last summary, or the last 24h (which is exactly the just-scanned batch for a fresh scanner).vision_actionsengine import is deferred off the API module-load path."Run now" button appears on the digest card (both the "nothing summarized yet" and populated states) and on every summary's run-history page, which refreshes the run list so the new run shows up immediately.
How did you test this code?
Automated (all run by me, the agent):
TestVisionActionRunNow(3): starts the workflow now withscheduled_atset andnext_run_atunchanged; coalesces onto an already-running workflow (already_running: true); rejects alerts (400, no workflow started). Fulltest_vision_actions_api.py: 41 passed.visionActionRunsLogicandscannerDigestLogiceach get arunNowtest (posts to/run/, clears loading, refreshes). Touched suites 8 passed; ran typegen, oxlint, oxfmt, ruff, ty (pre-commit), and scopedtsgo— clean.hogli build:openapiregenerated types.I did not manually click through the running app.
🤖 Agent context
Autonomy: Human-driven (agent-assisted) — Kim (@ksvat) is the DRI.
Built with Claude Code (Opus). Skills invoked:
/improving-drf-endpoints,/writing-kea-logics,/writing-tests.Design notes: run-now reuses the existing
ProcessVisionActionWorkflowand its deterministic workflow id rather than a bespoke one-off — that's what gives idempotency (no double-charge) and keeps the schedule cursor logic untouched (the sweep advancesnext_run_at, never the workflow). Showing the digest in the table reverses the #72428 filter decision (which I'd flagged as a judgment call at the time); it's now a first-class, chip-marked row.Commit 1 (digest visibility) is independent of commit 2 (run-now) if you'd prefer them split.
Created with PostHog Code