Skip to content

feat(inbox): instrument analytics for engagement events#2228

Merged
andrewm4894 merged 8 commits into
mainfrom
andrewm4894/inbox-analytics-events
May 21, 2026
Merged

feat(inbox): instrument analytics for engagement events#2228
andrewm4894 merged 8 commits into
mainfrom
andrewm4894/inbox-analytics-events

Conversation

@andrewm4894
Copy link
Copy Markdown
Member

@andrewm4894 andrewm4894 commented May 19, 2026

Problem

The inbox had a single analytics event (INBOX_INTEREST_REGISTERED, from the empty state for users without access). Once a user was in the inbox, nothing was captured: no view event, no open/close on reports, no signal that anyone actually engaged with what's in front of them. Compared to other surfaces in PostHog Code (tasks, setup, files, command menu — each with both *_VIEWED and action events), inbox was the only major surface where we couldn't build a funnel of viewed → opened → acted or compute engagement rate at all.

Backend state changes (e.g. dismiss → state: suppressed via signals/reports/{id}/state/) give us volume of dismisses, but no denominator, no actor session, no surface (toolbar vs detail), and no funnel join with the rest of the app's product analytics.

Changes

Adds five new typed events under ANALYTICS_EVENTS and wires them through the inbox feature. Engagement is not an event in the code — it's defined as a PostHog Action over these raw events, so the bar can be re-tuned in the UI without code deploys.

Events

  • Inbox viewed — once per inbox visit when data settles, with report_count, total_count, ready_count, source_product_filter, status_filter_count, is_empty.
  • Inbox report opened — every report select, with rank (visible 0-indexed position), list_size, open_method (click / click_cmd / click_shift / keyboard / deeplink), previous_report_id, plus the report's status, priority, and source_products.
  • Inbox report closed — pairs with each open, with time_spent_ms, scrolled (raw fact), and close_method (next_report / deselected / navigated_away / unmount).
  • Inbox report scrolled — fires once per open on the first scroll inside the detail pane, with time_since_open_ms so an Action can filter by dwell threshold.
  • Inbox report action — covers 15 action types: dismiss, snooze, delete, reingest, create_pr, open_pr, copy_link, expand_signal, collapse_signal, expand_signal_section, view_signal_external, expand_why, click_suggested_reviewer, expand_task_section, play_session_recording. Properties include surface (detail_pane / toolbar / keyboard / list_row), is_bulk, bulk_size, and contextual fields like signal_id + signal_source_product + signal_source_type for signal-card interactions, signal_section, why_field, task_section, dismissal_reason.

All four report-scoped events carry report_title (string | null) and report_age_hours (number, one-decimal precision) so the activity stream is human-readable and you can slice engagement by report freshness.

Engagement defined in PostHog, not code

Two Actions in project 2 over these raw events:

  • Inbox engagement (272793) — meaningful action (any action_type except thin caret clicks like expand_signal / expand_why / collapse_signal) OR Inbox report scrolled with time_since_open_ms >= 5000.
  • Inbox deep engagement (272794) — terminal commitment only: dismiss, snooze, delete, create_pr, open_pr, play_session_recording.

The Actions can be re-tuned in the PostHog UI without shipping code. New definitions read retroactively against the captured event stream.

Implementation

  • useInboxEngagementTracker hook owns the OPENED/CLOSED/SCROLLED lifecycle keyed on the currently-selected report, with refs for openedAt and hasScrolled. No more dwell timer, no client-side engagement computation.
  • Module-level pendingInboxOpenMethod register lets click / keyboard / deeplink call sites annotate the next OPENED without prop drilling.
  • SignalInteractionContext lets the shared signal-card helpers (CollapsibleBody, CodePathsCollapsible, DataQueriedCollapsible) fire interactions without touching the six source-specific card components. A delegated onClickCapture at the SignalCard root catches external-link clicks. A native onPlay on the inline session-recording video fires play_session_recording once per card per open.
  • ReportDetailPane exposes one fireDetailAction(actionType, extra?) helper that fills surface: "detail_pane", is_bulk: false, bulk_size: 1, plus report_title and report_age_hours — and one makeSignalInteractionHandler(signal) builder used by both signal lists.
  • SignalsToolbar fires INBOX_REPORT_ACTION for each report in a bulk action via a small fireBulkAction helper that looks up title + age per id.
  • ReportImplementationPrLink accepts an optional onLinkClick to fire open_pr without baking analytics into the link component.
  • ReportTaskLogs accepts an optional onSectionExpand for the Research / Implementation row toggles.

How did you test this?

Ran pnpm dev:code against the PostHog Code internal team's dev project and confirmed via HogQL on the events table:

  • Inbox viewed fired once on inbox entry.
  • Inbox report opened fired with correct rank, list_size, open_method: "click", and previous_report_id chaining each click to the prior one.
  • Inbox report closed fired on each selection change with realistic time_spent_ms values, close_method: "next_report", scrolled: false.

Scrolled / action events and the Action matches can be verified once flowing to project 2 by opening the activity feed and Action insight respectively. Type-checking and biome both pass under the pre-commit hook.

Publish to changelog?

no

…tion events

Adds five new INBOX_* analytics events covering inbox engagement end-to-end:
viewed (per visit), report opened/closed (with rank, time_spent_ms,
previous_report_id), engaged (5s dwell+scroll OR action), and action
(dismiss, snooze, delete, reingest, create_pr, open_pr, copy_link,
expand_signal[_section], view_signal_external, expand_why, expand_task_section,
click_suggested_reviewer).

Engagement is tracked via a new useInboxEngagementTracker hook that owns
the OPENED/CLOSED lifecycle keyed on the currently-selected report, with
a module-level pendingInboxOpenMethod register so click/keyboard/deeplink
paths can annotate the next OPENED without prop drilling. Signal card
interactions fan out through a SignalInteractionContext so the existing
source-specific card components stay untouched.
@andrewm4894 andrewm4894 self-assigned this May 19, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 19, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
apps/code/src/renderer/features/inbox/utils/pendingInboxOpenMethod.ts:1-13
**Stale pending method on no-op selection**

When the user clicks an already-selected report, `setPendingInboxOpenMethod("click")` is called, but `setSelectedReportIds` is a no-op (selection doesn't change), so `consumePendingInboxOpenMethod` in the engagement tracker effect is never reached. The module-level `pendingMethod` variable stays as `"click"`. If a report is then dismissed and the selection store auto-advances to the next item via a code path that doesn't call `setPendingInboxOpenMethod`, `INBOX_REPORT_OPENED` for the auto-advanced report would report `open_method: "click"` instead of `"unknown"`. A guard at the call site (only call `setPendingInboxOpenMethod` when the selection will actually change) would eliminate the stale value.

### Issue 2 of 2
apps/code/src/renderer/features/inbox/components/detail/ReportDetailPane.tsx:354-367
**Scroll listener not re-attached if the ScrollArea viewport re-mounts**

The effect queries `[data-radix-scroll-area-viewport]` from `scrollAreaRootRef.current` once, and only re-runs when `onScroll` changes. Since `onScroll` (`tracker.signalScroll`) is a stable `useCallback` that never changes identity, the effect runs exactly once per `ReportDetailPane` mount. If Radix ever replaces the viewport element internally (e.g., during a resize or virtualization), the listener would be attached to the now-detached element and scroll events would be silently lost. Adding `report.id` to the effect's dependency array would at minimum ensure the listener is re-registered whenever a new report is displayed.

Reviews (1): Last reviewed commit: "feat(inbox): instrument analytics for vi..." | Re-trigger Greptile

Makes events readable in the activity stream without needing to look up
report IDs, and adds report age in hours (one decimal precision) as a
queryable dimension so we can see how engagement varies with how fresh
the report is.

Both fields are populated on INBOX_REPORT_OPENED, _CLOSED, _ENGAGED, and
_ACTION. Engagement-lifecycle events read from the cached OpenInfo;
toolbar bulk actions and the dismiss-dialog confirm path look up the
report by id from the reports list at fire time.
Fires Inbox report action with action_type: play_session_recording the
first time a user clicks play on the inline session replay inside an
Evidence card. Per-card guard via ref so repeat plays during the same
open don't spam the event stream.
Drops the client-side Inbox report engaged event in favour of capturing
raw signals and deriving engagement as a PostHog Action. The Action can
then evolve in the PostHog UI without code deploys — change the dwell
threshold, exclude weak actions, add tiered definitions, all without
shipping.

Adds Inbox report scrolled (fires once per open on first scroll, with
report context plus time_since_open_ms so the Action can filter by
dwell). Removes the engaged boolean from Inbox report closed (scrolled
stays as a raw fact). Removes hasEngaged tracking and the dwell timer
from the tracker hook.
…tener per report

Two small fixes from review feedback:

1. The pending open_method register could leak a stale value if the call
   site sets it but the selection doesn't actually change (e.g. clicking
   the already-selected report). A 2s TTL on the pending value ensures a
   later, unrelated OPEN doesn't inherit it — falls back to "unknown".

2. The scroll-listener effect on ReportDetailPane re-runs on report.id
   change too, so the listener is rebound on every report swap and would
   survive a future Radix internal replacing the viewport element.
@andrewm4894 andrewm4894 requested a review from a team May 19, 2026 17:06
…ytics

When a bulk dismiss/snooze/delete/reingest (or single-report dismiss
confirm) resolves, the inbox query has already been invalidated and the
visible list has been re-queried without the affected report — so a
post-await rank lookup returns -1 and list_size reflects the smaller
post-mutation count. Now the call sites snapshot rank + list_size before
the await and pass them through; signalAction accepts optional overrides.

Also restores report.id to the scroll listener's dep array with a
biome-ignore for useExhaustiveDependencies (the rule auto-stripped it
since report.id isn't referenced inside the effect body). The dep
forces the listener to re-bind on report swap as defence against a
future Radix internal that might replace the viewport element.
The biome-ignore comment didn't survive --unsafe auto-fix; report.id kept
getting stripped from the dep array on every commit. Now the effect
assigns report.id to viewport.dataset.reportId on bind and clears it on
cleanup, which (a) gives biome a legitimate reactive use of report.id
that won't get stripped, and (b) adds a DOM marker handy for debugging
which report's viewport is currently attached.
@andrewm4894
Copy link
Copy Markdown
Member Author

Addressing a second round of bot feedback in 4863a9e + 121e1a0:

Pre-mutation snapshot on async actions — bulk dismiss/snooze/delete/reingest and the single-report dismiss-confirm path were firing INBOX_REPORT_ACTION after the mutation resolved, by which point the inbox query had been invalidated and the affected report was no longer in the visible list. signalAction was then computing rank: -1 and the post-mutation list_size. Now signalAction accepts optional rank/list_size overrides, and SignalsToolbar.fireBulkAction + InboxSignalsTab.handleDismissConfirm snapshot both values from the visible list before awaiting the mutation and pass them through. (4863a9e)

Scroll listener report.id dep — round two — my first attempt added report.id to the dep array, but biome check --write --unsafe (pre-commit hook) auto-stripped it as unused since the effect body didn't reference it. A biome-ignore comment didn't survive either. Now the effect writes report.id to viewport.dataset.reportId on bind and clears it on cleanup — biome sees a legitimate reactive use, and we also get a DOM marker indicating which report's viewport is currently attached, handy for debugging. (121e1a0)

@andrewm4894 andrewm4894 added the Stamphog This will request an autostamp by stamphog on small changes label May 20, 2026
…lytics-events

# Conflicts:
#	apps/code/src/shared/types/analytics.ts
@andrewm4894 andrewm4894 merged commit f6a4c91 into main May 21, 2026
15 checks passed
@andrewm4894 andrewm4894 deleted the andrewm4894/inbox-analytics-events branch May 21, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Stamphog This will request an autostamp by stamphog on small changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants