feat(slack_app): capture analytics event on @PostHog mention

[oliverb123]

Problem

We have no product-analytics event for when someone interacts with the @PostHog Slack bot. The bot handles Slack app_mention events in products/slack_app/backend/api.py, but that path only emits structlog application logs — nothing is captured into PostHog. So questions like "how often is the bot mentioned?", "how many are first-touch vs. follow-ups in a thread?", and "who is using it?" can't be answered from events today.

Changes

Adds a posthog code slack mention received event, captured at the single chokepoint where a mention is confirmed actionable and routed to the coding-agent workflow (_start_mention_workflow).

The Slack thread is the session:

• slack_session_id = <team>:<channel>:<thread_ts>
• is_first_message_in_session — true when the mention has no thread_ts or its thread_ts equals its own ts (i.e. it opens the thread)
• session_message_count — 1 for a first message; otherwise a best-effort count of thread replies via conversations_replies
• user identification — the acting Slack user is resolved to a PostHog User, so distinct_id is the real person's distinct id (with $set analytics metadata, and groups for org/project), per PostHog's server-side capture convention. When no PostHog user can be resolved, it falls back to a stable slack:<team>:<user> distinct id and sets posthog_user_identified=false.

Capture is fully wrapped in try/except — analytics is best-effort and can never break Slack event handling.

How did you test this code?

I'm an agent (PostHog Code). I added unit tests in products/slack_app/backend/tests/test_mention_analytics.py covering: first-message identification + count, follow-up thread counting, the unresolved-user fallback distinct id, and that a capture failure is swallowed.

I could not run the test suite in this environment — it has no flox/hogli, no pytest/django installed, and constrained disk. I verified both files parse (ast.parse) and that the change matches the existing patterns in the file (_resolve_posthog_user_from_event, groups() from posthog.event_usage, the structlog/best-effort idiom). A human should run hogli test products/slack_app/backend/tests/test_mention_analytics.py and lint (ruff check/format) before merge.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

No docs changes.

🤖 Agent context

Authored by PostHog Code (Claude Opus) on request to add an analytics event tracking @PostHog Slack mentions, including first-message-in-session, session message count, a session identifier, and user identification.

Key decisions:

• Placement: chose _start_mention_workflow as the single emit point — it fires for genuinely handled mentions (after ignore/scope/project checks), including repo-picker follow-ups, rather than for every raw webhook (which includes ignored edits and unroutable events). This keeps the event meaningful and guarantees a resolved target integration is available.
• Session model: reused the existing thread semantics (thread_ts) already used throughout the file as the session key, rather than inventing a new identifier.
• Message count: kept it best-effort with a bounded conversations_replies call (only for follow-ups) to avoid extra Slack API cost/latency on the common first-message case.
• User identity: reused the existing _resolve_posthog_user_from_event helper so attribution maps to a real PostHog user where possible, matching the server-side posthoganalytics.capture(distinct_id=user.distinct_id, ... groups=groups(...)) convention.

[github-actions]

Hey @oliverb123! 👋

It looks like your git author email on this PR isn't your @posthog.com address (oliverbrowne627@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[greptile-apps]

Prompt To Fix All With AI

Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 3
products/slack_app/backend/api.py:1667-1671
The `identified and posthog_user is not None` condition is redundant. `identified` is computed as `bool(posthog_user and posthog_user.distinct_id)`, which already guarantees `posthog_user` is not None when `identified` is True. The extra guard cannot trigger in practice.

```suggestion
        distinct_id = (
            posthog_user.distinct_id
            if identified
            else f"slack:{slack_team_id}:{slack_user_id or 'unknown'}"
        )
```

### Issue 2 of 3
products/slack_app/backend/api.py:1634
**Synchronous Slack API call in the webhook handler**

`conversations_replies` makes a real HTTP roundtrip on every follow-up mention, adding its latency to the time before the handler returns 202. Slack's 3-second retry window is already shared with Temporal workflow submission and user-info resolution; a slow or rate-limited `conversations_replies` response narrows that window further. Since `session_message_count` is a best-effort metric, consider moving this call outside the request path (e.g., deferring it into the analytics properties assembled by the Temporal workflow) or accepting `None` as the count for follow-ups when latency budget is tight.

### Issue 3 of 3
products/slack_app/backend/tests/test_mention_analytics.py:1-99
**Tests not parameterised**

The codebase prefers parameterised tests. The four methods here share the same `_report_slack_mention_received` entry point and differ only in event shape and mock return values. Scenarios like first-message vs. follow-up (`is_first_message_in_session`, `session_message_count`) and resolved vs. unresolved user (`distinct_id`, `posthog_user_identified`) are natural axes for `@pytest.mark.parametrize`, which would also make it easy to add the missing case where `thread_ts == ts` (thread root is the mention itself) without adding another full test method.

_{Reviews (1): Last reviewed commit: "chore(slack_app): fix import order and n..." | Re-trigger Greptile}

[Copilot]

Pull request overview

Adds server-side product analytics for actionable app_mention events handled by the PostHog Code Slack app, so mentions can be measured (session/thread semantics, first vs follow-up, user attribution) without affecting Slack event handling.

Changes:

• Captures a new posthog code slack mention received event at the _start_mention_workflow chokepoint, including session/thread identifiers and best-effort thread message counts.
• Resolves the acting Slack user to a PostHog User when possible and falls back to a stable Slack-derived distinct_id when not.
• Adds unit tests covering first-message detection, follow-up counting, fallback distinct id behavior, and best-effort failure swallowing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
products/slack_app/backend/api.py	Adds analytics capture helper(s) and hooks capture into mention workflow start.
products/slack_app/backend/tests/test_mention_analytics.py	Adds unit tests for the new analytics capture behavior and failure handling.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tests-posthog]

⏭️ Skipped snapshot commit because branch advanced to 5ff0548 while workflow was testing 7478165.

The new commit will trigger its own snapshot update workflow.

If you expected this workflow to succeed: This can happen due to concurrent commits. To get a fresh workflow run, either:

• Merge master into your branch, or
• Push an empty commit: git commit --allow-empty -m 'trigger CI' && git push

[tests-posthog]

Query snapshots: Backend query snapshots updated

Changes: 1 snapshots (1 modified, 0 added, 0 deleted)

What this means:

• Query snapshots have been automatically updated to match current output
• These changes reflect modifications to database queries or schema

Next steps:

• Review the query changes to ensure they're intentional
• If unexpected, investigate what caused the query to change

Review snapshot changes →

[joshsny]

nit: can we move these out of the api file

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-06-03 11:41 UTC	Run
prod-us	✅ Deployed	2026-06-03 12:05 UTC	Run
prod-eu	✅ Deployed	2026-06-03 12:09 UTC	Run