feat(seer-slack): Add analytics for Seer agent Slack responses#113318
Open
alexsohn1126 wants to merge 9 commits intomasterfrom
Open
feat(seer-slack): Add analytics for Seer agent Slack responses#113318alexsohn1126 wants to merge 9 commits intomasterfrom
alexsohn1126 wants to merge 9 commits intomasterfrom
Conversation
Record an ai.explorer.slack.responded analytics event each time the Seer Explorer agent responds to a Slack user. The event captures conversation type (app mention, direct message, or AI assistant thread), run id, and thread statistics (total messages, unique users, prior Seer runs) so downstream analysis can derive metrics like first-vs-continued engagement. SeerExplorerOperator.trigger_explorer now returns the run id alongside the count of prior runs for the same thread, which the task uses to populate the event. Refs ISWF-2482 Co-Authored-By: Claude <noreply@anthropic.com>
github-actions
bot
added
the
Scope: Backend
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
Top-level @mentions have thread_ts=None, but the analytics event field is typed str. Fall back to the message's own ts, which is the thread root once Seer replies. Co-Authored-By: Claude <noreply@anthropic.com>
Add linked_users_in_thread to the ai.explorer.slack.responded event so downstream analysis can see how many thread participants have a linked Sentry identity (proxy for team adoption stickiness). Adds an identity_ext_ids list filter to the identity RPC service so the lookup is a single batched query per mention rather than N+1. Includes a test against real data that exercises the new filter end-to-end. Co-Authored-By: Claude <noreply@anthropic.com>
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
alexsohn1126
commented
Apr 17, 2026
Member
Author
There was a problem hiding this comment.
There were no tests for identity service filtering, I made one
Compute seer_msgs_in_thread by counting bot messages already in the thread history instead of piggybacking on the Explorer runs count. The runs count was effectively boolean (0 or 1) since subsequent mentions continue the existing run rather than create new ones, so the field name didn't match what the data was saying. Reverts trigger_explorer back to returning just run_id; the operator no longer has an analytics concern and limit=1 is restored. Co-Authored-By: Claude <noreply@anthropic.com>
msg.get("user") on list[dict] returns Any | None, so the set
comprehension was typed set[Any | None]. Use isinstance in a walrus to
narrow to str, matching the _count_linked_users signature.
Co-Authored-By: Claude <noreply@anthropic.com>
Tasks enqueued by pre-analytics workers during a rolling deploy don't pass conversation_type. Without a default, those in-flight tasks fail with TypeError and keep failing on retry since the kwargs are immutable. Default to DIRECT_MESSAGE so those handful of tasks classify to a single bucket rather than crashing. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
record_seer_slack_event was a passthrough to analytics.record after the event-class consolidation. Inline the call in the task and remove the wrapper. Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f082809. Configure here.
@analytics.eventclass only converts the class and sets type; it doesn't add it to EventManager._event_types. Matches the pattern used by every other event class under src/sentry/analytics/events/. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We want to add analytics for Seer Agent in Slack.
After some discussion, we decided to send the following information for each chat with the Seer Agent.
[...]id__inqueries.sentry/src/sentry/identity/services/identity/model.py
Lines 39 to 45 in eba50df