Skip to content

feat(seer): Use Seer model_used response field for seer_model on GroupHashMetadata#111047

Merged
yuvmen merged 2 commits intomasterfrom
yuvmen/seer-grouping-record-model-used
Mar 19, 2026
Merged

feat(seer): Use Seer model_used response field for seer_model on GroupHashMetadata#111047
yuvmen merged 2 commits intomasterfrom
yuvmen/seer-grouping-record-model-used

Conversation

@yuvmen
Copy link
Member

@yuvmen yuvmen commented Mar 18, 2026

Use Seer's model_used response field to record what model Seer actually used, separately from what we requested.

When Sentry calls Seer for similarity grouping, GroupHashMetadata.seer_model was previously set from what Sentry requested. If Seer internally falls back from v2 to v1, we had no visibility into that. Seer now returns a model_used field in the response — this PR consumes it.

  • seer_model ← what Seer actually used (model_used from response, falling back to the requested version)
  • seer_latest_training_model ← what we sent (always the requested version)

Changes:

  • Add model_used: NotRequired[str] to SimilarIssuesEmbeddingsResponse
  • get_similarity_data_from_seer now returns tuple[list[SeerSimilarIssueData], str | None]
  • get_seer_similar_issues now returns a 3-tuple threading model_used through
  • maybe_check_seer_for_matching_grouphash sets seer_model from the response value

Backward compatible: model_used is parsed with .get() defaulting to None, and seer_model_used or model_version.value means behavior is identical to today if the field is absent. No migration needed.

…pHashMetadata

Record the model Seer actually used (from its response) separately from
the model we requested. This gives visibility into Seer-side fallbacks
(e.g. v2 requested but v1 used).

- Add model_used to SimilarIssuesEmbeddingsResponse type
- Return (results, model_used) tuple from get_similarity_data_from_seer
- Thread model_used through get_seer_similar_issues as 3rd return element
- Set seer_model from response (with fallback to requested version)
- Set seer_latest_training_model from what was requested

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 18, 2026
@yuvmen yuvmen requested a review from kddubey March 18, 2026 22:51
@yuvmen yuvmen marked this pull request as ready for review March 18, 2026 22:52
@yuvmen yuvmen requested review from a team as code owners March 18, 2026 22:52
Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: model_used discarded when Seer returns empty results
    • Changed return value from ([], None) to ([], model_used) at line 188 to preserve the model version when Seer returns no matches.

Create PR

Or push these changes by commenting:

@cursor push 47bcd7300d
Preview (47bcd7300d)
diff --git a/src/sentry/seer/similarity/similar_issues.py b/src/sentry/seer/similarity/similar_issues.py
--- a/src/sentry/seer/similarity/similar_issues.py
+++ b/src/sentry/seer/similarity/similar_issues.py
@@ -185,7 +185,7 @@
             sample_rate=options.get("seer.similarity.metrics_sample_rate"),
             tags={**metric_tags, "outcome": "no_similar_groups"},
         )
-        return ([], None)
+        return ([], model_used)
 
     # This may get overwritten as we process the results, but by this point we know that Seer at
     # least found *something*

This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

Backend Test Failures

Failures on 303bd12 in this run:

tests/sentry/deletions/test_grouphash.py::DeleteGroupHashTest::test_deleting_grouphash_matched_by_seerlog
tests/sentry/deletions/test_grouphash.py:60: in test_deleting_grouphash_matched_by_seer
    assert new_event.group_id == existing_group_id
E   assert 22 == 21
E    +  where 22 = <sentry.services.eventstore.models.Event at 0x7f473eb6e780: event_id=5d1b267a4c2c40839db1c2cdcdf9f316>.group_id
tests/sentry/deletions/test_grouphash.py::DeleteGroupHashTest::test_deleting_grouphash_matched_by_seer_after_unmergelog
tests/sentry/deletions/test_grouphash.py:104: in test_deleting_grouphash_matched_by_seer_after_unmerge
    assert new_event.group_id == existing_event.group.id
E   assert 30 == 29
E    +  where 30 = <sentry.services.eventstore.models.Event at 0x7fda3ffc8b50: event_id=da9678ec6b18489c8066c19d0bd5c866>.group_id
E    +  and   29 = <Group at 0x7fda45175090: id=29, project_id=4557816015618048>.id
E    +    where <Group at 0x7fda45175090: id=29, project_id=4557816015618048> = <sentry.services.eventstore.models.Event at 0x7fda3ffc8aa0: event_id=d18456c8c2c04611822494c587dc1a4b>.group
tests/sentry/deletions/test_group.py::DeleteGroupTest::test_delete_grouphashes_and_metadatalog
tests/sentry/deletions/test_group.py:322: in test_delete_grouphashes_and_metadata
    assert event_a.group_id == event_b.group_id
E   assert 19 == 20
E    +  where 19 = <sentry.services.eventstore.models.Event at 0x7f51c08e68e0: event_id=b618552c579c4350b3a5bafddb7e5ea5>.group_id
E    +  and   20 = <sentry.services.eventstore.models.Event at 0x7f51c0608680: event_id=db791883cfb741ff9a2da791afd09d37>.group_id

- Fix deletion tests that still mocked get_seer_similar_issues with
  2-tuples instead of 3-tuples
- Return model_used (instead of None) when Seer returns empty results,
  so fallback detection works even with no matches

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@yuvmen yuvmen merged commit f05ac3d into master Mar 19, 2026
76 checks passed
@yuvmen yuvmen deleted the yuvmen/seer-grouping-record-model-used branch March 19, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants