feat(autopilot): Prepare missing integration detection for experiment#107671
feat(autopilot): Prepare missing integration detection for experiment#107671vgrozdanic merged 1 commit intomasterfrom
Conversation
github-actions
bot
added
the
Scope: Backend
| ) | ||
| metrics.incr( | ||
| "autopilot.missing_sdk_integration_detector.issue_created", | ||
| tags={"project_id": str(project.id), "integration": integration}, |
There was a problem hiding this comment.
this is only temporary and it wouldn't be more than 200 projects and a couple of integration values, so no high cardinality concerns atm.
| """ | ||
| metrics.incr( | ||
| "autopilot.missing_sdk_integration_detector.run", | ||
| tags={"project_id": str(project_id)}, |
There was a problem hiding this comment.
this is only temporary and it wouldn't be more than 200 projects, so no high cardinality concerns atm.
| except Exception as e: | ||
| metrics.incr( | ||
| "autopilot.missing_sdk_integration_detector.error", | ||
| tags={"project_id": str(project.id), "error_type": type(e).__name__}, |
There was a problem hiding this comment.
this is only temporary and it wouldn't be more than 200 projects, so no high cardinality concerns atm.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| "autopilot.missing_sdk_integration_detector.issue_created", | ||
| tags={"project_id": str(project.id), "integration": integration}, | ||
| sample_rate=1.0, | ||
| ) |
There was a problem hiding this comment.
Metric emitted before issue creation succeeds
Medium Severity
The "autopilot.missing_sdk_integration_detector.issue_created" metric is emitted before create_instrumentation_issue() is called. If the issue creation fails, the code has already counted it as created, then an error metric is also emitted. This produces misleading metrics for the shadow run experiment — a single failed creation would show both an "issue_created" count and an error count. For an experiment designed to measure real-world performance through metrics and logs, this inaccuracy could skew the analysis.
…#107671) Prepares missing sdk integration detector for a shadow run experiement. We are planning to run this on 200 projects to get the feeling how often this could detect a problem in a real world traces and repositories. This will not create real issues as they will not get ingested, but we will use the metrics and logs to learn more about the performance of the detector. Closes [TET-1899: Implement shadow run for 200 pre-selected projects](https://linear.app/getsentry/issue/TET-1899/implement-shadow-run-for-200-pre-selected-projects)
…#107671) Prepares missing sdk integration detector for a shadow run experiement. We are planning to run this on 200 projects to get the feeling how often this could detect a problem in a real world traces and repositories. This will not create real issues as they will not get ingested, but we will use the metrics and logs to learn more about the performance of the detector. Closes [TET-1899: Implement shadow run for 200 pre-selected projects](https://linear.app/getsentry/issue/TET-1899/implement-shadow-run-for-200-pre-selected-projects)
2 participants
Prepares missing sdk integration detector for a shadow run experiement. We are planning to run this on 200 projects to get the feeling how often this could detect a problem in a real world traces and repositories. This will not create real issues as they will not get ingested, but we will use the metrics and logs to learn more about the performance of the detector.
Closes TET-1899: Implement shadow run for 200 pre-selected projects