Skip to content

feat(perf_issues): Add post_process pipeline for performance issues#39895

Merged
wedamija merged 3 commits into
masterfrom
danf/perf_issues_post_process_pipeline
Oct 12, 2022
Merged

feat(perf_issues): Add post_process pipeline for performance issues#39895
wedamija merged 3 commits into
masterfrom
danf/perf_issues_post_process_pipeline

Conversation

@wedamija

Copy link
Copy Markdown
Member

This adds in a post process pipeline for performance issues. We don't need most steps, just snoozes, inbox and processing issue alert rules.

We might want to add assignment in the future if we update the system to be able to match on parts of the transaction. Same for suspect commits if we can figure out how to make tha twork.

update_existing_attachments doesn't make sense for perf issues since there can be multiple issues per transaction, and this function assumes that there's only one issue. We don't have attachments to worry about here, afaik, so if this requirement comes up we can add it later.

We'll want to enable process_plugins in the future, but for now we can keep it disabled to limit any noise.

We'll probably want to enable process_service_hooks and process_resource_change_bounds later once we understand if we want to send them to the integrations.

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 11, 2022
Comment thread tests/sentry/tasks/test_post_process.py Outdated
Comment on lines 1017 to 1047

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent a while trying to generate these cases dynamically but couldn't figure it out in a reasonable amount of time. It might be easier using pure pytest instead of unittest

@ceorourke

ceorourke commented Oct 11, 2022

Copy link
Copy Markdown
Member

How does process_resource_change_bounds work with process_resource_change_bound (singular)? I know the latter is used heavily in sentry apps, is that what you mean by "once we understand if we want to send them to the integrations"? Also maybe worth keeping a running list somewhere of all the stuff we've disabled for perf issues that we may want to enable in the future since I'm losing track.

@wedamija

Copy link
Copy Markdown
Member Author

How does process_resource_change_bounds work with process_resource_change_bound (singular)? I know the latter is used heavily in sentry apps, is that what you mean by "once we understand if we want to send them to the integrations"? Also maybe worth keeping a running list somewhere of all the stuff we've disabled for perf issues that we may want to enable in the future since I'm losing track.

That's kind of what we need to work out. The code here https://github.com/getsentry/sentry/blob/d86d5b669653d0a025813f7a703d43b7d8603f14/src/sentry/tasks/post_process.py#L688-L703 looks like:

I'm also not really sure if we even want to send performance issues here at all, but I don't know what the downstream effects are there.

Comment thread tests/sentry/tasks/test_post_process.py Outdated
Comment thread tests/sentry/tasks/test_post_process.py Outdated
Comment thread tests/sentry/tasks/test_post_process.py Outdated
@ceorourke

Copy link
Copy Markdown
Member

@wedamija re: process_resource_change_bound downstream stuff - I am kinda nervous that letting transactions and performance issues through could interfere with existing integrations built on the integration platform, but we can have a larger conversation about it if we decide we want to head in that direction. Mostly I'm not sure if suddenly sending an unexpected event or issue type would blow up code other people have written when they relied on one format 😬 I was thinking about this today and if we want to allow it maybe that can be opt in but it'd have to be accompanied by a bunch of documentation about the schema (if that differs). Probably not worth addressing anytime super soon unless we get requests for it.

Comment thread tests/sentry/tasks/test_post_process.py Outdated
@wedamija

Copy link
Copy Markdown
Member Author

@wedamija re: process_resource_change_bound downstream stuff - I am kinda nervous that letting transactions and performance issues through could interfere with existing integrations built on the integration platform, but we can have a larger conversation about it if we decide we want to head in that direction. Mostly I'm not sure if suddenly sending an unexpected event or issue type would blow up code other people have written when they relied on one format 😬 I was thinking about this today and if we want to allow it maybe that can be opt in but it'd have to be accompanied by a bunch of documentation about the schema (if that differs). Probably not worth addressing anytime super soon unless we get requests for it.

That's kind of how I feel too. I think that for sure we don't want to send transactions here, since they're too high volume. Allowing users to control which issue types are sent probably makes sense both here and for service hooks, although I know nothing about how users actually configure these. I'm ok with putting this off as well

This adds in a post process pipeline for performance issues. We don't need most steps, just snoozes,
inbox and processing issue alert rules.

We might want to add assignment in the future if we update the system to be able to match on parts
of the transaction. Same for suspect commits if we can figure out how to make tha twork.

`update_existing_attachments` doesn't make sense for perf issues since there can be multiple issues
per transaction, and this function assumes that there's only one issue. We don't have attachments to
worry about here, afaik, so if this requirement comes up we can add it later.

We'll want to enable `process_plugins` in the future, but for now we can keep it disabled to limit
any noise.

We'll probably want to enable `process_service_hooks` and `process_resource_change_bounds` later
once we understand if we want to send them to the integrations.
@wedamija wedamija force-pushed the danf/perf_issues_post_process_pipeline branch from d86d5b6 to c1b3615 Compare October 12, 2022 00:40
Comment on lines +1058 to +1069
group_states = (
[
{
"id": group_id,
"is_new": is_new,
"is_regression": is_regression,
"is_new_group_environment": is_new_group_environment,
}
]
if group_id
else None
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, any reason not to add group_states as an optional arg on call_post_process_group.

The test PostProcessGroupPerformanceTest.test_full_pipeline_with_group_states passes in two GroupState objects but this won't support that.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we could allow group_states to be passed as an override. tbh I was just calling it directly from that test since it's a special case

@wedamija wedamija merged commit c9bbf4d into master Oct 12, 2022
@wedamija wedamija deleted the danf/perf_issues_post_process_pipeline branch October 12, 2022 18:25
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants