Skip to content

Conversation

@yuvmen
Copy link
Member

@yuvmen yuvmen commented Nov 20, 2025

Our previous attempt (#103495) at correcting processing timeouts for MonitorCheckIn deletions failed , and introduced integrity errors because MonitorCheckIn cant be bulk deleted without manually removing child dependancies (no CASCADE in db FKs).
Added deletion of MonitorIncident directly before attempting to bulk delete MonitorCheckIns, basically removing the children of MonitorCheckins before deleting them.

@yuvmen yuvmen requested a review from wedamija November 20, 2025 22:50
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 20, 2025
Our previous attmept at correcting processing timeouts for MonitorCheckIn deletions failed, and
introduced integrity errors because MonitorCheckIn cant be bulk deleted without manually removing
child dependancies (no CASCADE in db FKs). Instead we add proper deletion tasks to the different
models.
Note we also use `get_child_relations_bulk` here which is actually an old method but not in use
in our codebase today. Using it though we might be able to speed up querying for child relations
of models with complex relations, when they cant just be outright bulk deleted using BulkModelDeletionTask.
Copy link
Member

@wedamija wedamija left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think maybe we should cascade this like

Monitor -> MonitorIncident -> MonitorCheckin rather than Monitor -> MonitorCheckin -> MonitorIncident

If we delete the MonitorIncident rows first, then the checkins won't have to cascade delete to the MonitorIncident table and you can use the bulk deletion stuff.

This also helps with the MonitorEnvironment deletions, since they won't be referenced by MonitorIncident either

Comment on lines 7 to 29
class MonitorCheckInDeletionTask(ModelDeletionTask[MonitorCheckIn]):
def get_child_relations_bulk(
self, instance_list: Sequence[MonitorCheckIn]
) -> list[BaseRelation]:
"""
Return bulk child relations for MonitorCheckIn deletion.
Uses __in queries to efficiently delete MonitorIncidents that reference these check-ins.
"""
from sentry.monitors import models

checkin_ids = [ci.id for ci in instance_list]

return [
ModelRelation(
models.MonitorIncident,
{"starting_checkin_id__in": checkin_ids},
),
ModelRelation(
models.MonitorIncident,
{"resolving_checkin_id__in": checkin_ids},
),
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove this if we cascade in the other direction

…itor and monitor_environment flows

allow MonitorIncident to be deleted not in bulk - there shouldnt be so many to cause problems without it
@yuvmen
Copy link
Member Author

yuvmen commented Nov 20, 2025

Refactored according to discussion with Dan, will leverage two facts:

  1. Child relations are deleted in order, and so we can treat the deletions of Monitor and MonitorEnvironment as lists on how to delete the entire tree beneath them in order.
  2. We can delete MonitorIncident non bulk, since there shouldnt be that many of them, and this way we can handle the fact the leaf beneath them - MonitorEnvBrokenDetection has no key to the Monitor/MonitorEnvironment to be deleted from the tasks directly.

@yuvmen yuvmen enabled auto-merge (squash) November 20, 2025 23:50
@yuvmen yuvmen merged commit e9e5959 into master Nov 21, 2025
66 checks passed
@yuvmen yuvmen deleted the yuvmen/fix-monitorcheckin-deletion branch November 21, 2025 00:00
Gust-feng added a commit to Gust-feng/sentry that referenced this pull request Nov 21, 2025
yuvmen added a commit that referenced this pull request Nov 24, 2025
In #103786, we refactored the way MonitorCheckIn gets deleted through
Monitor and MonitorEnvironment deletion, but in the process hurt direct
MonitorCheckIn cleanup which also happens daily. Added a task to handle
it and set the relations to use Bulk Deletion instead of defaulting to
Bulk.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants