Skip to content

metrics(cleanup): Track script status and progress #101098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 8, 2025
Merged

metrics(cleanup): Track script status and progress #101098

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 43 additions & 28 deletions src/sentry/runner/commands/cleanup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,42 +226,57 @@ def is_filtered(model: type[Model]) -> bool:

deletes = models_which_use_deletions_code_path()

_run_specialized_cleanups(is_filtered, days, silent, models_attempted)
# Track timing for each deletion stage to monitor execution progress and identify bottlenecks
with metrics.timer(
"cleanup.stage", instance=router, tags={"stage": "specialized_cleanups"}
):
_run_specialized_cleanups(is_filtered, days, silent, models_attempted)

# Handle project/organization specific logic
project_id, organization_id = _handle_project_organization_cleanup(
project, organization, days, deletes
)
# Handle project/organization specific logic
project_id, organization_id = _handle_project_organization_cleanup(
project, organization, days, deletes
)
Copy link
Contributor

@cursor cursor bot Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Cleanup Timing Error Skews Metrics

The _handle_project_organization_cleanup call is incorrectly placed inside the specialized_cleanups metrics timer. As this function handles project/organization specific logic, its execution time is misattributed, leading to inaccurate performance metrics for the specialized_cleanups stage.

Fix in Cursor Fix in Web


# This does not use the deletions code path, but rather uses the BulkDeleteQuery class
# to delete records in bulk (i.e. does not need to worry about child relations)
run_bulk_query_deletes(
is_filtered,
days,
project,
project_id,
models_attempted,
)
with metrics.timer("cleanup.stage", instance=router, tags={"stage": "bulk_query_deletes"}):
run_bulk_query_deletes(
is_filtered,
days,
project,
project_id,
models_attempted,
)

run_bulk_deletes_in_deletes(
task_queue,
deletes,
is_filtered,
days,
project,
project_id,
models_attempted,
)
with metrics.timer(
"cleanup.stage", instance=router, tags={"stage": "bulk_deletes_in_deletes"}
):
run_bulk_deletes_in_deletes(
task_queue,
deletes,
is_filtered,
days,
project,
project_id,
models_attempted,
)

run_bulk_deletes_by_project(
task_queue, project, project_id, is_filtered, days, models_attempted
)
with metrics.timer(
"cleanup.stage", instance=router, tags={"stage": "bulk_deletes_by_project"}
):
run_bulk_deletes_by_project(
task_queue, project, project_id, is_filtered, days, models_attempted
)

run_bulk_deletes_by_organization(
task_queue, organization_id, is_filtered, days, models_attempted
)
with metrics.timer(
"cleanup.stage", instance=router, tags={"stage": "bulk_deletes_by_organization"}
):
run_bulk_deletes_by_organization(
task_queue, organization_id, is_filtered, days, models_attempted
)

remove_file_blobs(is_filtered, silent, models_attempted)
with metrics.timer("cleanup.stage", instance=router, tags={"stage": "file_blobs"}):
remove_file_blobs(is_filtered, silent, models_attempted)

finally:
# Shut down our pool
Expand Down
Loading