feat(web-analytics): add dagster job to clear precompute oom pins - #69969
Merged
Conversation
Contributor
|
Reviews (1): Last reviewed commit: "feat(web-analytics): add management comm..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
lricoy
changed the title
Contributor
🤖 CI report |
…-pins # Conflicts: # products/web_analytics/backend/hogql_queries/web_lazy_precompute_common.py
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Teams whose precompute inserts OOM get pinned (inserts capped to 1-day windows, self-expiring in 14 days). When the underlying pressure is gone - a cluster upscale, a stored-row cap reduction - there is no operator tool to clear a pin early, so the team pays fragmented inserts for up to two weeks.
Changes
clear_team_oom_pin,list_oom_pinned_team_ids.web_analytics_clear_precompute_oom_pins, run manually from the Dagster UI:team_id: <id>: clears that team's pinclear_all: true: clears every pin (explicit config is the confirmation)team_id+clear_alltogether fail the run without clearing anythingStarted as a management command; converted to a Dagster job so operators get run history, logs, and the UI instead of shell access to a pod.
How did you test this code?
execute_in_processcover the four config shapes: dry run clears nothing, single-team clear leaves other pins,clear_allclears everything, conflicting config fails closed. These catch the two review-flagged regressions (unguarded global clear, silently resolved conflicting modes) in the Dagster form.Automatic notifications
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Authored with Claude Code. First iteration was a management command; review added a
--yesguard and mutually exclusive flags, then Lucas asked for a Dagster job instead - the dry-run-by-default config replaces the confirmation flag, and the conflicting-config failure preserves the exclusivity guarantee.