Skip to content

fix: bound action scheduler retention queries - #3087

Merged
chubes4 merged 2 commits into
mainfrom
fix-3084-bounded-retention
Aug 9, 2026
Merged

fix: bound action scheduler retention queries#3087
chubes4 merged 2 commits into
mainfrom
fix-3084-bounded-retention

Conversation

@chubes4

@chubes4 chubes4 commented Aug 9, 2026

Copy link
Copy Markdown
Member

Summary

  • select bounded terminal action-ID batches through Action Scheduler's shipped indexes before touching the oversized logs table
  • delete child logs with both an action-ID boundary and row limit, then delete only the selected parent actions
  • bound row-ceiling probes per indexed status arm and replace apply-path InnoDB COUNT(*) scans with metadata estimates
  • isolate retention smoke filter overrides so the same query-contract coverage passes in pure PHP and real WordPress

Root cause

The old log deletion put actionscheduler_logs on the driving side of a join and applied LIMIT only to the result. On the Events shape, MySQL could examine a substantial portion of the 24.6M-row logs table before finding a batch, so one SQL statement could consume minutes before PHP could observe its wall-clock deadline. The row-ceiling probe also sorted by last_attempt_gmt for a hook even though Action Scheduler has no (hook, last_attempt_gmt) index, and the post-pass guardrail ran exact COUNT(*) scans over both oversized InnoDB tables.

Algorithm and scale safety

Each retention iteration now:

  1. Selects at most the configured batch of terminal action IDs.
  2. Uses status_last_attempt_gmt for global windows or hook_status_scheduled_date_gmt for per-hook windows.
  3. Deletes logs only where action_id IN (<bounded IDs>), with an additional row LIMIT.
  4. Deletes only that bounded parent-ID set after its logs are drained.
  5. Rechecks the shared iteration and wall-clock budgets between statements.

The row-ceiling cutoff combines three status-specific index scans, each capped at max_rows + 1, before applying the cross-status offset. This preserves the existing combined ceiling semantics while bounding the filesort input. InnoDB table-size guardrails now read information_schema.TABLES.TABLE_ROWS; SQLite retains exact counts.

Locking, indexing, and compatibility

  • Log lookups use Action Scheduler's actionscheduler_logs.action_id index and lock no more than one configured log batch per statement.
  • Parent deletes use the actions primary key and lock no more than the selected action batch.
  • Global and per-hook selectors force indexes shipped by the supported Action Scheduler ^3.9 schema; no production schema migration or table rebuild is introduced.
  • The native Action Scheduler cleaner remains unchanged as the pre-claim backstop. It does not replace Data Machine's failed-action, per-hook, ceiling, diagnostic, or catch-up behavior.
  • Optional OPTIMIZE TABLE behavior remains opt-in and unchanged.

Verification

  • php tests/retention-action-scheduler-batching-smoke.php - 39 assertions passed
  • php tests/action-scheduler-native-retention-smoke.php - 10 assertions passed
  • php -l inc/Engine/AI/System/Tasks/Retention/RetentionCleanup.php - passed
  • php -l tests/retention-action-scheduler-batching-smoke.php - passed
  • homeboy review data-machine lint --path /var/lib/datamachine/workspace/data-machine@fix-3084-bounded-retention --changed-only --summary - passed, no baseline drift
  • homeboy review test data-machine --path /var/lib/datamachine/workspace/data-machine@fix-3084-bounded-retention --changed-since origin/main --summary - real-WordPress smoke passed, 1 passed / 0 failed
  • Homeboy PR audit - no introduced findings in touched scope
  • Production-schema EXPLAIN only: global selector used status_last_attempt_gmt; per-hook selector and all ceiling arms used hook_status_scheduled_date_gmt

The first Homeboy umbrella run exposed inherited real-WordPress filter callbacks in the smoke fixture. The fixture now installs deterministic late-priority overrides and restores only its own callbacks; the final clean-checkout Homeboy test passes.

Operational follow-up

The existing Events backlog still requires repeated bounded catch-up passes; this change intentionally favors incremental progress over a single large drain. Operators should keep OPTIMIZE TABLE disabled during the drain unless they separately provision the lock and temporary disk space for a rebuild. Metadata row estimates may lag exact counts, so verify backlog progress through deleted-row results and periodic table-size observations rather than expecting per-pass file-size shrinkage.

Fixes #3084

@chubes4
chubes4 merged commit 778c821 into main Aug 9, 2026
14 checks passed
@chubes4
chubes4 deleted the fix-3084-bounded-retention branch August 9, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: bound retention queries on oversized Action Scheduler tables

1 participant