feat!: implement group-level pause and resume#81
Merged
Conversation
…use_reasons column Foundation for group-level pause/resume (plan 042, step 1): - Migration 010: creates `paused_groups` table and adds `pause_reasons` bitmask column to `tasks` with backfill for existing paused rows - PauseReasons newtype with PREEMPTION/MODULE/GLOBAL/GROUP bit constants and contains/with/without/is_empty/bits/from_bits operations - TaskRecord gains `pause_reasons` field; row mapping reads it with fallback to 0 for backward compatibility
Plan 042 step 2: plumb PauseReasons through every existing pause and resume code path so multiple pause sources can coexist without stranded-task bugs. Store changes (cancel_expire.rs): - pause() now accepts a reason parameter and ORs the bit into pause_reasons - pause_pending_by_type_prefix() ORs the MODULE bit; also touches already-paused tasks so the bit accumulates - resume_paused_by_type_prefix() two-step: fully resume sole-reason tasks, clear bit from multi-reason tasks - New: preemption_paused_tasks(), resume_preempted(), clear_pause_bit() Scheduler changes: - cancel_pause_emit (dispatch.rs) accepts reason; callers pass PREEMPTION, MODULE, or GLOBAL respectively - Auto-resume (run_loop.rs) scoped to PREEMPTION bit only via preemption_paused_tasks + resume_preempted - resume_all (control.rs) clears GLOBAL bit from DB Schema: - pause_reasons column added to 001_tasks.sql base schema - Migration 010 registered in migrate() with idempotent handling Tests: 5 new store tests covering bitmask accumulation, preemption filtering, clear_pause_bit, and module pause/resume with assertions on pause_reasons values.
… helpers Group pause state management (pause_group_state, resume_group_state, paused_groups, is_group_paused, groups_due_for_resume), bulk task pause/resume by group key with bitmask integration, and per-group pending/paused count queries. Pure store additions exercised by tests.
Wire SchedulerInner with in-memory paused_groups set, builder startup loading from DB, GroupPaused/GroupResumed events, PausedGroupInfo in snapshots, and ActiveTaskMap::pause_group for cancelling running tasks.
- pause_group / resume_group / pause_group_until public API in control.rs - Gate admission rejects tasks whose group is paused - Finalizer deferral: re-queue parent finalize when group is paused - Time-boxed auto-resume checked every ~5s in the run loop - maybe_restore_fast_dispatch re-evaluates when last group is resumed
…utcome change
Wire group-pause awareness into submit, dependency resolution, recurring
instance creation, and task-completion unblocking so newly-eligible tasks
are downgraded to paused when their group is paused.
Breaking: SubmitOutcome::Inserted is now a struct with { id, group_paused }
instead of a tuple variant.
Expose pause_group / resume_group / is_group_paused / paused_groups on
DomainHandle and ModuleHandle, delegating to the scheduler.
…p 6) Cover submit-to-paused-group, recurring next-instance downgrade, blocked→paused transition, multi-reason pause interaction, and DomainHandle delegation. Also remove duplicated ALTER TABLE from migration 010 (now lives in the earlier pause_reasons migration).
deepjoy
force-pushed
the
pause-all-resume-all
branch
from
d3f9b67 to
2756398
Compare
Merged
Contributor
Benchmark ComparisonClick to expand |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PauseReasonsbitmask (PREEMPTION | MODULE | GLOBAL | GROUP) so multiplepause sources can coexist without stranding tasks — a task only resumes when
all reasons are cleared. Retrofit every existing pause/resume path (preemption,
module, global) to use bitmask operations instead of bare status flips.
paused_groupsSQLite table andpause_reasonscolumn ontasks, within-memory
HashSetmirror for fast gate checks during dispatch.Scheduler::pause_group/resume_group/pause_group_untilwithfull lifecycle: persist state → update in-memory set → pause pending tasks →
cancel running tasks → emit
GroupPaused/GroupResumedevents. Time-boxedpauses auto-resume via a throttled (5 s) run-loop check.
but inserted directly as paused with the GROUP bit. Recurring next-instances and
blocked→pending transitions also check
paused_groupsand downgrade accordingly.DomainHandleandModuleHandle(delegates to scheduler).downgrade, blocked→paused transition, multi-reason interaction, and handle delegation.
Closes #36
Breaking changes
SubmitOutcome::Inserted(id)→SubmitOutcome::Inserted { id, group_paused }—callers must update pattern matches.
TaskStore::pause(id)now requires aPauseReasonsargument:pause(id, reason).PauseReasonsandPausedGroupInfoadded to public re-exports.