feat(server): cancel a running job by id#169
Merged
Merged
Conversation
A CLI-submitted job could not be stopped once launched: the work runs as a fiber forked into the server's layer scope, so Ctrl-C in the CLI only detached the log follow while the server-side job ran to completion. Unlike an app run directly (IntelliJ Stop = root-fiber interrupt), there was no channel to reach that fiber. This adds cancellation by id. JobRunner now retains a per-process handle to each job fiber (`runningFibers`, a Promise registered *before* the fork so `release`'s de-register can't race ahead of the register). `cancel(id)` registers the id in `cancelRequested`, then `interruptFork`s the fiber so the HTTP handler returns promptly. The job's own `.onInterrupt` records the new `Cancelled` terminal state -- but ONLY if its id is in `cancelRequested`. That gate is load-bearing: `layerScope` interrupts every in-flight job on server shutdown too, and those must stay `Running` so the next boot's `markOrphansAsFailed` records them as `Failed`/"Service restarted", not a spurious "Cancelled by operator". The interrupt-observing write also fixes a latent bug where an interrupted job bypassed `foldZIO` and left its row stuck at `Running`. Cancellation is best-effort/async: an in-flight uninterruptible blocking JDBC statement runs to completion before the interrupt lands, and `markCancelled`'s `WHERE status = Running` guard makes the terminal write a no-op if the job reached a terminal state first. `cancel` is per-process, so it inherits the single-server-per-DB assumption (#110/#111); an absent local fiber returns 404 rather than blindly marking a row that may be Running elsewhere. Surface: `POST /api/jobs/{jobId}/cancel` (200 CancelResult / 404) and the `ccas cancel <job-id>` subcommand. A job cancelled out from under an active `ccas logs` follow renders "was cancelled" and exits non-zero. Also unify all `completed_at` writers (`updateStatus`, `markCancelled`, `markOrphansAsFailed`) on a caller-supplied `Clock.instant` instead of SQL `NOW()`, so every job timestamp comes from the one testable app clock and is coherent with `started_at` (no app-vs-DB host skew). Tested: cancel interrupts a live job -> Cancelled (real JobRunner.live); the shutdown-style self-interrupt stays Running; markCancelled guard; the cancel route 200/404; CLI parse + completion drift; follow exit code; and an end-to-end wire smoke (TestJobCancelWire) driving the real CcasApiClient over a real HTTP server against real JobRoutes + JobRunner.live. Fixes #168 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 17, 2026
Sootopolis
added a commit
that referenced
this pull request
Jul 19, 2026
#169 shipped cancel-by-id but Ctrl-C during a follow still only detached the log stream while the server job ran on. That is the surprising behaviour: for a mistyped club/alias you want out fast, and Ctrl-C == stop is the universal CLI expectation. So a follow interrupt now cancels the server job by default. No prompt -- the keystroke is the intent, and prompting would mean reading stdin inside an interrupt finalizer as the process tears down (fragile). It is also cheap to be wrong: CCAS jobs are idempotent/resumable, so an accidental cancel just stops early and the re-run resumes; a cancelled recruit already leaves candidates Deferred, so cancel never burns invites. The `.onInterrupt` is attached OUTSIDE `.timeout`, so a genuine `maxWait` expiry (which returns None on the success path) still detaches rather than cancelling -- only a real fiber interruption fires the cancel POST, which is `.ignore`d so a 404 (job already terminal) or a teardown transport error can't turn a clean Ctrl-C into a stack trace. The notice routes through the follow's own sink -- render-lock-safe `renderer.logLine` in bars mode, stderr otherwise -- so it can't tear a live progress bar (the finalizer runs before withBars' `renderer.clear`). For "keep the data fresh, don't watch" (notably history, whose output is not user-facing -- StatsApp consumes it), add `--detach` on membership/history/ stats: submit, print the job id, return; reattach with `ccas logs <id>`, which replays the full log from offset 0 so nothing is lost. Not offered on recruit (its invite delivery/confirmation needs the follow) or logs (which IS the follow). Making Ctrl-C-cancel easy exposes a rough edge: a resubmit of the same (kind, club) that races the in-flight cancel would hit the unique-running index and read the baffling "already running" for a job the operator just killed -- the fiber flips to Cancelled only after its current blocking statement finishes. `submit` now checks `cancelRequested` and, for a blocker that is being cancelled, returns "finishing cancellation -- retry in a moment" so the collision reads as self-resolving. Tested: interrupt-during-follow fires the cancel POST and interrupt-during the post-follow recruit confirm does not (stub follower); `--detach` parses and defaults false on the three commands; a resubmit racing an in-flight cancel gets the retry wording (real JobRunner.live); and an end-to-end wire test drives the real JobFollower against a real HTTP server, interrupts it once the follow fiber is Suspended, and asserts the job reaches Cancelled -- proving the finalizer's POST lands over real HTTP during teardown. Completions regenerated (TestCcasCompletion drift guard). Fixes #170 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sootopolis
added a commit
that referenced
this pull request
Jul 19, 2026
#169 shipped cancel-by-id but Ctrl-C during a follow still only detached the log stream while the server job ran on. That is the surprising behaviour: for a mistyped club/alias you want out fast, and Ctrl-C == stop is the universal CLI expectation. So a follow interrupt now cancels the server job by default. No prompt -- the keystroke is the intent, and prompting would mean reading stdin inside an interrupt finalizer as the process tears down (fragile). It is also cheap to be wrong: CCAS jobs are idempotent/resumable, so an accidental cancel just stops early and the re-run resumes; a cancelled recruit already leaves candidates Deferred, so cancel never burns invites. The `.onInterrupt` is attached OUTSIDE `.timeout`, so a genuine `maxWait` expiry (which returns None on the success path) still detaches rather than cancelling -- only a real fiber interruption fires the cancel POST, which is `.ignore`d so a 404 (job already terminal) or a teardown transport error can't turn a clean Ctrl-C into a stack trace. The notice routes through the follow's own sink -- render-lock-safe `renderer.logLine` in bars mode, stderr otherwise -- so it can't tear a live progress bar (the finalizer runs before withBars' `renderer.clear`). For "keep the data fresh, don't watch" (notably history, whose output is not user-facing -- StatsApp consumes it), add `--detach` on membership/history/ stats: submit, print the job id, return; reattach with `ccas logs <id>`, which replays the full log from offset 0 so nothing is lost. Not offered on recruit (its invite delivery/confirmation needs the follow) or logs (which IS the follow). Making Ctrl-C-cancel easy exposes a rough edge: a resubmit of the same (kind, club) that races the in-flight cancel would hit the unique-running index and read the baffling "already running" for a job the operator just killed -- the fiber flips to Cancelled only after its current blocking statement finishes. `submit` now checks `cancelRequested` and, for a blocker that is being cancelled, returns "finishing cancellation -- retry in a moment" so the collision reads as self-resolving. Tested: interrupt-during-follow fires the cancel POST and interrupt-during the post-follow recruit confirm does not (stub follower); `--detach` parses and defaults false on the three commands; a resubmit racing an in-flight cancel gets the retry wording (real JobRunner.live); and an end-to-end wire test drives the real JobFollower against a real HTTP server, interrupts it once the follow fiber is Suspended, and asserts the job reaches Cancelled -- proving the finalizer's POST lands over real HTTP during teardown. Completions regenerated (TestCcasCompletion drift guard). Fixes #170 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds cancellation of a running job by id —
POST /api/jobs/{jobId}/canceland theccas cancel <job-id>CLI subcommand.A CLI-submitted job couldn't be stopped once launched: the work runs as a fiber forked into the server's layer scope, so Ctrl-C in the CLI only detached the log follow while the server-side job ran to completion. (Contrast: an app run directly in IntelliJ is the root fiber, so Stop = interrupt.) There was no channel to reach that fiber — this adds one.
Mechanism
JobRunnerretains a per-process handle to each job fiber (runningFibers, aPromiseregistered before the fork sorelease's de-register can't race the register).cancel(id)registers the id incancelRequested, theninterruptForks the fiber (handler returns promptly). The job's own.onInterruptrecords the newCancelledstate only if its id is incancelRequested.layerScopeinterrupts every in-flight job on server shutdown too — those stayRunningso the next boot'smarkOrphansAsFailedrecords them asFailed/"Service restarted", not a spurious "Cancelled by operator". The interrupt-observing write also fixes a latent bug where an interrupted job bypassedfoldZIOand left its row stuck atRunning.markCancelled'sWHERE status = Runningguard makes the write a no-op if the job reached a terminal state first. Per-process, so it inherits the single-server-per-DB assumption (JobRunner orphan-marking has no instance ownership — a 2nd server kills the 1st's live jobs #110/Rate limiter is per-process: N instances = N× the Chess.com per-IP request rate #111); an absent local fiber → 404, never a blind mark.ccas logsfollow renders "was cancelled" and exits non-zero.Also unifies all
completed_atwriters (updateStatus,markCancelled,markOrphansAsFailed) on a caller-suppliedClock.instantinstead of SQLNOW(), so every job timestamp comes from the one testable app clock, coherent withstarted_at(no app-vs-DB host skew).Docs (README + CLAUDE.md) and the committed bash completion updated.
Fixes
Fixes #168.
Testing
sbt test— 1059 pass. New coverage:cancelinterrupts a live job →Cancelled, against the realJobRunner.live(realforkIn/interrupt/onInterrupt, real DB).Running— proves thecancelRequestedgate.markCancelledRunning-only guard (SQL); cancel route 200/404; CLI parse; completion drift; follow exit code.TestJobCancelWire): a real zio-http server on an ephemeral port serving realJobRoutes, driven by the realCcasApiClientover loopback — fullccas cancelpath incl. the real fiber interrupt and the 404 branch.Follow-up (separate PR)
Ctrl-C-during-
ccas logs-follow → optional prompt-to-cancel (the closest analog to the IntelliJ graceful-stop flow), gated so an accidental Ctrl-C doesn't kill a long run.🤖 Generated with Claude Code