Cancel available runs

[taylordowns2000]

Description

Closes #1622 by allowing users to cancel available runs, essentially removing them from the queue (:available -> :cancelled) and changing the state of their associate work orders from :pending -> :cancelled.

Action availability by work order state:

WO State	Available Actions
:pending (runs :available)	Cancel
:running	None (all disabled)
All final states (:success, :failed, :crashed, :killed, :exception, :lost, :cancelled, :rejected)	Retry, Retry from

Validation steps

(Probably easiest to start your server with RTM=false iex -S mix phx.server so runs don't run.)

1. Enqueue a lot of runs
2. Cancel all runs related to a work order one by clicking on the work order row button
3. Cancel a single run by clicking on the run item button (both in the list and on the run detail page)
4. Select multiple work orders and cancel all runs for those selected by clicking on the "Cancel" multi-item action button.
5. Select work orders with mixed states and note that you can't retry or cancel them.
6. Select work orders with only running states and note that you can't retry or cancel them.
7. Select work orders with non-running, non-pending states and note that you can retry them.
8. Select work orders with pending states and note that you can cancel them.
9. Select a work order with pending state, open the cancel modal, let it start running, and then confirm the cancellation; note it doesn't cancel it.
10. Navigate to the run detail page for an enqueued run. Cancel it via the button.

Additional notes for the reviewer

1. ‼️ Yes, I've also removed FifoQueue.dequeue and Run.dequeue since these functions deleted runs! They were in the code base, but (thankfully) never used. You can't delete runs.
2. ⚠️ Note that Stu and I discussed "requeueing" runs, but in the end I realized that this breaks the "one-way state machine" that's fundamental to our audit trail. Left it out for now to obey the current rules; we can add it as a feature later with more thought to audit compliance.

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

• I have used Claude Code
• I have used another model
• I have not used AI

You can read more details in our
Responsible AI Policy

Pre-submission checklist

• I have performed an AI review of my code (we recommend using /review
  with Claude Code)
• I have implemented and tested all related authorization policies.
  (e.g., :owner, :admin, :editor, :viewer)
• I have updated the changelog.
• I have ticked a box in "AI usage" in this PR

[codecov]

Codecov Report

❌ Patch coverage is 93.91892% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.53%. Comparing base (2a78d3e) to head (8e31765).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/lightning_web/live/run_live/index.ex	89.36%	5 Missing ⚠️
lib/lightning/runs.ex	94.11%	1 Missing ⚠️
lib/lightning/runs/run.ex	0.00%	1 Missing ⚠️
lib/lightning/work_orders.ex	88.88%	1 Missing ⚠️
lib/lightning_web/live/run_live/show.ex	91.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4578      +/-   ##
==========================================
- Coverage   89.54%   89.53%   -0.02%     
==========================================
  Files         441      443       +2     
  Lines       21249    21375     +126     
==========================================
+ Hits        19028    19138     +110     
- Misses       2221     2237      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[brandonjackson]

Thanks. I approve, but would like to request that before you wrap up, you create a separate ticket describing the work that is needed to fully support the cancel operation for jobs that are both in the queue and running. To me this only gets us halfway.

[taylordowns2000]

Thanks. I approve, but would like to request that before you wrap up, you create a separate ticket describing the work that is needed to fully support the cancel operation for jobs that are both in the queue and running. To me this only gets us halfway.

So far this has been tracked here: #4574

I feel like that's what we need. (And we could prio this fairly high, if you're keen to have the "whole" solution sooner rather than later.)

[elias-ba]

Hey @taylordowns2000, great work on this - the concurrency handling with FOR UPDATE SKIP LOCKED is really solid and the test coverage is thorough. The main concern is the authorization gap on the cancel-run handlers in both index.ex and show.ex - since Runs.get/1 is unscoped, a user could potentially cancel a run from another project. The rest of the comments are minor nits.

[taylordowns2000]

This is great, @elias-ba . I think your concerns have been addressed here:

#	Issue	Fix	Files
1	Auth gap — Runs.get/1 unscoped	Added Runs.get_for_project/2 that joins through work_order → workflow to verify project ownership	runs.ex
2	with result <- always matches	Replaced with with if/case so {:error, _} is properly handled	index.ex
3	Oban insert failure ignored	Pattern match on cancel_many_async return; propagate errors	index.ex
4	Duplicated cancel-run logic	Extracted CancelHelper.cancel_run/2 used by both LiveViews	cancel_helper.ex, index.ex, show.ex
5	Modal title inconsistency	Changed "Cancel Work Orders" → "Cancel Pending Runs"	components.ex

[github-actions]

Security Review

⚠️ Automated security review did not complete.

Claude hit the max-turns limit or encountered an error before posting findings.
A manual review of S0 (project-scoped data access), S1 (authorization policies),
and S2 (audit trail coverage) is recommended for this PR.

See the workflow run for details.

[elias-ba]

Hey @taylordowns2000 thanks for the changes and for this huge work. All is looking good to me 👏🏽 🎉

I pushed two small fixes:

• Added missing {:error, _} clause in the single WO cancel handler (index.ex)
• Updated cancel_many typespec to include {:error, any()} (was failing dialyzer)

One question (cc @stuartc): the "selected" cancel path runs synchronously in the LiveView process, while rerun always goes through Oban. I assume that's intentional since cancel is just a single UPDATE query bounded by page size - but wanted to confirm: was there a deliberate decision to keep it sync for small batches, or should we align it with the rerun pattern for consistency?

[taylordowns2000]

One question (cc @stuartc): the "selected" cancel path runs synchronously in the LiveView process, while rerun always goes through Oban. I assume that's intentional since cancel is just a single UPDATE query bounded by page size - but wanted to confirm: was there a deliberate decision to keep it sync for small batches, or should we align it with the rerun pattern for consistency?

It was a specific decision to keep it sync for small batches. To my knowledge, Stu was not part of this decision. Happy for you to go wherever you want with it depending on your/Brandon's/Stu's cost:benefit analysis.

[elias-ba]

Thanks for the quick response, Taylor. That makes sense to me - a bounded sync UPDATE is the right call here. Merging! 🚀