Distinguish closed-but-unmerged PRs from shipped work on the kanban

[aptracebloc]

Confirmed the problem and mapped the auto-rules. Here's the lay of the land and a few options:

Why it happens

Your project has 6 active workflows. The relevant ones:

#	Workflow	Trigger	Action
7	Item closed	any issue/PR closed	sets Status = Done
8	Pull request merged	PR merged	sets Status = Done

Workflow #7 fires regardless of whether a PR was merged or just closed — so abandoned/superseded PRs land in Done alongside shipped work. (Quick scan: at least 18 closed-but-unmerged PRs are currently sitting in Done — including some of yours like .github#1, tracebloc-py-package#40, and the dependabot bumps backend#578-588.)

Issues have the same problem in two flavors: closed-as-completed vs closed-as-not_planned both also land in Done.

Current Status options: Backlog → Ready → In progress → In review → Validation → Done.

Options

Option A — Add a "Cancelled" status, repurpose built-ins (lowest effort)

1. Add Status option Cancelled (or Won't ship).
2. Change workflow #7 "Item closed" → set Status = Cancelled.
3. Keep workflow ci: add reusable set-pr-status workflow #8 "Pull request merged" → set Status = Done.

How it routes:

• ✅ PR merged → #7 sets Cancelled, then ci: add reusable set-pr-status workflow #8 overrides to Done
• ✅ PR closed unmerged → #7 sets Cancelled
• ❌ Issue closed-as-completed → ends up in Cancelled (wrong)
• ✅ Issue closed-as-not_planned → Cancelled

Works perfectly for PRs (~98% of your board today). Mis-routes completed issues — survivable but not great. Also depends on workflow #8 firing after #7, which is the observed behavior but not formally guaranteed.

Option B — Custom GitHub Action that routes properly (cleanest)

1. Add Status option Cancelled.
2. Disable built-in workflow #7 ("Item closed").
3. Add a small repo-level Action (in .github) listening to pull_request: closed and issues: closed, which calls the GraphQL updateProjectV2ItemFieldValue mutation to set:
   • PR merged → Done
   • PR closed unmerged → Cancelled
   • Issue state_reason: completed → Done
   • Issue state_reason: not_planned → Cancelled

Handles all four cases correctly. ~50 lines of Action code. Costs you maintaining one workflow file, but you already have .github infrastructure for this kind of thing (advance-deploy-env, set-pr-status are exactly this shape).

Option C — Add a separate "Outcome" field, leave Status alone

Add a new single-select field Outcome with values Shipped / Cancelled / Superseded, populated by an Action on close. Status stays as-is (everything closed = Done), but views can group/filter by Outcome.

Less disruptive to existing views and saved filters, but the kanban swimlanes still mix shipped and abandoned work — you have to remember to add an Outcome filter to "what shipped" reports.

Option D — View-only filter (no schema change)

Create a saved view "Done (shipped)" that filters to status:Done is:merged. Cheap, but doesn't change the underlying truth — abandoned PRs still pollute counts and most views.

My recommendation

Option B, with Option A as a fast interim if you want it landed today. Option A gets you 95% of the value in a 2-minute config change; Option B is the proper fix and is in keeping with the .github reusable-workflow pattern you've already been rolling out across repos.

Want me to draft the Action for Option B, or just flip the workflow for Option A?

[LukasWodka]

Thanks for the diagnosis — solid breakdown. Status update from this morning:

What's done now (Option A interim)

• ✅ Cancelled Status option added (RED) — appears between Done and the end of the list
• ✅ Backfilled 19 closed-not-merged PRs that were sitting in Done — now correctly in Cancelled (kanban auto-add bumps · my own .github#1 · CLAUDE.md aborts · the Dependabot bumps you mentioned · etc.)
• ✅ Backfilled 107 items missing Squad/Area classification (the gap we just hit on this issue itself — it's now properly tagged DevEx · CI / DevEx · Tech-debt · P2 and visible in your views)
• ✅ PR ci: add cron auto-classifier (Squad / Area / Deploy environment) #16 opened: cron-based auto-classifier so the Squad/Area gap stays closed automatically — runs every 10 min, idempotent

What's still needed (Option B — your offer)

• One UI step Lukas owes: Workflows page → Item closed → change target Status from Done to Cancelled. With the new option in place, this is a 30-second flip and gets us 95% of your Option A behavior.
• Your proposed Option B (custom Action listening on pull_request:closed + issues:closed with state_reason routing) — please go ahead and draft it. Same pattern as the existing reusable workflows in tracebloc/.github/.github/workflows/ (advance-deploy-env.yml, set-pr-status.yml, customer-priority-bump.yml). Take a look at those for the shape.
• Once your Action is in, the temporary Item closed UI flip can be reverted (your Action handles all four cases correctly: PR merged → Done · PR closed unmerged → Cancelled · Issue completed → Done · Issue not_planned → Cancelled).

Assigning you. The kanban issue #14 is now Status: Ready and is yours to drive. Ping back if you hit anything blocking or want to discuss alternatives.

Thanks again for the thorough writeup.

[LukasWodka]

Update — went ahead and shipped Option B end-to-end this morning rather than waiting:

What landed

• ✅ Reusable workflow .github/workflows/kanban-closure-router.yml — already on main of tracebloc/.github (PRs #17 + #18 merged)
• ✅ 16 caller PRs opened across the active repos, all assigned to Asad for batch-review:

Repo	PR
averaging-service	#67
backend	#600
claude-skills	#6
client	#75
client-runtime	#19
data-ingestors	#42
design-system	#11
docs	#10
documentation	#11
frontend-app	#436
model-zoo	#54
start-training	#9
tracebloc-client	#200
tracebloc-py-package	#106
tracebloc-website	#297
.github	#19

Routing once callers merge

Event	Status
PR merged	Done
PR closed without merging	Cancelled
Issue closed as completed	Done
Issue closed as not_planned (or no reason)	Cancelled

After they all merge

The temporary UI flip on the built-in "Item closed" workflow can be reverted (or that workflow disabled entirely). The custom Action handles all 4 cases correctly without it.

Closing this issue. Thanks again for the diagnosis — it surfaced both this kanban bug AND the auto-classify gap (now also fixed via .github#16).