Han Feedback: gap-analysis — 2026-05-29

[mjansen401]

Han Feedback — 2026-05-29

Skills used: han:gap-analysis
Context: Comparing the proposal-roi skill (current state) against the newly updated proposals/CLAUDE.md spec (desired state) to identify what the skill needs before a redesign pass
Outcome: 13 gaps identified across discovery questions, output schema, ROI formula, and swag mode; report written to docs/research/proposal-roi-gap-analysis-report.md

---

What worked well

• All 10 original gaps confirmed, zero false positives. The investigator verified each finding against actual line numbers in the skill file. That's the bar — concrete evidence, not pattern-matching.
• The actor sweep found real gaps. The junior-developer's sweep across two actor types (Mike in full-discovery mode vs. the swag batch runner) surfaced three genuine additions: G-011 (batch summary line omits tier — the triage output is tier-blind at the decision level), G-012 (no "do not fabricate" guardrail in swag mode), and G-013 (status enum staleness). All three were confirmed in the second round. Without the two-actor framing, the analysis would have been discovery-mode-centric and missed the higher-severity swag failures.
• The reclassification of GAP-006 changed the fix scope. The analyzer called it "Divergent formula." The validator correctly identified that the skill has no formula at all — the two-term inference was the analyzer's, not the skill's text. That's a different (larger) fix: write a formula from scratch rather than add a missing term to an implied one. That kind of precision matters when you're about to redesign something.
• Coupling G-009 and G-013 was actionable. The validator flagging that two stale enums are in adjacent lines of the same schema block — fix them together or leave a partial repair — saved a future mistake. Small finding, high practical value.
• The skill handled a markdown skill file, not code. The gap analysis worked correctly on a plain-text instructions document, not a codebase. That's good generality.

---

What didn't work

• V7 (spec provenance) added meta-noise. The validator flagged that proposals/CLAUDE.md is an uncommitted same-session artifact and that the gap analysis is therefore circular. Technically correct. Practically, this is always going to be true when you're running a gap analysis against a spec you just wrote in the same session — which is a common and legitimate use case. The finding was handled correctly (commit the spec before implementing), but surfacing it as a high-confidence validator finding made it feel like a problem when it was really just a workflow reminder.
• The second-round gap-analyzer pass was protocol-required but added little. Trigger A fired (≥3 proposed new gaps). The second-round pass confirmed the three new gaps and rejected one meta-candidate. But the swarm agents had already described all three thoroughly — the second pass was confirming what was already confirmed. At this gap count and specificity level, the second round felt like ceremony.
• Junior-developer sweep required explicit actor framing in the brief. The two actors (interactive Mike vs. batch runner) are not obvious from the skill file alone. I had to describe them explicitly to get a useful sweep. If the brief had been generic, the actor sweep would have been discovery-mode-centric and missed the swag-specific findings. That means the quality of G-011/G-012 was dependent on brief quality — the skill didn't discover the actors itself.
• Report length. 13 gaps generates a long document. The executive summary is useful; the per-gap Section 2 entries are thorough but reading all 13 in sequence is a lot. There's no "give me the top 5 that block redesign" shortcut in the output format.

---

Overall

han:gap-analysis is well-suited for this use case — a structured skill or spec document compared against its desired behavior. The medium swarm at 4 agents was right-sized: the investigator added evidence rigor, the junior-developer's actor sweep was the highest-value addition, the validator's reclassification of GAP-006 changed fix scope, and PM synthesis made Section 4 readable. The main friction was the second-round trigger adding time without materially new findings, and the spec-provenance finding adding process noise. The report output was thorough enough to actually use as a redesign brief, which is the right bar for this skill.

---

Rating (informal)

Dimension	Score
Analysis quality (gaps found / false positives)	5/5
Actor sweep usefulness	4/5
Validation signal-to-noise	3/5
Output length appropriateness	3/5
Skill-to-problem fit	5/5
Actionability of findings	4/5