💡 Automated Workflow Quality Gates: Rubric-Based Output Verification with Avatar Scorecard

[github-actions[bot]]

Summary

After a workflow completes, a grading subagent automatically evaluates the output against domain-specific rubrics (e.g., "brainstorming session produced 10+ diverse ideas across 3+ categories"). The avatar presents a visual scorecard showing pass/fail per criterion with plain-language explanations, giving non-technical users confidence that their output meets quality standards without needing expertise to evaluate it themselves.

Market Signal

Anthropic's Outcomes feature (shipped May 2026 in Managed Agents) improved task success by up to 10 percentage points. Harvey saw 6x task completion rate improvement with dreaming + outcomes. Wisedocs cut document review time 50% using outcomes. The rubric-based grading pattern is now production-proven at scale — the question is no longer "does this work?" but "how do we bring it to non-technical users?"

User Signal

Non-technical users (TalkTerm's core audience) cannot independently assess whether agent output is "good enough." They lack the domain expertise to evaluate brainstorming completeness, PRD thoroughness, or analysis rigor. This creates a trust gap that limits adoption. No existing idea addresses post-execution quality verification — #210 (Outcome-Driven Workflows) focuses on user-defined success criteria upfront, not automated grading after the fact.

Technical Opportunity

The Claude Agent SDK supports subagent spawning. A grading subagent can run in a fresh context window (avoiding self-bias) with a rubric prompt specific to the workflow type. Results map naturally to TalkTerm's ComparisonTable display mode (UX-DR13) for the scorecard presentation. Workflow templates (from Epic 6) can bundle rubrics alongside step definitions — so every workflow type ships with its own quality criteria.

Assessment

Dimension	Score	Rationale
Feasibility	high	Subagent spawning is a core SDK capability. Rubric evaluation is a single-prompt operation. ComparisonTable display mode is already planned in the architecture.
Impact	high	Directly addresses the trust gap for non-technical users. Transforms TalkTerm from "I hope this is good" to "this meets quality standards." Critical for enterprise adoption.
Urgency	med	Builds on Epic 6 (workflow execution). Most valuable once users are running real workflows. Not MVP-blocking but should ship soon after workflow execution is live.

Adversarial Review

Strongest objection: Running a grading pass adds latency and API cost to every workflow. Users must wait for the quality check, and the additional Claude API tokens increase BYOK expenses.

Rebuttal: The quality gate can be opt-in (enabled by default for first-time workflows, skippable for experienced users) and runs in parallel during the avatar's verbal summary (FR26), hiding most of the latency. The API cost is modest — a rubric evaluation is typically one short prompt with the output context, far less expensive than the workflow itself. The confidence value far outweighs the cost for non-technical users who would otherwise have no way to validate output quality.

Suggested Next Step

Define rubric schemas for the 3 MVP workflow types (brainstorming, PRD creation, competitive analysis) and prototype a grading subagent that scores output and returns structured pass/fail results mapped to the ComparisonTable display mode.