Add spec 188: Knowledge Backflow in SDD Workflow

[Copilot]

Chinese community requested (看是否可以增加一下知识回流) mechanisms to keep specs synchronized with implementation reality. Specs currently document original design but rarely capture what actually happened during coding, misleading future AI agents and developers.

Spec 188: Progressive Enhancement with Agent-Based Validation

Phase 1: Workflow Integration (v0.5.0)

Extend completion verification (spec 174) to check spec freshness:

• Block completion if README unchanged since status: in-progress
• Add "Implementation Learnings" section to templates
• Update AGENTS.md with explicit backflow step

// In CompletionVerifier
fn verify_freshness(spec_path: &Path) -> Result<FreshnessResult> {
    let in_progress_time = get_status_transition_time("in-progress")?;
    let spec_modified = get_file_modified_time(spec_path)?;
    
    Ok(FreshnessResult {
        is_fresh: spec_modified > in_progress_time,
        warning: "Spec unchanged since starting work. Did implementation match design exactly?"
    })
}

Phase 2: Orchestrator + Reviewer Agent Validation (v0.6.0)

Revised approach using orchestrator + reviewer agents instead of heuristic detection to avoid false positives and leverage clean context advantage:

• Orchestrator agent (spec 168) prepares validation task with spec requirements + implementation code
• Reviewer agent validates with fresh context (no history pollution)
• Identifies missing features, architecture drift, outdated technology mentions
• Generates actionable spec update suggestions
• CLI: lean-spec agent review <spec> command
• Desktop UI: "Validate with AI Reviewer" button

Key advantages over heuristic detection:

• ✅ Clean context agent (no context window degradation)
• ✅ Semantic understanding vs keyword matching
• ✅ Fewer false positives (<5% target vs ~10% with heuristics)
• ✅ Leverages existing orchestration infrastructure (specs 168, 171)

Phase 3: Proactive Spec Sync (Future) - High Token Cost ⚠️

Note: Requires careful cost/benefit analysis before implementation.

• Monthly cost estimate: ~$15.60 for 100 specs with weekly reviews
• Annual cost at scale (1,000 specs): ~$1,870
• Decision criteria: Only implement after Phase 1 >80% adoption + Phase 2 proven valuable + budget approved
• Alternative: Start with on-demand sync only (lean-spec sync <spec>) to avoid ongoing costs

Implementation Approach

Phase 1 first (low complexity, high value):

• Mostly workflow/docs changes
• Immediate behavioral change
• Target: >80% specs updated within 7 days of completion

Phase 2 uses intelligent agent validation:

• Orchestrator dispatches reviewer agent for validation
• Higher accuracy with clean context
• Token-efficient focused validation task

Phase 3 only if cost-justified - requires Phase 1+2 success validation, budget approval, and measurable ROI.

Files Created

• specs/188-knowledge-backflow-sdd-workflow/README.md (2,280 tokens)
• specs/188-knowledge-backflow-sdd-workflow/DESIGN.md (technical architecture)
• specs/188-knowledge-backflow-sdd-workflow/ANALYSIS.md (problem deep-dive)

Dependencies: 047 (git history analysis), 122 (content/metadata alignment), 174 (completion hooks), 018 (validation framework), 168 (orchestration platform), 171 (burst mode orchestrator)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>看是否可以增加一下知识回流</issue_title>
<issue_description>在SDD的流程中，自动增加一些知识回流的流程。
1、主要是能保证需求功能、架构设计等伴随spec实现后，可以同步更新。
2、另外，就是spec内部的文档的自动关联处理。
3、流程的归属于正向回流，实现后功能已经落地并实现，需要做一个功能、架构等逆向回流。</issue_description>

<agent_instructions>Please check this issue and create a new spec to discuss feasibility and solutions, making sure it aligns with our current infra and business value</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes 看是否可以增加一下知识回流 #100

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[tikazyq]

@copilot I agree with "Workflow Integration" which automates the knowledge/spec sync. But I think it's not enough to trust AI agents to self check their specs and codebase following the expectation based on the hints provided by CLI/MCP. We can get quick win with Phase 1 implementation, but we should keep in mind it's a better idea to have an orchestrator agent assigning specs to coding sub-agents and ask a reviewer agent to validate claimed completed specs against actual implementation in codebase (we have related specs on this), so I think we should replace "Automated Drift Detection" with this approach. The reason for this approach is simple: context window shortage (performance degrades with long context, so a clean context agent will work smart enough to validate the work). I think "Automated Drift Detection" will definitely cause a lot of false positive so we shouldn't use that. For "AI-Assisted Sync", I like the idea, but we should understand this will require proactive agents to periodically check and question specs against codecase and provide suggestions or direct updates on specs, so it may require a lot of tokens and cost.

[Copilot]

@copilot I agree with "Workflow Integration" which automates the knowledge/spec sync. But I think it's not enough to trust AI agents to self check their specs and codebase following the expectation ba...

Agreed and revised in commit bd7fa3d.

Changes Made:

1. Phase 1 (Workflow Integration) - Kept as-is ✅ Quick win approach

2. Phase 2 - Replaced "Automated Drift Detection" with Orchestrator + Reviewer Agent Validation:

   • Leverages specs 168 and 171 orchestration infrastructure
   • Reviewer agent gets clean context (no history pollution)
   • Higher accuracy: semantic understanding vs keyword matching
   • Fewer false positives: <5% target vs heuristic ~10%
   • Implementation: lean-spec agent review <spec> command + Desktop UI button

3. Phase 3 - Updated to acknowledge token cost concerns:

   • Added detailed cost analysis: ~$15.60/month for 100 specs weekly
   • Scaling cost: ~$1,870/year for 1,000 specs
   • Requires Phase 1+2 success validation + budget approval
   • Alternative: Start with on-demand sync only
   • Decision criteria: Only if manual maintenance is measurably expensive

Why Orchestrator + Reviewer Approach:

• ✅ Clean context advantage (performance doesn't degrade)
• ✅ No false positive issues from heuristic pattern matching
• ✅ Builds on existing orchestration platform
• ✅ Focused validation task: "Does implementation match this spec?"

[Copilot]

Pull request overview

This PR introduces Spec 188, which addresses knowledge backflow in the SDD workflow - a mechanism to keep specifications synchronized with implementation reality. The spec was created in response to a Chinese community request for ensuring specs remain accurate after implementation, preventing misleading documentation that can confuse both AI agents and human developers.

Key Changes

• Progressive three-phase approach: Workflow integration → Orchestrator + Reviewer agent validation → Proactive spec sync
• Phase 2 design revision: Shifted from heuristic-based drift detection to orchestrator + reviewer agent validation for higher accuracy and fewer false positives
• Cost-aware design: Detailed token cost analysis for Phase 3 with clear decision criteria before implementation

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File	Description
specs/188-knowledge-backflow-sdd-workflow/README.md	Main spec document defining the three-phase progressive enhancement approach with implementation plan, test strategy, and dependency references
specs/188-knowledge-backflow-sdd-workflow/DESIGN.md	Technical architecture detailing completion verification enhancement, orchestrator + reviewer agent integration, cost analysis, and phased rollout strategy
specs/188-knowledge-backflow-sdd-workflow/ANALYSIS.md	Deep problem analysis covering root causes of spec-reality drift, alternative approaches considered, success criteria, and risk assessment

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The function format_implementation_files is called but never defined or imported in this code example. This will cause a compilation error. Consider adding a comment indicating this is a helper function that needs to be implemented, or provide its definition/import statement.

Suggested change

	spec.content,
	spec.content,
	// `format_implementation_files` is a helper that should format the implementation
	// files for inclusion in the prompt; it must be implemented or imported by the user.

[Copilot]

The ANALYSIS.md file references "Layer 2: Automated Detection" with "Heuristics: tech mentions, file staleness, broken links", but the main design has shifted to using "Orchestrator + Reviewer Agent Validation" for Phase 2 instead of heuristic detection. This inconsistency between ANALYSIS.md and the actual design in DESIGN.md/README.md could confuse readers. Consider updating ANALYSIS.md to reflect the current design approach or adding a note that the approach evolved.

Suggested change

	**Layer 2: Automated Detection** (Medium value)
	- System identifies potential drift
	- Heuristics: tech mentions, file staleness, broken links
	- Provides actionable warnings
	- Catches what humans miss
	**Layer 2: Orchestrated Validation** (Medium value)
	- Orchestrator monitors for potential drift signals and triggers a Reviewer agent
	- Reviewer agent compares recent implementation changes against the spec and flags discrepancies
	- Surfaces structured “update needed” findings into the same human-in-the-loop workflow
	- (Earlier idea of pure heuristic detection—tech mentions, file staleness, broken links—evolved into this orchestrator + reviewer approach)

[Copilot]

The code reads frontmatter from spec_path (line 74) but then tries to join it with "README.md" on line 84. This suggests spec_path might be used inconsistently - either it points to the README.md file directly (in which case line 74 is correct but line 84's join is wrong), or it points to the spec directory (in which case line 74 should also use a join). The implementation needs to be consistent about what spec_path represents.

Suggested change

	let spec_modified = get_file_modified_time(spec_path.join("README.md"))?;
	let spec_modified = get_file_modified_time(spec_path)?;

[Copilot]

The FreshnessResult struct is constructed with three fields (is_fresh, spec_modified, in_progress_time) but is never defined in the code example. This will cause a compilation error. Consider adding the struct definition or noting that it needs to be defined elsewhere.

[Copilot]

The orchestrator reads both the spec and implementation code in step 3, but in the comment on line 193 it says "Reads: Spec requirements" which should be just one of two read operations. Consider adding a second "Reads: Implementation code" comment or combining them for clarity.

Suggested change

	│ │ Reads: Spec requirements │
	│ │ Reads: Implementation code │
	│ │ Reads: Spec requirements + implementation code │

[Copilot]

In the JSON output format example, the boolean values "true/false" should be actual JSON boolean literals without quotes. The current format suggests using the string "true/false" which would be invalid JSON. Change to just "true or false" in the description, or use proper JSON boolean values in the example.

Suggested change

	"complete": true/false,
	"complete": true,

[Copilot]

The annual cost calculation rounds to $187 but should be $187.20 (12 × $15.60). While this is a minor rounding discrepancy, it's worth being precise in cost projections to maintain credibility in financial estimates.

[Copilot]

The example suggests correcting "receive" to "receive", which are identical words. This appears to be a placeholder or copy-paste error in the documentation explaining the spelling correction feature. Consider using an actual typo example like "recieve" to "receive" for clarity.