Skip to content

feat: Wave 2 — project knowledge system (decisions + pitfalls)#140

Closed
dean0x wants to merge 5 commits intomainfrom
feat/project-knowledge-99
Closed

feat: Wave 2 — project knowledge system (decisions + pitfalls)#140
dean0x wants to merge 5 commits intomainfrom
feat/project-knowledge-99

Conversation

@dean0x
Copy link
Owner

@dean0x dean0x commented Mar 14, 2026

Summary

Closes #99. Implements the Project Knowledge system — structured extraction of architectural decisions (ADR-NNN) and known pitfalls (PF-NNN) from command flows, with lightweight injection at session start.

  • 2 knowledge files in .memory/knowledge/: decisions.md (append-only ADRs) and pitfalls.md (area-specific gotchas)
  • Session-start injection: TL;DR comment headers only (~30-50 tokens via head -1), graceful degradation if files missing
  • Agent consumption: Coder reads decisions before implementation, Reviewer checks pitfalls against diff, Skimmer includes decision count in orientation
  • Knowledge extraction: /implement → Phase 11.5 records ADRs; /code-review → Phase 4.5, /debug → Phase 4.5/7.5, /resolve → Phase 5.5 record pitfalls
  • Both variants: Base and Teams command files updated (teammate prompts include knowledge reads)
  • CLI: createMemoryDir() now creates knowledge/ subdirectory
  • Tests: 8 new tests covering TL;DR parsing, sequential numbering, deduplication, graceful degradation, and TL;DR update after append (246 total, all passing)

Key design choices

  • Token conservative: only TL;DR headers at session start, agents read full files on demand
  • No new agents or CLI commands — extraction is inline in existing command phases
  • 50-entry cap per file with mkdir-based lock for concurrent write safety
  • Seed entries from existing codebase knowledge (Result types decision, synthesizer glob pitfall)

Test plan

  • npm run build passes (31 skills, 17 plugins)
  • npm test — 246/246 tests pass
  • Session-start hook verified: TL;DR injected into additionalContext
  • Manual: run /implement with architectural decisions, verify ADR appended
  • Manual: run /code-review on branch with issues, verify PF appended
  • Manual: delete knowledge files, confirm graceful degradation (no errors)

Add structured knowledge extraction and injection so agents learn from
past architectural decisions and known pitfalls across sessions.

- Create `.memory/knowledge/decisions.md` (ADR-NNN) and `pitfalls.md` (PF-NNN)
- Inject TL;DR headers at session start (~30-50 tokens)
- Coder reads decisions before implementation, Reviewer checks pitfalls
- /implement records ADRs, /code-review + /debug + /resolve record pitfalls
- Both base and Teams command variants updated
- createMemoryDir() now creates knowledge/ subdirectory
- 8 new tests (246 total), all passing
module structure, and architectural conventions relevant to this task.
4. Document findings with file:path references.
5. Document findings with file:path references.
5. Report completion: SendMessage(type: "message", recipient: "team-lead",
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate step numbering: Step 5 appears twice consecutively. The "Report completion: SendMessage" step should be numbered 6, not 5, since step 2 was inserted above it.

Suggested fix:

    5. Document findings with file:path references.
    6. Report completion: SendMessage(type: "message", recipient: "team-lead",

This issue affects all four explorer prompts (architecture, integration, reusable-code, edge-case). Use consistent sequential numbering (2-6) after inserting the new knowledge-reading step.

1. **Load focus skill** - Read the pattern skill file for your focus area from the table above. This gives you detection rules and patterns specific to your review type.

1.5. **Check known pitfalls** - If `.memory/knowledge/pitfalls.md` exists, read it. Check if any pitfall Areas overlap with files in the current diff. Verify the Resolution was applied. Flag if a known pitfall pattern is being reintroduced.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-standard step numbering: Using "1.5" breaks sequential integer numbering used throughout the codebase. Agents may interpret this as a sub-step rather than a full responsibility.

Suggested fix: Renumber to use sequential integers (1, 2, 3, ..., 11) since you're inserting a new responsibility between steps 1 and 2. All subsequent steps would be incremented accordingly.

Alternative: If fractional numbering is intentional for non-breaking additions, document this convention in CLAUDE.md.

You are reviewing PR #{pr_number} on branch {branch} (base: {base_branch}).
1. Read your skill: `Read ~/.claude/skills/security-patterns/SKILL.md`
2. Read review methodology: `Read ~/.claude/skills/review-methodology/SKILL.md`
2.5. Read `.memory/knowledge/pitfalls.md` if it exists. Check for known pitfall patterns in the diff.
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-standard step numbering: Step "2.5" is inserted between sequential steps, breaking the standard integer pattern. Agents may not recognize this as a full sequential step in the review process.

Suggested fix: Renumber all steps sequentially (2 → 3, old 3 → 4, etc.) so the pitfall-reading step is step 3 and subsequent steps follow consecutively. Apply this consistently across all four reviewer teammate prompts in this file.

@dean0x
Copy link
Owner Author

dean0x commented Mar 14, 2026

Code Review Summary

Inline Comments Created (High Confidence ≥80%)

I've created targeted inline comments for critical findings:

  • Duplicate step 5 in implement-teams.md explorer prompts (95% confidence)
  • Fractional numbering "1.5" in reviewer.md responsibilities (82% confidence)
  • Fractional numbering "2.5" in code-review-teams.md reviewer prompts (82% confidence)

Additional High-Confidence Findings (Consolidated)

Architectural Issues (88-85% confidence):

  • Duplicated knowledge extraction logic across 8 command files (DRY violation). Recommend extracting 7-step pitfall-recording and decision-recording procedures to docs-framework/references/knowledge-procedures.md.
  • Knowledge injection hook has minor inconsistent newline handling when WORKING-MEMORY.md is missing.

Lock Specification Inconsistency (82-85% confidence):

  • Debug/resolve commands omit timeout/recovery parameters from lock specification, while code-review/implement include (30s timeout, 60s stale recovery). Standardize across all 8 command files to prevent orphaned locks.

Test Coverage Gaps (80% confidence):

  • Knowledge format tests are self-referential — they test inline logic, not exported TypeScript functions or actual shell hook behavior.
  • No integration tests for session-start-memory hook's new knowledge TL;DR injection.
  • No test coverage for shell hook failure paths.

Missing Integration (80% confidence):

  • /specify and /self-review commands not updated to read knowledge files (may be intentional scoping).

Performance (Low impact, 80% confidence):

  • Session-start hook: head -1 | sed could combine into single sed to reduce subprocess spawns.

Documentation (80% confidence):

  • Asymmetry between Skimmer (reads TL;DR only) and Coder (reads full file) not documented.

Verification Checklist

  • BLOCKING: Fix duplicate step 5 numbering in all 4 explorer prompts (implement-teams.md)
  • Standardize lock specification across debug/resolve/code-review/implement commands
  • Extract duplicated pitfall/decision procedures to shared reference
  • Document fractional numbering convention or standardize to sequential
  • Consider integration tests for shell hooks in follow-up PR

Assessment

Blocking Issues: 3 (step numbering issues and lock inconsistencies)
Should Fix: 6 (DRY violations, test coverage, missing integration)
Pre-existing: Several (noted but not blocking)

Recommendation: CHANGES_REQUESTED
The step numbering bugs should be fixed. Lock standardization is important for reliability. Consider the DRY violation extraction as a follow-up if the scope is too large for this PR.


Created by Claude Code — Analysis based on 9 review reports (Architecture, Performance, Tests, Regression, Documentation, TypeScript, Security, Complexity, Consistency)

Batch 1 — Mechanical fixes:
- Fix duplicate step 5→6 in 4 explorer prompts (implement-teams.md)
- Fix fractional 2.5→3 renumbering in 4 reviewer prompts (code-review-teams.md)
- Fix fractional 1.5→2 renumbering in reviewer.md responsibilities
- Update stale counts: 24→31 skills, 8→17 plugins (file-organization.md)
- Add missing ambient-prompt.sh to hooks listing (file-organization.md)

Batch 2 — DRY extraction via knowledge-persistence skill:
- Create shared/skills/knowledge-persistence/SKILL.md with canonical procedure
  (ADR/PF formats, lock protocol, capacity cap, dedup, TL;DR update)
- Replace 8 inline extraction procedures with skill references in
  implement, code-review, debug, resolve (base + teams variants)
- Register skill in 4 plugin manifests (plugin.json + plugins.ts)

Batch 3 — Hook improvements:
- Collapse head+sed+grep into single sed -n for TL;DR extraction
- Guard section joins to prevent leading newlines when Section 1 skipped

Batch 4 — Quick fixes:
- Add TL;DR-only rationale comment to skimmer.md step 6
- Log createMemoryDir errors in verbose mode instead of silent catch
- Add 3 hook integration tests (TL;DR injection, no leading newlines,
  no knowledge section when files absent)

Issues dismissed: #9 (TL;DR sanitization — trust model), #10 (integrity
verification — instruction-enforced), #17 (mkdir failure test — low value),
#18 (phase ordering cosmetic), #19 (intentional scope exclusion)
@dean0x
Copy link
Owner Author

dean0x commented Mar 14, 2026

Code Review Resolution Status

Commit 40b167b resolves findings from the 9-reviewer code review.

Fixed (14 issues)

# Issue Resolution
1 Duplicate step "5" in explorer prompts Renumbered to 5→6 in all 4 prompts
2 DRY: extraction procedure duplicated 8× Created knowledge-persistence skill — single source of truth
4 Inconsistent lock specs Solved by #2 — unified in skill
5 Fractional "2.5" in reviewer teammate prompts Renumbered 2.5→3, shifted 3→10 in all 4 prompts
6 Fractional "1.5" in reviewer.md Renumbered 1.5→2, shifted 2→11
7 Inconsistent template headers Solved by #2 — templates defined once in skill
8 No hook integration test Added 3 integration tests for session-start-memory hook
11 Undocumented Skimmer asymmetry Added TL;DR-only rationale comment to step 6
12 Leading newlines when Section 1 skipped Guarded section joins with emptiness check
13 Stale counts in file-organization.md Updated 24→31 skills, 8→17 plugins
14 Missing ambient hook in file-organization.md Added ambient-prompt.sh to hooks listing
15 Skimmer instruction/output mismatch Aligned step 6 to mention <!-- TL;DR: ... --> format
16 Silent catch in createMemoryDir Logs error in verbose mode
20 Redundant subprocesses Collapsed head+sed+grep into single sed -n

Deferred (3 issues)

# Issue Rationale
9 TL;DR input sanitization Trust model: .memory/ is gitignored, local-only. Requires filesystem access to exploit.
10 Knowledge integrity verification Same trust model. Append-only is instruction-enforced, matching all agent behavior.
17 Missing createMemoryDir failure test mkdir({ recursive: true }) basically never fails. Low value.

Dismissed (2 issues)

# Issue Rationale
18 Phase ordering code-review vs teams Both record pitfalls post-synthesis. Display difference is cosmetic.
19 Missing /specify and /self-review integration Intentional scope: /specify has no implementation decisions, /self-review is ephemeral.

Verification

  • npm run build — 32 skills, 17 plugins ✅
  • npm test — 249/249 tests pass ✅
  • Hook integration — TL;DR injection verified, no leading newlines ✅
  • All 8 command files reference knowledge-persistence skill instead of inline procedures ✅

…eMemoryDir failure test

- /specify: Add Phase 2.5 to read decisions.md + pitfalls.md before
  exploration. Constraints and Failure Modes explorers now receive
  prior decisions and known pitfalls as context.
- /self-review: Phase 0 reads knowledge files. Simplifier and
  Scrutinizer receive KNOWLEDGE_CONTEXT to check for reintroduced
  pitfall patterns and verify architectural consistency.
- Add createMemoryDir failure test (file blocking mkdir path).

Resolves review issues #17 and #19 from PR #140.
@dean0x
Copy link
Owner Author

dean0x commented Mar 14, 2026

Follow-up: Additional Resolutions (b52b8b1)

After discussing the deferred/dismissed issues:

# Issue Updated Status
17 Missing createMemoryDir failure test Fixed — added test that blocks mkdir with a file
19 Missing /specify and /self-review integration Fixed — added knowledge awareness (read-only) to both commands

/specify: New Phase 2.5 reads decisions.md + pitfalls.md before exploration. Constraints and Failure Modes explorers now receive prior decisions and known pitfalls as context.

/self-review: Phase 0 now reads knowledge files. Both Simplifier and Scrutinizer receive KNOWLEDGE_CONTEXT to check for reintroduced pitfall patterns and verify architectural consistency.

Confirmed deferred (after discussion):

Confirmed non-issue:

Tests: 250/250 pass ✅

Dean Sharon added 2 commits March 14, 2026 23:12
The base specify.md got Phase 2.5 (knowledge read) but the teams
variant was missed. Both now read decisions.md + pitfalls.md before
exploration.
…mmands

Eliminates all Phase X.5 numbering introduced by the knowledge-recording
phases, replacing with sequential integers and shifting subsequent phases.

Files: specify, implement, code-review, debug, resolve (base + teams variants)
@dean0x dean0x closed this Mar 15, 2026
@dean0x dean0x deleted the feat/project-knowledge-99 branch March 15, 2026 11:58
dean0x added a commit that referenced this pull request Mar 15, 2026
…istency (#145)

## Summary

Post-merge cleanup for the Wave 2 project knowledge system (#99).
Addresses consistency audit findings from #142 and code review findings
from #140.

- Add `knowledge-persistence` skill with full extraction procedure, lock
protocol, loading instructions, and examples
- Add knowledge awareness (Load Project Knowledge phase) to `/debug`,
`/specify`, `/self-review` commands
- Add pitfall recording phase to `/code-review` and `/resolve` commands
- Renumber fractional phases (1.5, 2.5) to sequential integers across
all 12 command files
- Fix CLAUDE.md stale skill count (31→32), add skill to architecture
reference
- Remove conditional "(if available)" phrasing from knowledge loading
phases
- Add `createMemoryDir` failure test for missing directory edge case
- Resolve 14 code review findings from PR #140

## Files Changed (28 files, +814/-227)

**New files (2):** `knowledge-persistence/SKILL.md`,
`knowledge-persistence/references/examples.md`
**Commands (12):** All 6 command pairs updated with knowledge phases +
sequential numbering
**Agents (3):** coder, reviewer, skimmer — knowledge context references
**Plugins (4):** plugin.json manifest updates for knowledge-persistence
skill
**Tests (1):** +230 lines of memory/migration tests  
**Docs (4):** CLAUDE.md, skills-architecture, file-organization,
docs-framework

## Test plan

- [x] `npm run build` passes (32 skills, 17 plugins)
- [x] `npm test` passes (250 tests, 13 test files)
- [x] Grep confirms zero "(if available)" in knowledge-related contexts
- [ ] Manual: verify `/debug` loads pitfalls.md before hypothesis
generation
- [ ] Manual: verify `/specify` passes knowledge context to explore
agents

---------

Co-authored-by: Dean Sharon <deanshrn@gmain.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Persistent Project Knowledge (Stateful Agents / Persistent Minds)

1 participant