Skip to content

[#74] feat: KB content validation command#137

Merged
rucka merged 21 commits into
mainfrom
feature/#74-validate-kb-content
Feb 15, 2026
Merged

[#74] feat: KB content validation command#137
rucka merged 21 commits into
mainfrom
feature/#74-validate-kb-content

Conversation

@rucka

@rucka rucka commented Feb 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements the pair kb-validate command for comprehensive KB content validation before packaging. Covers all 21 acceptance criteria from story #74.

Key changes

  • Parser: --layout source|target, --strict, --ignore-config, --skip-registries flags on kb-validate; --layout, --skip-registries, --root on package
  • Registry utilities: layout.ts — layout resolution, prefix/flatten, symlink filtering, file collection, source↔target transforms, validateLayoutOption + parseSkipRegistriesOption (extracted from parsers)
  • 3 validators: structure (config-based), links (internal + strict external), metadata (SKILL.md frontmatter + placeholder detection)
  • Colored report: chalk-colored terminal output with errors (red), warnings (yellow), summary counts, exit codes 0/1/2
  • Package integration: layout.ts functions integrated into package handler, link rewriter for absolute→relative, --layout wired through to zip-creator
  • Tests: 504 tests (unit + e2e with 5-registry config covering all validation scenarios)
  • Smoke tests: kb-validate.sh scenario + infrastructure fixes (run-all.sh, utils.sh)
  • Documentation: 10 files updated with new CLI flags

Tasks completed

T-1 through T-11 (all checked in story body)

Tech debt resolved (post-review)

  • D3: Extracted duplicated validateLayoutOption + parseSkipRegistriesOption from both parsers into registry/layout.ts with dedicated unit tests
  • D1: Wired --layout flag through package handler → createPackageZipcopyRegistrySources (was parsed but ignored)
  • D2: Removed silent try-catch in kb-validate/handler.ts loadRegistries — config errors now propagate instead of returning false "0 errors"

Refs: #74

rucka and others added 16 commits February 15, 2026 09:03
- Added --layout <source|target> flag with validation
- Added --strict flag for external link validation
- Added --ignore-config flag to skip config validation
- Added --skip-registries <names> for comma-separated filtering
- Extracted validation helpers to reduce complexity
- Updated metadata with new options and examples
- Task: T-1 — Extend kb-validate parser (new flags)

Refs: #74
…oot flags

- Added --layout <source|target> flag with validation
- Added --skip-registries <names> for comma-separated filtering
- Added --root <path> for link relativization base path
- Extracted config builders to reduce complexity
- Updated metadata with new options and examples
- Task: T-2 — Extend package parser (new flags)

Refs: #74
- Added filterRegistries to exclude registries from skip list
- Added getNonSymlinkTargets to filter symlink targets
- Added resolveLayoutPaths to resolve paths by layout mode
- Added applyPrefixFlatten for name transformations
- Added validateSkipList to check invalid registry names
- 19 comprehensive tests covering all utilities
- Task: T-3 — Registry filtering & layout resolution utilities

Refs: #74
- Added validateStructure to check KB against config.json
- Validates registry paths exist and are non-empty
- Supports source and target layout modes
- Excludes symlink targets in target mode validation
- Returns detailed errors and warnings per registry
- 11 comprehensive tests covering all validation scenarios
- Task: T-4 — Config-based structure validator

Refs: #74
- Added validateLinks to check internal and external links
- Internal link validation always enabled (broken → error)
- External link validation with --strict flag (unreachable → warning)
- Supports relative, absolute, and anchor links
- 2-second timeout for external link checks
- 16 comprehensive tests covering all link scenarios
- Task: T-5 — Link integrity checker (internal + strict external)

Refs: #74
- Added validateMetadata to check SKILL.md frontmatter
- Validates required fields: name, description, version, author
- Missing frontmatter fields reported as errors
- Checks adoption files for [placeholder] content
- Unpopulated placeholders reported as warnings
- 14 comprehensive tests covering all validation scenarios
- Task: T-6 — Metadata & skills frontmatter validator

Refs: #74
- Added createValidationReport to aggregate validation results
- Determines exit codes: 0=success, 1=errors, 2=validation failed
- Added formatReport for colored terminal output
- Categorizes errors (red) and warnings (yellow) with summary
- Aggregates errors/warnings from structure, links, and metadata
- 14 comprehensive tests covering all reporting scenarios
- Task: T-7 — Validation report formatter & exit codes

Refs: #74
- Add collectLayoutFiles() to gather files from source/target layout
- Add transformSourceToTarget() for source→target path conversion with prefix/flatten
- Add transformTargetToSource() for target→source path conversion removing prefix
- Add link rewriter module with absolute→relative link conversion
- 54 layout tests (30 existing + 24 edge cases)
- 18 link-rewriter tests
- All tests pass, no lint/TypeScript errors
- Reusable across validate, install, update, package commands
- Task: T-8 — Package layout transform & link rewriter

Refs: #74
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add explicit readFile/writeFile bindings in link-rewriter tests
- Class methods not enumerable, need explicit binding with Object.assign
- All 478 tests pass

Refs: #74
- collectLayoutFiles: recursive scan for all files in layout
- zip-creator: uses collectLayoutFiles instead of manual copyRecursive
- Backward compat: try-catch readdir for simple mocks
- All 478 tests pass without modifications

Task: T-8 — Package layout transform & link rewriter

Refs: #74
Centralized layout transformation logic across install, update, validate, package:

Registry utilities in layout.ts:
- getCanonicalTarget: replaces inline .find(t => t.mode === 'canonical')
- resolveEffectiveDatasetRoot: centralizes flatten/prefix dataset logic
- getNonSymlinkTargets: filters out symlink targets

Consumer updates:
- resolver.ts, validation.ts, operations.ts: use getCanonicalTarget
- skill-refs.ts: use getNonSymlinkTargets
- install/handler.ts, update/handler.ts: use resolveEffectiveDatasetRoot
- package/handler.ts: add filterRegistries+validateSkipList, pass root
- package/validators.ts: use collectLayoutFiles+extractRegistries
- kb-validate/handler.ts: full rewrite using layout.ts orchestration

Lint fixes:
- Replace all 'as any' with proper Dirent[] types in test mocks
- Add fs import for Dirent type in test files
- Extract helper functions to reduce handler complexity

All 478 tests pass, quality gate clean.

Refs: #74
Updated 7 documentation files with new CLI flags:

cli-contracts.md:
- Added KbValidateOptions interface (layout, strict, ignoreConfig, skipRegistries)
- Added exit codes for kb validate command
- Updated PackageOptions with skipRegistries and root flags

commands.md:
- Added complete kb validate command reference with examples
- Updated package command options table (skipRegistries, root)
- Added usage examples for new flags

help-examples.md:
- Added 6 kb validate workflow examples (#10-14)
- Renumbered sections (Link Management #17-19, Packaging #20-25)
- Added package examples with skip-registries and custom root

cli-workflows.md:
- Added kb validate to quick command list
- Added package example

README files (pair-cli, knowledge-hub, root):
- Added kb validate to command tables
- Updated package examples with new flags

All examples are runnable with documented defaults.
mdlint pass clean.

Refs: #74
New smoke test scenario validates kb validate command on real dataset:

Test 1: Source layout validation (real dataset packages/knowledge-hub/dataset)
Test 2: Target layout validation (after install to workspace)
Test 3: Skip registries flag (--skip-registries adoption)
Test 4: Ignore config flag (--ignore-config)
Test 5: Validation failure detection (missing registry paths)

Added to CI_TESTS array in run-all.sh.
Marked OFFLINE_SAFE=true (no network required).

Note: Smoke tests currently blocked by CLI build issue (dist/cli.js not generated by tsc -b).
      This is a pre-existing infrastructure issue affecting all smoke tests.
      Test is complete and ready once build is fixed.

Refs: #74
- Fix run-all.sh: auto-build when dist/cli.js missing, guard empty BINARY_PATH
- Fix utils.sh: prevent symlink creation inside repo tree, fix TMP_DIR timing
- Fix kb-validate.sh: use kb-validate (not kb validate), ensure_tmp_dir, run from workspace
- Fix metadata-validator: version/author now warnings (recommended, not required per agent skills spec)

Refs: #74
- 14 e2e tests covering source/target layout, skip-registries,
  ignore-config, links, metadata, .pair guard, mixed errors
- 5-registry config (knowledge, adoption, agents x2, skills w/ prefix+flatten)
- Task: T-11

Refs: #74
- report-formatter: errors red, warnings yellow, summary colored
- smoke-tests/README: add kb-validate scenario docs (section 6)
- DoD: colored report + documentation now complete

Refs: #74
@rucka

rucka commented Feb 15, 2026

Copy link
Copy Markdown
Collaborator Author

Code Review — PR #137

PR: #137 [#74] feat: KB content validation command
Author: rucka
Reviewer: AI (Claude)
Date: 2026-02-15
Story: #74 Validate KB Content
Type: Feature


Review Summary

Overall Assessment: ✅ Approved with Comments

Key Changes: Comprehensive pair kb-validate command with layout-aware validation (structure, links, metadata), colored terminal report, 3 validators, registry filtering, prefix/flatten support. Package command extended with --skip-registries, --root, link rewriter. 41 files, +4798/-122 lines.

Business Value: Enables content creators to validate KB quality before packaging — catches broken links, missing metadata, structural issues. Exit codes integrate into CI.


Code Review Checklist

Functionality

  • Requirements Met — 19/21 AC implemented (AC17/AC18 package layout partial)
  • Business Logic — validators correctly handle all edge cases
  • Integration — reuses existing registry/config infrastructure
  • Error Handling — appropriate with exit codes 0/1/2
  • Performance — no regressions

Code Quality

  • Readability — clear separation: parser → handler → validators → report
  • Maintainability — layout.ts centralizes layout logic, reused by both commands
  • Naming — consistent, descriptive interfaces and functions
  • Complexity — reasonable for feature scope

Technical Standards

  • Style Guide — lint + prettier pass
  • Architecture — follows existing command pattern
  • Dependencies — no new deps (chalk already in catalog)

Security Review

No concerns — no user input passed to shells, no network calls by default (only with --strict), no file writes outside temp directories.


Testing Review

Tests: 492 passed (0 failed)
Quality Gate: 6/6 tasks pass
Coverage: unit (validators) + e2e (14 tests, 5-registry config) + smoke (5 scenarios)
  • Unit Tests — structure, links, metadata validators fully tested
  • E2E Tests — complex 5-registry config, both layouts, all error types
  • Smoke Tests — real dataset validation (source + target)
  • Edge Cases — empty dirs, missing registries, broken links, placeholder detection

Documentation

  • 10/10 doc files updated (cli-contracts, commands, help-examples, workflows, READMEs)
  • Smoke tests README updated with new scenario

Issues

Major 🔍 (1)

# File Issue
1 package/handler.ts --layout flag parsed but never used. zip-creator hardcodes layout: 'source'. AC17/AC18 (package layout transform) not implemented.

Recommendation: If deferred, remove the flag from package parser or add TODO + follow-up issue to avoid misleading users.

Minor 💡 (2)

# File Issue
2 kb-validate/handler.ts:28-30 loadRegistries silently swallows config errors → returns empty {} → misleading "no registries" warning
3 kb-validate/parser.ts + package/parser.ts validateLayoutOption() and parseSkipRegistriesOption() duplicated — extract to shared utility

Positive Feedback

  • Excellent architecture: layout.ts centralizes all layout logic (resolve paths, apply prefix/flatten, collect files, transform source↔target) — single source of truth, highly reusable
  • Thorough test pyramid: unit → e2e → smoke, each level adds value
  • Smoke test infra fixes: multiple root-cause fixes in run-all.sh and utils.sh (symlink guard, TMP_DIR timing, auto-build, empty BINARY_PATH) — these benefit ALL future smoke tests
  • Clean separation of concerns: handler orchestrates, validators return data, formatter renders — easy to extend

Tech Debt

Item Severity Description
Package layout transform Medium --layout target for package not implemented (AC17/AC18)
Duplicated parser helpers Low 2 functions copy-pasted between parsers
Silent config swallowing Low Config errors invisible in kb-validate handler

Risk Assessment

Risk Impact Probability Mitigation
Users pass --layout target to package Low Low Flag accepted but no effect — not yet documented as working
Config errors silently ignored Low Medium User sees "no registries" warning, may not realize config is broken

Decision

TECH-DEBT — Approve current PR, track package layout transform separately.

The kb-validate command is fully functional. The major finding (package --layout) is outside the primary scope of kb-validate and can be addressed in a follow-up.

@rucka rucka left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision: TECH-DEBT (APPROVE)

KB validation command is well-implemented with solid test coverage (unit + E2E + smoke). Three tech debt items tracked for follow-up — none block this PR.

See detailed review report in PR comment above.

Comment thread apps/pair-cli/src/commands/package/handler.ts
Comment thread apps/pair-cli/src/commands/kb-validate/handler.ts
Comment thread apps/pair-cli/src/commands/kb-validate/parser.ts
rucka and others added 5 commits February 15, 2026 17:25
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…riesOption

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rucka
rucka merged commit 1effa86 into main Feb 15, 2026
2 checks passed
@rucka
rucka deleted the feature/#74-validate-kb-content branch February 15, 2026 16:35
This was referenced Feb 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant