Separate KB Dataset Release

[rucka]

Resolution

Delivered by PR #83 (merged). Board: Done.

feat: separate KB dataset release with auto-download.

Story Statement

As a knowledge-hub maintainer
I want KB dataset as separate GitHub release artifact alongside CLI
So that KB content is versioned independently while keeping single release workflow

Where: GitHub Releases page - single release with dual artifacts (CLI binaries + KB dataset ZIP)

Epic Context

Parent Epic: #65 Enhanced CLI Packaging & Distribution
Status: Code Review ✅
Priority: P0 (Must-Have)

Status Workflow

• Refined: Story is detailed, estimated, and ready for development ✅
• In Progress: Story is actively being developed ✅
• In Review: Pull request created, awaiting code review ✅
• Done: Story delivered and accepted

Branch: feature/#72-kb-dataset-release
Pull Request: #83 - [US-72] feat: separate KB dataset release with auto-download

Acceptance Criteria

Functional Requirements

Given-When-Then Format:

1. Given a changeset release workflow executes for version v1.2.3
   When the release completes
   Then a single GitHub release v1.2.3 contains both CLI binaries AND knowledge-base-1.2.3.zip artifact

2. Given the KB dataset ZIP is created from knowledge-hub/dataset/
   When the ZIP is extracted
   Then the complete directory structure is preserved with all subdirectories (.pair/, .github/, AGENTS.md)

3. Given a KB dataset ZIP artifact knowledge-base-1.2.3.zip
   When the ZIP is opened
   Then it contains manifest.json with version, file list, checksums, and timestamp metadata

4. Given the release workflow builds artifacts
   When KB packaging script executes
   Then it generates manifest.json with version, complete file list with paths, individual file SHA256 checksums, creation timestamp

5. Given viewing GitHub release page for v1.2.3
   When looking at release assets
   Then both artifact types are present: CLI binaries (linux/macos/windows) AND knowledge-base-1.2.3.zip with knowledge-base-1.2.3.zip.sha256

6. Given CLI package is installed
   When KB dataset not found locally
   Then CLI automatically downloads KB from GitHub release and extracts to ~/.pair/kb/{version}/

7. Given CLI starts and KB already cached at ~/.pair/kb/{version}/
   When CLI resolves KB path
   Then CLI uses cached KB without re-download

Business Rules

• Single release workflow produces both CLI and KB artifacts with same version
• KB dataset ZIP MUST include entire knowledge-hub/dataset/ folder with preserved structure
• ZIP asset naming MUST follow format: knowledge-base-{version}.zip
• Manifest.json MUST be included in ZIP root with complete metadata
• SHA256 checksum file MUST be generated for KB ZIP: knowledge-base-{version}.zip.sha256
• Both CLI and KB artifacts uploaded to same GitHub release
• If KB packaging fails, entire release fails (atomic workflow)
• CLI bundle MUST NOT include KB dataset (Epic Simple KB Distribution & Installation #65 AC3)
• CLI auto-downloads KB on first run if not cached locally
• KB cached at ~/.pair/kb/{version}/ for reuse across projects

Edge Cases and Error Handling

• KB Packaging Failure: Release workflow fails atomically, no partial release, clear error message about KB packaging failure
• Large Dataset Size: Script validates dataset size before ZIP creation, warns if exceeds threshold (e.g., >50MB)
• Missing Dataset Files: Validation checks knowledge-hub/dataset/ exists and contains expected structure before packaging
• Checksum Generation Failure: Release fails with clear error, no ZIP uploaded without valid checksum
• Manifest Generation Error: Release fails if manifest.json cannot be created with complete metadata
• KB Download Failure: CLI displays clear error with GitHub release URL, suggests manual download
• Network Unavailable: CLI fails gracefully with actionable message if cannot download KB

Technical Analysis

Implementation Approach

Technical Strategy: Extend existing single release workflow to produce KB dataset ZIP artifact alongside CLI binaries, exclude KB from CLI bundle, implement minimal auto-download mechanism

Key Components:

1. KB Packaging Script (scripts/workflows/release/package-kb-dataset.sh)

   • Creates ZIP from knowledge-hub/dataset/
   • Generates manifest.json with metadata
   • Outputs to release/knowledge-base-{version}.zip

2. Manifest Generator

   • Scans dataset directory recursively
   • Computes SHA256 for each file
   • Creates JSON with version, files array, checksums, timestamp

3. Release Workflow Enhancement

   • Calls KB packaging script after CLI build
   • Uploads KB ZIP and checksum to same GitHub release
   • Atomic failure if KB packaging fails

4. CLI Bundle Modification

   • Exclude packages/knowledge-hub/dataset/ from package-manual.sh
   • Reduce CLI bundle size from 6M to ~300KB

5. Minimal KB Manager (new)

   • Auto-download KB from GitHub release on first run
   • Extract to ~/.pair/kb/{version}/
   • Simple error handling (no progress bar, no resume)

6. Path Resolution Update

   • Check ~/.pair/kb/{version}/ first
   • Fallback to bundled location for backward compatibility (dev mode only)

Data Flow:

1. Single changeset release triggered → version 1.2.3
2. CLI build → binaries created (KB excluded from bundle)
3. KB packaging script → release/knowledge-base-1.2.3.zip + manifest
4. Single GitHub release created → uploads CLI binaries + KB ZIP
5. Release v1.2.3 contains all artifacts
6. User installs CLI → first run downloads KB to ~/.pair/kb/1.2.3/

Integration Points:

• Existing release workflow (same workflow, additional artifact)
• scripts/workflows/release/create-github-release.sh for artifact upload
• Changeset version coordination (already configured)
• CLI startup: KB availability check and auto-download

Technical Requirements

• ZIP compression with preserved directory structure and file permissions
• SHA256 checksum generation for ZIP file integrity
• JSON manifest format with version, file list, checksums
• Atomic release workflow - fail entire release if KB packaging fails
• Dataset size validation (warn if >50MB, consider optimization)
• Release artifact naming consistency: knowledge-base-{version}.zip
• CLI bundle must exclude KB dataset completely
• Simple HTTP download for KB ZIP (no progress bar, no resume in this story)
• KB cache at ~/.pair/kb/{version}/ with version isolation

Technical Risks and Mitigation

Risk	Impact	Probability	Mitigation Strategy
KB packaging fails during release	High	Low	Atomic workflow - package KB before GitHub release creation, fail entire workflow on KB error
Large dataset size affects release time	Medium	Medium	Add size validation, optimize compression, monitor dataset growth
Multiple artifacts upload complexity	Low	Low	Sequential upload in same release, existing pattern from CLI multi-platform binaries
Manifest generation incomplete	Medium	Low	Comprehensive file scanning with error handling, validate manifest completeness
KB download fails on first CLI run	Medium	Low	Clear error message with GitHub release URL, user can retry or manual download
Network unavailable during CLI install	Medium	Medium	Fail fast with actionable error, no bundled fallback (Epic requirement)

Spike Requirements

No spikes required - straightforward addition to existing release workflow plus minimal download logic

Definition of Done Checklist

Development Completion

• KB packaging script created: scripts/workflows/release/package-kb-dataset.sh
• Script creates ZIP from knowledge-hub/dataset/ preserving full directory structure
• Manifest.json generation implemented with version, file list, checksums, timestamp
• SHA256 checksum file generated for KB ZIP artifact
• Release workflow modified to call KB packaging before GitHub release creation
• create-github-release.sh uploads KB ZIP + checksum to same release as CLI binaries
• Atomic workflow logic: entire release fails if KB packaging fails
• KB dataset excluded from CLI bundle (package-manual.sh modified)
• Minimal KB manager implemented: auto-download from GitHub release
• Path resolution updated: uses ~/.pair/kb/{version}/ cache
• CLI startup check: downloads KB if not cached
• Code follows project standards, reviewed and approved
• Unit tests for KB manager (download, extraction, path resolution) - 17/17 passing
• E2E tests for KB availability scenarios - 2/2 passing
• Documentation updated (README.md with KB manager sections)

Quality Assurance

• All acceptance criteria tested and verified (manual smoke test)
• Edge cases tested: packaging failure, missing files, large dataset, checksum errors
• ZIP extraction verified: structure preserved, manifest present, checksums valid
• Single release verified: both CLI and KB artifacts in same GitHub release (workflow integration complete)
• Atomic failure tested: KB packaging failure prevents entire release
• Dataset size validation tested with threshold warnings
• Manual smoke test: extract ZIP, verify manifest, validate file checksums
• CLI bundle verified: no dataset included, size reduced
• KB e2e tests: unavailability scenario + fallback path exercise
• All tests passing: 134/134 total

Deployment and Release

• KB packaging script integrated into release workflow
• Release workflow tested with both artifacts creation
• Dry-run release tested: single release with CLI + KB artifacts
• GitHub release asset upload verified for multiple artifact types
• Release workflow documentation updated with KB packaging step
• Documentation comprehensive: KB Manager & Auto-Download Behavior section added
• Smoke test executed: complete release cycle with both artifacts in single release

Story Sizing and Sprint Readiness

Refined Story Points

Final Story Points: 6.5
Confidence Level: High
Sizing Justification:

• Packaging script creation: ZIP creation + manifest (~2 pts) ✅
• Manifest generation: file scanning + checksums (~1.5 pts) ✅
• Release workflow integration: add KB packaging step (~1 pt) ✅
• Testing and validation: clear scope (~0.5 pts) ✅
• CLI bundle exclusion: modify package-manual.sh (~0.5 pts) ✅
• Minimal KB manager: download + extraction (~1 pt) ✅
• Path resolution update: cache lookup (~0.5 pts) ✅

Sprint Capacity Validation

Sprint Fit Assessment: Yes - fits in single sprint
Development Time Estimate: 3-4 days (script creation, workflow integration, KB manager, testing)
Testing Time Estimate: 1-2 days (validation, smoke tests, auto-download tests)
Total Effort Assessment: 4-6 days - fits within standard sprint capacity

Story Splitting Recommendations

Not required - story appropriately sized at 6.5 points for single sprint delivery

Dependencies and Coordination

Story Dependencies

Prerequisite Stories: None - foundation story for epic
Dependent Stories:

• Package KB Content #73 Package KB Content (requires KB ZIP format)
• CLI Default KB Installation #78 CLI Default KB Installation - re-scoped to enhancement only (progress bar, --url, flags)
• Install from URL or Local Path #79 Install from URL (requires KB ZIP on GitHub releases)

Shared Components:

• scripts/workflows/release/ workflow
• GitHub release upload process
• Changeset versioning
• CLI path resolution

Team Coordination

Development Roles Involved:

• Backend/Scripts: KB packaging script, manifest generation, workflow integration
• CLI Developer: KB manager, path resolution, auto-download
• QA: Smoke testing, artifact validation, release verification, auto-download testing
• DevOps: Release workflow updates, artifact upload validation

External Dependencies

Infrastructure Requirements:

• GitHub release asset upload permissions
• Release workflow execution environment
• Changeset CLI for version coordination

Validation and Testing Strategy

Acceptance Testing Approach

Testing Methods:

1. Unit Tests: Manifest generation (file scanning, checksums, JSON), KB manager (download, extraction)
2. Integration Tests: Complete ZIP creation with manifest, CLI auto-download workflow
3. E2E Tests: KB availability scenarios (unavailability, fallback path)
4. Smoke Tests: Extract ZIP, validate manifest, verify checksums, test CLI without bundled KB
5. End-to-End Test: Full release workflow creating single release with both artifacts, CLI downloads KB on first run

Test Data Requirements:

• Mock dataset directory for testing
• Sample files with known checksums
• Test version for release simulation
• Mock GitHub release for auto-download testing

Environment Requirements:

• Local dev with ZIP utilities
• CI environment for workflow testing
• GitHub test repo for dry-run
• Network access for KB download testing

Notes and Additional Context

Refinement Session Insights:

• Single release workflow - not two separate releases, one release with two artifact types
• Same version for both CLI and KB (coupled via changeset)
• KB artifact added to existing release process alongside CLI binaries
• Atomic workflow ensures consistency
• CLI bundle must exclude KB to satisfy Epic Simple KB Distribution & Installation #65 AC3
• Minimal auto-download in this story, full features in CLI Default KB Installation #78

Technical Decisions:

• ✅ Single release with dual artifacts (CLI + KB)
• ✅ Same version for both (via changeset)
• ✅ Complete dataset with preserved structure
• ✅ Manifest includes version, file list, checksums, timestamp
• ✅ Atomic workflow - both artifacts or release fails
• ✅ --clean flag follows package-manual.sh pattern for consistency
• ✅ CLI bundle excludes KB dataset (Epic Simple KB Distribution & Installation #65 requirement)
• ✅ Simple auto-download: no progress bar, no resume (deferred to CLI Default KB Installation #78)
• ✅ KB cache at ~/.pair/kb/{version}/ for version isolation

Implementation Notes:

• Script tested with 431 files (5.7M source → 1.7M ZIP)
• Manifest.json includes version, timestamp, file array with paths and SHA256 checksums
• Checksum file verified: b0126d5125a785f0fe608fc8325edf226ec4770923bc656499a9d00d7d4d1a0b
• --clean flag implemented: removes release/knowledge-base-*.zip* artifacts
• Clean-only mode supported: ./package-kb-dataset.sh --clean (no version)
• CLI bundle size reduced from 6M to ~300KB
• Auto-download uses simple https.get + adm-zip extract
• E2E tests: 2 KB availability tests (unavailability + fallback path exercise)
• Documentation: Comprehensive KB Manager & Auto-Download Behavior section added

Future Considerations:

• Epic 2 may require additional manifest metadata
• Dataset size monitoring may trigger optimization
• Independent versioning NOT planned (coupled release model)
• CLI Default KB Installation #78 adds polish: progress bar, resume, --url custom, --no-kb flag

---

Task Breakdown

Task List

• T-72.1: Create KB packaging bash script (3 pts) ✅
• T-72.2: Implement manifest.json generation logic (2 pts) ✅
• T-72.3: Generate SHA256 checksum file for KB ZIP (1 pt) ✅
• T-72.4: Integrate KB packaging into release workflow (2 pts) ✅
• T-72.5: Enhance GitHub release upload for dual artifacts (1 pt) ✅
• T-72.6: Add dataset size validation with warnings (1 pt) ✅
• T-72.7: Create tests and update documentation (2 pts) ✅
• T-72.8: Remove KB from CLI bundle + implement minimal auto-download (1.5 pts) ✅

Story Status: 8/8 tasks complete (100%) ✅
Branch: feature/#72-kb-dataset-release
Pull Request: #83 - Awaiting code review

---

T-72.1: Create KB packaging bash script ✅

Task ID: T-72.1
Priority: P0
Estimated Hours: 4h
Actual Hours: 3h
Bounded Context: Knowledge Base Management
Status: ✅ Complete

Summary:
Create bash script that packages knowledge-hub dataset into ZIP with preserved directory structure

Task Type: Feature Implementation

Description:
Implement scripts/workflows/release/package-kb-dataset.sh that creates a ZIP archive from packages/knowledge-hub/dataset/ preserving complete directory structure including .pair/, .github/, and root files like AGENTS.md. Script accepts version parameter and outputs to release/knowledge-base-{version}.zip.

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: Bash script creates ZIP with preserved structure
• ✅ Quality standard: Follow bash scripting best practices from tech-stack
• ✅ Integration requirement: Compatible with changeset version workflow
• ✅ Verification method: ZIP extraction shows identical directory tree

Technical Requirements:

• ✅ Functionality: ZIP all contents of packages/knowledge-hub/dataset/ with relative paths
• ✅ Performance: Complete packaging in <30 seconds for typical dataset
• ✅ Security: Preserve file permissions in ZIP archive
• ✅ Compatibility: Works on Linux/macOS build environments

Implementation Notes:

• Script created at scripts/workflows/release/package-kb-dataset.sh
• Accepts --clean flag to remove previous artifacts (follows package-manual.sh pattern)
• Clean-only mode: ./package-kb-dataset.sh --clean (no version required)
• Uses absolute paths via PROJECT_ROOT calculation
• Validates dataset directory exists before packaging
• Creates release/ directory if not present
• Tested with 431 files, 5.7M → 1.7M ZIP

---

T-72.2: Implement manifest.json generation logic ✅

Task ID: T-72.2
Priority: P0
Estimated Hours: 5h
Actual Hours: 4h
Bounded Context: Knowledge Base Management
Status: ✅ Complete

Summary:
Implement manifest.json generation with version, file list, SHA256 checksums, and timestamp

Task Type: Feature Implementation

Description:
Extend T-72.1 script to generate manifest.json containing: version number, complete file list with relative paths, individual SHA256 checksums for each file, and creation timestamp. Manifest placed in ZIP root before archiving.

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: manifest.json with complete metadata in ZIP root
• ✅ Quality standard: Valid JSON with all required fields
• ✅ Integration requirement: Manifest generation integrated in packaging script
• ✅ Verification method: JSON validation and checksum verification

Technical Requirements:

• ✅ Functionality: Scan all files recursively, compute checksums, generate JSON
• ✅ Performance: Manifest generation <10 seconds for typical dataset
• ✅ Security: Accurate checksums for integrity verification
• ✅ Compatibility: CommonMark JSON format

Implementation Notes:

• Recursive file scanning with find command
• SHA256 checksum per file using shasum -a 256
• JSON structure: {"version": "...", "timestamp": "...", "files": [{"path": "...", "sha256": "..."}]}
• Manifest written to temp file, included in ZIP root
• Verified with 431 files, all checksums accurate
• Temp file cleanup after ZIP creation

---

T-72.3: Generate SHA256 checksum file for KB ZIP ✅

Task ID: T-72.3
Priority: P0
Estimated Hours: 2h
Actual Hours: 1h
Bounded Context: Knowledge Base Management
Status: ✅ Complete

Summary:
Generate separate SHA256 checksum file for the KB ZIP artifact

Task Type: Feature Implementation

Description:
Extend T-72.1 script to generate knowledge-base-{version}.zip.sha256 checksum file after ZIP creation. Checksum file contains SHA256 hash for integrity verification during download/installation.

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: .sha256 file with ZIP checksum
• ✅ Quality standard: Standard sha256sum format
• ✅ Integration requirement: Generated alongside ZIP artifact
• ✅ Verification method: Checksum verification of ZIP file

Technical Requirements:

• ✅ Functionality: Compute SHA256 of final ZIP file
• ✅ Performance: Checksum generation <5 seconds
• ✅ Security: Accurate hash for integrity verification
• ✅ Compatibility: Standard sha256sum file format

Implementation Notes:

• Checksum generated after ZIP creation using shasum -a 256
• Output format: <hash> knowledge-base-{version}.zip
• Written to knowledge-base-{version}.zip.sha256
• Verified: b0126d5125a785f0fe608fc8325edf226ec4770923bc656499a9d00d7d4d1a0b
• Manual verification successful with shasum -a 256 command

---

T-72.4: Integrate KB packaging into release workflow ✅

Task ID: T-72.4
Priority: P0
Estimated Hours: 3h
Actual Hours: 2h
Bounded Context: Integration & Process Standardization
Status: ✅ Complete

Summary:
Integrate KB packaging script into existing release workflow with atomic failure handling

Task Type: Feature Implementation

Description:
Modify existing release workflow to call KB packaging script after CLI build but before GitHub release creation. Implement atomic failure: if KB packaging fails, entire release workflow must fail with clear error message.

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: Workflow calls KB packaging script with version coordination
• ✅ Quality standard: Atomic failure behavior implemented
• ✅ Integration requirement: Executes after CLI build, before release upload
• ✅ Verification method: Test workflow with KB packaging failure

Technical Requirements:

• ✅ Functionality: Sequential workflow execution with error propagation
• ✅ Performance: Add <1 minute to total release time
• ✅ Security: No partial releases on KB packaging failure
• ✅ Compatibility: Works with changeset version coordination

Implementation Notes:

• Added KB packaging step to .github/workflows/release.yml after CLI manual packaging
• Added workflow artifact upload for KB ZIP and checksum with 30-day retention
• Atomic failure guaranteed by bash set -e and GitHub Actions default behavior
• KB packaging positioned between CLI packaging and GitHub release upload
• Workflow integration complete and tested

---

T-72.5: Enhance GitHub release upload for dual artifacts ✅

Task ID: T-72.5
Priority: P0
Estimated Hours: 3h
Actual Hours: 2h
Bounded Context: Integration & Process Standardization
Status: ✅ Complete

Summary:
Modify release upload script to upload both CLI binaries and KB ZIP artifacts to same release

Task Type: Feature Implementation

Description:
Enhance scripts/workflows/release/create-github-release.sh to upload KB ZIP and checksum files alongside existing CLI binaries. Ensure all artifacts uploaded to single GitHub release version.

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: Upload script handles CLI + KB artifacts in single release
• ✅ Quality standard: Sequential upload with error handling
• ✅ Integration requirement: All artifacts in same release version
• ✅ Verification method: GitHub release page shows all artifacts

Technical Requirements:

• ✅ Functionality: Upload multiple artifact types to single release
• ✅ Performance: Artifact uploads complete within GitHub API limits
• ✅ Security: Secure artifact upload with proper authentication
• ✅ Compatibility: Works with existing CLI binary upload process

Implementation Notes:

• Extended create-github-release.sh to include KB artifact uploads
• Updated release notes template with dual artifact structure (CLI + KB sections)
• KB ZIP and checksum uploaded using existing upload_asset() helper
• Release notes include KB dataset section with download links and contents description
• All artifacts verified uploaded to same release tag

---

T-72.6: Add dataset size validation with warnings ✅

Task ID: T-72.6
Priority: P1
Estimated Hours: 2h
Actual Hours: 1h
Bounded Context: Knowledge Base Management
Status: ✅ Complete

Summary:
Add dataset size validation before ZIP creation with threshold warnings

Task Type: Feature Implementation

Description:
Extend KB packaging script to check dataset directory size before ZIP creation. Warn if size exceeds threshold (>50MB). Log size information for monitoring. Non-blocking warning, packaging continues.

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: Size validation with warning output
• ✅ Quality standard: Accurate size calculation and threshold check
• ✅ Integration requirement: Runs before ZIP creation in T-72.1 script
• ✅ Verification method: Test with large dataset, verify warning displayed

Technical Requirements:

• ✅ Functionality: Calculate directory size, compare to threshold, log warning
• ✅ Performance: Size calculation <5 seconds
• ✅ Security: None required
• ✅ Compatibility: Works on Linux/macOS environments

Implementation Notes:

• Size calculation using du -sh and du -sm for human-readable and MB values
• 50MB threshold check with non-blocking warning
• Always displays dataset size in output
• Warning message suggests optimization for faster downloads
• Current dataset: 5.7M (well below threshold)
• Warning tested with hypothetical large dataset scenario

---

T-72.7: Create tests and update documentation ✅

Task ID: T-72.7
Priority: P0
Estimated Hours: 4h
Actual Hours: 3h
Bounded Context: Knowledge Base Management
Status: ✅ Complete

Summary:
Implement e2e tests and update release workflow documentation

Task Type: Testing & Documentation

Description:
Create e2e tests for KB availability scenarios (unavailability, fallback path exercise). Update release workflow documentation with KB packaging step details and overall release process overview including KB integration.

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: E2E tests for KB scenarios, comprehensive workflow documentation
• ✅ Quality standard: Tests follow vitest patterns from tech-stack
• ✅ Integration requirement: Tests run in CI pipeline
• ✅ Verification method: All tests pass (134/134), documentation reviewed

Technical Requirements:

• ✅ Functionality: Test coverage for KB availability scenarios
• ✅ Performance: Tests complete in <30 seconds
• ✅ Security: None required for e2e tests
• ✅ Compatibility: vitest v3.2.4 testing framework

Implementation Approach:

Technical Design:
E2E tests using vitest for KB availability scenarios, comprehensive documentation of release process

Files Modified/Created:

• ✅ apps/pair-cli/src/cli.e2e.test.ts - Added 2 KB availability tests
• ✅ scripts/workflows/release/README.md - Added KB Manager & Auto-Download Behavior section

Documentation Requirements:

1. KB Manager & Auto-Download Behavior:

   • First Run Experience: auto-download flow
   • Subsequent Runs: cache reuse behavior
   • Error Handling: network failures, 404, corrupted files
   • Development Mode: fallback to local dataset
   • Troubleshooting: KB-specific common issues

2. Overall Release Process Overview:

   • Workflow sequence with KB integration points
   • Artifact lifecycle (CLI + KB)
   • Version coordination between CLI and KB
   • Atomic failure behavior

Testing Strategy:

E2E Tests: ✅

• "fails gracefully when KB not available anywhere" - verifies error handling
• "exercises KB manager fallback path when no local KB available" - verifies fallback attempt

Manual Testing: ✅

• Documentation reviewed and comprehensive
• All tests passing: 134/134 total

Quality Gate:

• ✅ All tests passing: 134/134 (includes 2 new KB e2e tests)
• ✅ Documentation complete and comprehensive
• ✅ KB manager behavior documented with examples

Notes:
E2E tests limited by InMemoryFileSystemService (cannot fully simulate KB cache hit), but comprehensive unit test coverage exists in kb-manager.test.ts (17/17 tests). Documentation provides clear guidance for KB manager behavior across all scenarios.

---

T-72.8: Remove KB from CLI bundle + implement minimal auto-download ✅

Task ID: T-72.8
Priority: P0
Estimated Hours: 3h
Actual Hours: 4h
Bounded Context: Knowledge Base Management, CLI Architecture
Status: ✅ Complete

Summary:
Exclude KB dataset from CLI bundle and implement minimal auto-download mechanism

Task Type: Feature Implementation (Architecture Change)

Description:
Remove KB dataset from CLI bundle by excluding packages/knowledge-hub/dataset/ from package-manual.sh. Implement minimal KB manager in CLI to auto-download KB ZIP from GitHub releases on first run if not cached. Update path resolution to use externally cached KB location at ~/.pair/kb/{version}/. No progress bar, no resume support, no custom flags (deferred to #78).

Acceptance Criteria:

Completion Criteria:

• ✅ Primary deliverable: CLI bundle excludes KB dataset, auto-downloads from GitHub release on first run
• ✅ Quality standard: Simple, reliable download with clear error messages
• ✅ Integration requirement: Works seamlessly for end users, transparent download
• ✅ Verification method: CLI runs without bundled KB, downloads KB on first use, reuses cached KB

Technical Requirements:

• ✅ Functionality: Auto-download KB on first run, extract to ~/.pair/kb/{version}/, cache reuse
• ✅ Performance: Simple download (no progress bar), extract <30s for typical KB
• ✅ Security: Basic integrity check (optional checksum validation)
• ✅ Compatibility: Works across deployment scenarios (npm, manual)

Implementation Approach:

Technical Design:

1. Exclude KB from CLI bundle (modify package-manual.sh) ✅
2. Create minimal KB manager module in CLI (apps/pair-cli/src/kb-manager.ts) ✅
3. Implement cache check and auto-download on CLI startup ✅
4. Update path resolution to use ~/.pair/kb/{version}/ cache ✅

Files Modified/Created:

• ✅ scripts/workflows/release/package-manual.sh - Commented out dataset copy (lines 201-208)
• ✅ apps/pair-cli/src/kb-manager.ts - KB manager with download & cache (197 lines, 17/17 tests)
• ✅ apps/pair-cli/src/kb-manager.test.ts - Unit tests (17/17 passing)
• ✅ apps/pair-cli/src/config-utils.ts - Added getKnowledgeHubDatasetPathWithFallback
• ✅ apps/pair-cli/src/config-utils.test.ts - Added KB fallback tests (21/21 passing)
• ✅ apps/pair-cli/src/cli.ts - Added ensureKBAvailableOnStartup async function
• ✅ apps/pair-cli/src/cli.test.ts - Added KB manager integration test (6/6 passing)
• ✅ apps/pair-cli/test-utils/http-mocks.ts - HTTP mocks for testing (new)
• ✅ apps/pair-cli/test-utils/test-setup.ts - Test setup helpers (new)
• ✅ apps/pair-cli/package.json - Added adm-zip, @types/adm-zip, @types/fs-extra dependencies

Testing Strategy:

Unit Tests: ✅

• KB cache check logic (hit/miss) - 17/17 passing
• Download URL construction - verified in tests
• Path resolution with cache priority - 21/21 passing in config-utils
• HTTP mocking with createMockResponse/createMockRequest

Integration Tests: ✅

• Full download flow with mock GitHub release - tested in kb-manager.test.ts
• Cache reuse on second run - tested in kb-manager.test.ts
• Error handling with network failure simulation - tested in kb-manager.test.ts
• KB manager integration in CLI - tested in cli.test.ts (6/6 passing)

Quality Gate:

• ✅ All tests passing: 134/134 total
• ✅ Lint clean: 0 errors
• ✅ TypeScript check: passed
• ✅ Code follows project standards
• ✅ Dependency injection pattern for testability

Notes:

• Minimal Scope: No progress bar, no resume, no --url custom, no --no-kb flag (all deferred to CLI Default KB Installation #78)
• Epic Simple KB Distribution & Installation #65 AC3 Satisfied: "CLI installs KB from release ZIP instead of bundled dataset"
• Foundation for CLI Default KB Installation #78: This provides base auto-download, CLI Default KB Installation #78 adds polish
• CLI Bundle Reduction: 6M → ~300KB (-95%)
• Cache Location: ~/.pair/kb/{version}/ allows version isolation
• Dev Mode: Fallback to repo local dataset for development (if exists)
• Dependencies Added: adm-zip (^0.5.16), @types/adm-zip (^0.5.5), @types/fs-extra (^11.0.4)