Story: Data Collection Infrastructure for Dashboard Stats

[prosdev]

Branch: feat/stats-aggregation
Priority: Highest
Parent Epic: #145
Type: Story

Description

Implement the data collection infrastructure needed to support dashboard visualizations. This includes extending the IndexStats interface, creating a StatsAggregator class, and updating scanners to track detailed metadata about languages, component types, and package structure.

This is foundational work that must be completed before CLI enhancements or web dashboard can display rich statistics.

Acceptance Criteria

• Extended IndexStats interface includes language breakdown
• Extended IndexStats interface includes component type counts
• Extended IndexStats interface includes package-level stats (for monorepos)
• StatsAggregator class efficiently collects stats during indexing
• Scanners (TypeScript, Go) report language and component metadata
• Indexer aggregates stats with <5% performance overhead
• Stats are persisted to state file for retrieval
• Backward compatibility maintained (old state files still work)

Technical Requirements

Storage: Extend existing indexer-state.json file

New Types:

interface DetailedIndexStats extends IndexStats {
  byLanguage: Record<SupportedLanguage, LanguageStats>;
  byComponentType: Record<ComponentType, number>;
  byPackage?: Record<string, PackageStats>;
}

New Classes:

• StatsAggregator class in packages/core/src/indexer/stats-aggregator.ts
• Methods: addFile(), addComponent(), getAggregatedStats()
• Efficient incremental aggregation (no post-processing)

Scanner Updates:

• TypeScript scanner: Report language (TS vs JS) per file
• Go scanner: Report Go language per file
• Both: Include component type in metadata

Testing:

• Unit tests for StatsAggregator
• Integration tests with scanners
• Performance benchmarks (<5% overhead)
• Backward compatibility tests

Files to Create:

• packages/core/src/indexer/stats-aggregator.ts
• packages/core/src/indexer/__tests__/stats-aggregator.test.ts

Files to Modify:

• packages/core/src/indexer/types.ts
• packages/core/src/indexer/index.ts
• packages/core/src/scanner/typescript.ts
• packages/core/src/scanner/go.ts

[prosdev]

✅ Implementation complete!

PR created: #155

What's included:

• StatsAggregator class with streaming O(1) aggregation
• Language-specific stats (TypeScript, JavaScript, Go, Markdown)
• Component type counting (function, class, interface, type, variable)
• Package/monorepo detection infrastructure
• TypeScript vs JavaScript distinction based on file extension

Testing:

• 20 new tests (14 unit + 6 integration)
• All 508 core tests passing
• Performance: <5% overhead verified

Next Steps:
Ready for review and merge. Once merged, can proceed with #147 (Enhanced CLI).