Architecture Overview

Architecture Overview

codebase-vis transforms source code into interactive dependency graphs through a multi-stage pipeline.

High-Level Pipeline

┌──────────┐    ┌───────────┐    ┌──────────┐    ┌───────────┐    ┌──────────┐
│ Discover │───→│   Parse   │───→│  Build   │───→│  Enrich   │───→│  Export  │
│  Files   │    │   ASTs    │    │  Graph   │    │Communities│    │HTML+JSON │
└──────────┘    └───────────┘    └──────────┘    └───────────┘    └──────────┘
     │               │               │               │               │
     ▼               ▼               ▼               ▼               ▼
 .agentignore   WorkerPool      graphology       Louvain        codebase-out/
 (ignore)     fork() × CPU-1   directed MGraph  detection      graph.html
                                                                 graph.json

Pipeline Stages

1. File Discovery

discoverFiles() walks the target directory, checking every entry against .agentignore patterns. Only known file extensions (.js, .ts, .py, .cpp, .html, .css, .rs, .go, .java) are collected. Symlinks and files > 2 MB are skipped. Directory recursion runs with 32-way concurrency.

2. AST Parsing

Files are parsed using tree-sitter — incremental, error-tolerant parsers that produce concrete syntax trees. Each language has dedicated S-expression queries that capture import statements and entity declarations (classes, functions, methods, docstrings). Parsing is CPU-bound, so it runs in a parallel worker pool.

3. Graph Construction

buildGraph() creates a directed multi-graph using graphology. File nodes are connected by dependency edges. Entity sub-nodes (classes, functions) are connected to their parent files via contains edges. External packages (npm, etc.) are added as separate nodes.

4. Community Detection

enrichNodes() runs Louvain community detection to group files into modules. Communities are named by their most frequent directory and assigned colors from a 12-color palette. This powers the color-coded legend in the visualizer.

5. Export

The graph is exported as graph.json (graphology format) and embedded into graph.html — a self-contained interactive visualizer with vis-network, ForceAtlas2 physics, search, minimap, and cycle overlay.

Key Design Principles

• Zero-cloud local-first — every feature except explain runs 100% locally. No accounts, no telemetry, no servers.
• Incremental by default — the .cache.json file skips re-parsing of unchanged files on repeated runs.
• Sandboxed output — all generated files are constrained to codebase-out/. Path traversal is blocked.
• Resilient parsing — the worker pool handles crashes gracefully. A single corrupted file can't stall the pipeline.

Module Structure

src/
├── cli/
│   ├── shared.js              Shared CLI utilities (loadGraph, resolveNode, etc.)
│   └── commands/
│       ├── index.js           Barrel exports
│       ├── init.js            .agentignore creator
│       ├── generate.js        Main pipeline orchestrator
│       ├── serve.js           HTTP server
│       ├── query.js           Dependency inspector
│       ├── path.js            Shortest path finder
│       ├── detect.js          Cycle detector
│       ├── explain.js         LLM summarizer
│       └── clean.js           Output cleaner
├── parser/
│   ├── index.js               Parser orchestrator + WorkerPool
│   ├── languages.js           Language definitions + tech stack markers
│   ├── stack-detector.js      Tech stack detection
│   ├── parse-worker.js        Forked child process worker
│   ├── javascript.js          JS/JSX parser
│   ├── typescript.js          TS/TSX parser
│   ├── python.js              Python parser
│   ├── cpp.js                 C/C++ parser
│   ├── java.js                Java parser
│   ├── go.js                  Go parser
│   ├── rust.js                Rust parser
│   ├── html.js                HTML parser
│   └── css.js                 CSS parser
├── graph/
│   ├── builder.js             Graph construction from parsed data
│   ├── enricher.js            Louvain community detection + visual attributes
│   ├── formatter.js           JSON export
│   └── cycle-detector.js      DFS-based cycle detection
├── utils/
│   ├── file-system.js         Output directory + sandboxed writes
│   ├── traversal.js           Recursive file walker
│   ├── cache.js               Incremental parse cache
│   └── worker-pool.js         Fork-based worker pool
└── templates/
    ├── graph-template.js      Template loader
    └── graph.html             Self-contained visualizer

Data Flow

Source files
    │
    ▼
discoverFiles()  ─── filtered list of file paths
    │
    ▼
splitFilesByCache()  ─── { toParse[], cachedResults[] }
    │
    ├── cached → skip parsing
    └── toParse → parseFileBatch() → WorkerPool
                     │
                     ▼
              For each file:
                readFile()
                tree-sitter parse
                query dependencies
                query entities
                return { id, dependencies, entities }
    │
    ▼
buildGraph()  ─── graphology Graph instance
    │
    ▼
enrichNodes()  ─── communities + colors + sizes
    │
    ▼
exportGraphToJson() → graph.json
getHtmlTemplate()  → graph.html
buildUpdatedCache() → .cache.json

Tech Stack

Component	Library
CLI framework	commander
AST parsing	tree-sitter + language grammars
Graph engine	graphology
Community detection	graphology-communities-louvain
Ignore patterns	ignore
Terminal UI	@clack/prompts, picocolors
Visualization	vis-network + ForceAtlas2
LLM API	Groq (OpenAI-compatible)
Runtime	Node.js >= 18 (ESM)