generate command

generate — Parse & Build the Graph

The core command. Discovers source files, parses them with tree-sitter ASTs, builds a dependency graph, detects communities, and exports the interactive visualizer.

codebase-vis generate

Pipeline

Source files
    │
    ▼
.discoverFiles() ──── .agentignore (5-layer filter)
    │
    ▼
.splitFilesByCache() ── .cache.json (mtime + size fingerprint)
    │
    ├── cached → skip
    └── fresh → WorkerPool (fork() × CPU-1)
                    │
                    ▼
              tree-sitter AST per language
                    │
                    ▼
              { dependencies, entities }
                    │
                    ▼
buildGraph() ── graphology (directed multi-graph)
    │
    ▼
enrichNodes() ── Louvain community detection
    │               → undirected subgraph
    │               → graphology-communities-louvain
    │               → directory-based naming (disambiguated)
    │               → 12-color palette assignment
    │
    ▼
exportGraphToJson() ── graph.json
getHtmlTemplate()  ── graph.html (self-contained)

Step by Step

1. File Discovery

discoverFiles() walks the target directory recursively, checking every entry against .agentignore patterns. Only files with known extensions (.js, .ts, .py, .cpp, .html, .css, .rs, .go, .java, etc.) are collected. Symlinks are skipped. Files larger than 2 MB are skipped.

2. Cache Lookup

splitFilesByCache() compares each discovered file's current mtime and size against the previous run's .cache.json. Matches are restored from cache (zero parsing cost). Mismatches and new files are queued for fresh parsing.

3. Parallel Parsing

Changed files are distributed to a WorkerPool of child processes (one per CPU core minus 1). Each worker:

• Loads its own tree-sitter parsers (grammars cached per-process)
• Parses the file's AST
• Extracts import/require/include statements as dependencies
• Extracts classes, functions, methods, and docstrings as entities
• Sends results back to the main process via IPC

Results are collected in input order regardless of completion order. If a worker crashes, the pool spawns a replacement and re-queues the task.

4. Graph Construction

buildGraph() creates a directed multi-graph using graphology:

• File nodes — one per source file
• Entity sub-nodes — classes, functions, methods in their parent file, linked via contains edges
• External packages — npm packages and other imports not found locally, marked external: true

Dependency resolution handles relative paths (./, ../), bare imports (checked against local files first), and npm package detection via package.json.

5. Community Detection

enrichNodes() runs Louvain community detection on the file nodes:

1. Builds an undirected subgraph of file-to-file edges
2. Runs graphology-communities-louvain to maximize modularity
3. Names each community by its most frequent directory
4. Disambiguates duplicates with #1, #2 suffixes
5. Assigns colors from a 12-color palette
6. Sets visual attributes: size proportional to degree, initial positions for ForceAtlas2

6. Export

• graph.json — full graph data in graphology export format
• graph.html — self-contained interactive visualizer

Flags

Flag	Default	Description
[paths...]	"." (cwd)	One or more directories/files to scan
--ignore	—	Comma-separated additional ignore patterns
--verbose	false	Show detailed processing info
--jobs	CPU - 1	Number of parallel parse workers
--no-clear	false	Don't clear terminal before output

Examples

codebase-vis generate                                          # Scan entire project
codebase-vis generate src/ lib/                                # Scan specific directories
codebase-vis generate --ignore tests,fixtures                  # Additional ignores
codebase-vis generate --jobs 4                                 # Use 4 workers
codebase-vis generate --verbose                                # Verbose logging
codebase-vis generate src/ --verbose --jobs 2                  # Combine flags

Output

All output goes to codebase-out/:

• graph.json — graph data
• graph.html — interactive visualizer
• .cache.json — incremental cache for next run

See Output Files for details.