tokenlean

Lean CLI tools for AI agents — maximum insight, minimum tokens

Installation • Quick Start • Tools • AI Integration • Configuration

---

Why tokenlean?

AI coding assistants are powerful, but they have a fundamental constraint: context windows. Every file read, every search result consumes tokens. This matters because:

Problem	Impact
Cost	More tokens = higher API costs
Quality	Overstuffed context = worse responses
Speed	Larger contexts = longer processing
Limits	Hit the ceiling = lost information

tokenlean provides 32 specialized CLI tools that give you exactly the information needed — no more, no less.

Instead of reading a 500-line file    →  tl-exports (~50 tokens)
Instead of reading all type files     →  tl-types (~100 tokens)
Instead of guessing impact            →  tl-impact (know for sure)

Installation

npm install -g tokenlean

Development setup

git clone https://github.com/edimuj/tokenlean.git
cd tokenlean
npm link

Requirements

• Node.js >= 18.0.0
• ripgrep (rg) — for search-based tools
• git — for history/blame/diff tools

Quick Start

# Get project overview
tl-structure

# Check file sizes before reading
tl-context src/api/

# Get function signatures without bodies
tl-symbols src/utils.ts

# Understand what a module exports
tl-exports src/lib/

# Check what would break if you change a file
tl-impact src/core/auth.ts

AI Agent Integration

Add tokenlean instructions to your AI tool's config:

AI Tool	Config File	Command
Claude Code	CLAUDE.md	tl-prompt >> CLAUDE.md
Cursor	.cursorrules	tl-prompt --minimal >> .cursorrules
GitHub Copilot	.github/copilot-instructions.md	tl-prompt >> .github/copilot-instructions.md
Windsurf	.windsurfrules	tl-prompt --minimal >> .windsurfrules

The --minimal flag produces a compact version that uses fewer tokens.

Tools Reference

Before Reading Files

Understand code structure without reading full implementations.

Tool	Description	Example
tl-structure	Project overview with token estimates	tl-structure src/
tl-context	Estimate token usage for files/dirs	tl-context src/api/
tl-symbols	Function/class signatures without bodies	tl-symbols src/utils.ts
tl-types	Full TypeScript type definitions	tl-types src/types/
tl-exports	Public API surface of a module	tl-exports src/lib/
tl-component	React component analyzer (props, hooks)	tl-component Button.tsx
tl-docs	Extract JSDoc/TSDoc documentation	tl-docs src/utils/
tl-entry	Find entry points and main files	tl-entry src/
tl-schema	Extract DB schema from ORMs/migrations	tl-schema

Before Modifying Files

Understand dependencies and impact before making changes.

Tool	Description	Example
tl-impact	Blast radius — what depends on this file	tl-impact src/auth.ts
tl-deps	Show what a file imports (with tree mode)	tl-deps src/api.ts --tree
tl-related	Find tests, types, and importers	tl-related src/Button.tsx
tl-flow	Call graph — what calls this, what it calls	tl-flow src/utils.ts
tl-coverage	Test coverage info for files	tl-coverage src/
tl-complexity	Code complexity metrics	tl-complexity src/ --threshold 10

Understanding History

Track changes and authorship efficiently.

Tool	Description	Example
tl-diff	Token-efficient git diff summary	tl-diff --staged
tl-history	Recent commits for a file	tl-history src/api.ts
tl-blame	Compact per-line authorship	tl-blame src/api.ts
tl-hotspots	Frequently changed files (churn)	tl-hotspots --days 30
tl-pr	Summarize PR/branch for review	tl-pr feature-branch
tl-changelog	Generate changelog from commits	tl-changelog --from v1

Finding Things

Search and discover code patterns.

Tool	Description	Example
tl-search	Run pre-defined search patterns	tl-search hooks
tl-secrets	Find hardcoded secrets & API keys	tl-secrets --staged
tl-todo	Find TODOs/FIXMEs in codebase	tl-todo src/
tl-env	Find environment variables used	tl-env --required-only
tl-unused	Find unused exports/files	tl-unused src/
tl-api	Extract REST/GraphQL endpoints	tl-api src/routes/
tl-routes	Extract routes from web frameworks	tl-routes app/

Utilities

Tool	Description	Example
tl-cache	Manage ripgrep result cache	tl-cache stats
tl-config	Show/manage configuration	tl-config --init
tl-name	Check name availability (npm/GH)	tl-name myproject -s
tl-prompt	Generate AI agent instructions	tl-prompt --minimal

Common Options

All tools support these flags:

-l N, --max-lines N    Limit output to N lines
-t N, --max-tokens N   Limit output to ~N tokens
-j, --json             Output as JSON (for piping)
-q, --quiet            Minimal output (no headers/stats)
-h, --help             Show help

Configuration

Create .tokenleanrc.json in your project root or ~/.tokenleanrc.json globally:

{
  "output": {
    "maxLines": 100,
    "maxTokens": null
  },
  "skipDirs": [
    "generated",
    "vendor"
  ],
  "skipExtensions": [
    ".gen.ts"
  ],
  "importantDirs": [
    "domain",
    "core"
  ],
  "importantFiles": [
    "ARCHITECTURE.md"
  ],
  "searchPatterns": {
    "hooks": {
      "description": "Find React hooks",
      "pattern": "use[A-Z]\\w+",
      "glob": "**/*.{ts,tsx}"
    }
  }
}

Full configuration reference

{
  "output": {
    "maxLines": 100,
    "maxTokens": null
  },
  "skipDirs": [
    "generated",
    "vendor"
  ],
  "skipExtensions": [
    ".gen.ts"
  ],
  "importantDirs": [
    "domain",
    "core"
  ],
  "importantFiles": [
    "ARCHITECTURE.md"
  ],
  "searchPatterns": {
    "hooks": {
      "description": "Find React hooks",
      "pattern": "use[A-Z]\\w+",
      "glob": "**/*.{ts,tsx}"
    }
  },
  "hotspots": {
    "days": 90
  },
  "structure": {
    "depth": 3
  },
  "cache": {
    "enabled": true,
    "ttl": 300,
    "maxSize": "100MB",
    "location": null
  }
}

Config values extend built-in defaults (they don't replace them).

Caching

tokenlean caches expensive ripgrep operations with git-based invalidation — automatically invalidates on commits or file changes.

tl-cache stats      # View cache statistics
tl-cache clear      # Clear cache for current project
tl-cache clear-all  # Clear all cached data

Disable with TOKENLEAN_CACHE=0 or in config: {"cache":{"enabled":false}}

Example Workflows

Starting on an unfamiliar codebase

tl-structure                    # Get the lay of the land
tl-entry                        # Find entry points
tl-exports src/lib/             # Understand the public API
tl-docs src/utils/              # Read documentation, not code
tl-types src/types/             # Understand data shapes
tl-schema                       # Understand the database

Before refactoring a file

tl-impact src/core/auth.ts      # What would break?
tl-deps src/core/auth.ts        # What does it depend on?
tl-related src/core/auth.ts     # Find the tests
tl-coverage src/core/auth.ts    # Is it well tested?
tl-complexity src/core/auth.ts  # How complex is it?

Understanding a component

tl-component src/Button.tsx     # Props, hooks, dependencies
tl-symbols src/Button.tsx       # Function signatures
tl-history src/Button.tsx       # Recent changes
tl-blame src/Button.tsx         # Who wrote what

Finding technical debt

tl-complexity src/ --threshold 15  # Complex functions
tl-unused src/                     # Dead code
tl-todo                            # Outstanding TODOs
tl-hotspots                        # Frequently changed (unstable?)

Security check before committing

tl-secrets                         # Scan for hardcoded secrets
tl-secrets --staged                # Only check staged files
tl-secrets --min-severity high     # Only high severity issues

Reviewing a PR

tl-pr feature-branch               # Summary of branch changes
tl-pr 123                          # GitHub PR #123 (needs gh CLI)
tl-pr --full                       # Include files, stats, commits

Preparing a release

tl-changelog --unreleased          # What's new since last tag
tl-changelog v0.1.0..v0.2.0        # Between versions
tl-changelog --format compact      # Quick summary

Starting a new project

tl-name coolproject awesomelib     # Check npm, GitHub, domains
tl-name myapp -s                   # Suggest variations if taken
tl-name myapp --tld io             # Check .io domain

Design Principles

1. Single purpose — Each tool does one thing well
2. Minimal output — Show only what's needed
3. Token-conscious — Every tool saves context tokens
4. Composable — Tools work together with JSON output for piping
5. Fast — No heavy parsing or external services
6. Universal — Works with JS/TS projects, most tools support Python/Go too

Other tools for Claude Code

Project	Description
claude-mneme	Persistent memory for Claude Code — remember context across sessions
claude-simple-status	Minimal statusline showing model, context usage, and quota
vexscan-claude-code	Security scanner protecting against untrusted plugins, skills, MCPs, and hooks

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT © Edin Mujkanovic