Skip to content

v0.4.0

Latest

Choose a tag to compare

@github-actions github-actions released this 08 Jan 09:25

Breaking Changes

  • Default output format is now "compact" with minimal markers (~15% token reduction)
    • Use --format standard for the previous verbose format
  • estimate_tokens() signature changed: added model and use_tiktoken parameters

New Features

Compact Output Format (New Default)

The compact format uses ## FILE: path headers instead of verbose --- START FILE: path --- markers, significantly reducing token overhead:

# Project Context

## FILE: src/main.py
```python
print("hello")

FILE: logo.png

[BINARY: PNG 256×256, 45.3 KB]


### Model-Specific Token Estimation (`--model`)
Get more accurate token estimates based on your target LLM:
- `--model claude` - Uses chars/3.5 heuristic (Anthropic recommendation)
- `--model gpt-4` - Uses chars/4 heuristic (default)
- `--model gemini` - Uses chars/4 heuristic
- `--model llama` - Uses chars/3.8 heuristic

### Optional Tiktoken Support (`--tokenizer`)
For 100% accurate GPT token counts:
```bash
pip install llmcontext[tiktoken]
llmcontext . output.txt --tokenizer tiktoken

Installation

pip install llmcontext
# Or with tiktoken support:
pip install llmcontext[tiktoken]

Full Changelog

See CHANGELOG.md