v0.8.1 - Display Input/Output tokens

🔧 Display

wtk gain now shows Input tokens / Output tokens lines alongside Input chars / Output chars when cl100k data is present:

Total commands:    3
Input chars:       3.8K
Output chars:      1.5K
Chars saved:       2.3K (60.2%)
Input tokens:      1.1K
Output tokens:     556
Tokens saved:      516 (48.1%) (2/3 rows, cl100k)

Real numbers reveal the chars/4 heuristic gap — chars saved 60.2% but real token savings only 48.1% on this sample.

Full Changelog: v0.8.0...v0.8.1

v0.8.0 - Real BPE tokenizer (cl100k) + honesty fixes

✨ Features

Real LLM tokenizer (opt-in)

WTK now ships an optional cl100k_base BPE tokenizer (OpenAI GPT-3.5/4, ~95% Claude proxy). Default stays the fast bytes / 4 heuristic — turn on real counts when you need precision:

# Per-call
WTK_TOKENIZER=cl100k wtk find . -name "*.rs"

# Persistent
export WTK_TOKENIZER=cl100k

# Or in ~/.config/wtk/config.toml
[tracking]
tokenizer = "cl100k"

Adds ~5-30ms per filter call. The cl100k BPE table loads lazily on first use.

Reports show both chars and tokens

wtk gain now displays Chars saved and (when DB has tokenized rows) Tokens saved side by side, with a N/M rows, cl100k coverage indicator. The Active tokenizer: line at the bottom of the summary confirms the live mode.

🗄️ Database migration

Forward-only on next open: adds tokens_input, tokens_output, tokenizer_kind columns. Existing rows keep NULL for these — no data loss, queries handle the mixed state via COALESCE / IS NOT NULL.

🐛 Honesty fixes

• Removed misleading "tokens" labels where the underlying number was bytes
• ≈ X tokens line now reads ~X (heuristic: chars÷4, ASCII English) with explicit caveat
• Internal format_tokens renamed to format_count to match its actual job

📦 Filter coverage

Heavy hitters migrated to retain raw text for accurate cl100k counting: find / fd, grep / rg, git, env, windows/* (tasklist, ipconfig, ...), all PowerShell cmdlets. Other filters keep the legacy path and contribute char counts only when cl100k is active.

⚠️ Why this matters

Live test on wtk ls /tmp (large file dump):

• Chars saved: 173.5K
• Heuristic estimate: ~43K tokens
• Real cl100k count: 89.4K tokens — heuristic was off by 2x

File paths and structured output don't fit the chars/4 rule. cl100k mode reveals the truth.

📦 Install

```powershell
irm https://raw.githubusercontent.com/Lukreitor/wtk/master/install.ps1 | iex
```

Binary size: 5MB → 7.1MB (+2MB for cl100k BPE table).

Full Changelog: v0.7.0...v0.8.0

v0.7.0 - Search filters + real-world stats

✨ Features

• filters: Add grep / rg / ripgrep filter — group matches by file, configurable context
• filters: Add find / fd / fdfind filter — group results by directory, compact listings
• filters: Add env / printenv filter — hide system vars, mask *_KEY / *_SECRET / *_TOKEN, format PATH

📊 Real-World Stats

Updated README with actual wtk gain data:

• 924 commands tracked over 30 days
• 8.6M tokens saved (95.3% efficiency)
• 9.0M input tokens filtered → 420.6K output tokens
• Recursive find scans = top savers (60K–6.4M tokens per call)

📦 Install

```powershell
irm https://raw.githubusercontent.com/Lukreitor/wtk/master/install.ps1 | iex
```

Full Changelog: v0.6.1...v0.7.0

v0.6.1 - Fix Claude Code Hook Protocol

Bug Fixes

hooks: Fix matcher format for `wtk init --claude-code`

• Was: `"matcher": { "tool_name": "Bash" }` (invalid object — Claude Code ignores it)
• Now: `"matcher": "Bash"` (correct string format per Claude Code spec)

rewrite: Fix hook stdin protocol

• Was: Required positional `` argument — all hook invocations failed with "required argument not provided"
• Now: Reads JSON from stdin when invoked without args, extracts `tool_input.command` per Claude Code PreToolUse hook protocol
• Backwards compatible: Still accepts optional positional arg for manual testing

Upgrade

Replace your binary or run:
```powershell
cargo install --path .
wtk init --claude-code # Reinstall hook with correct config
```

Or reinstall the hook manually in `~/.claude/settings.json`:
```json
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [{ "type": "command", "command": "C:/path/to/wtk.exe rewrite" }]
}
]
}
}
```

v0.6.0 - Fix Claude Code Hook Installation

Bug Fixes

• hooks: Fix Claude Code hook installation to use absolute path with forward slashes
  • Changed from relative wtk to absolute C:/Users/.../.cargo/bin/wtk.exe
  • Added path normalization with canonicalize() + UNC prefix removal
  • Claude Code now correctly invokes wtk on Windows
  • Resolves hook execution failures on system paths

Improvements

• Improved hook installer compatibility with Windows path formats
• Better cross-platform path handling in hook configuration

Installation

Run wtk init --all --global to reconfigure hooks with the fix.

---

Full Changelog: v0.2.4...v0.6.0

v0.2.4

Full Changelog: v0.2.3...v0.2.4

v0.2.3

Full Changelog: v0.2.2...v0.2.3

v0.2.2

Full Changelog: v0.2.1...v0.2.2

v0.2.1

Full Changelog: v0.2.0...v0.2.1

v0.2.0

Full Changelog: v0.1.0...v0.2.0