-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Summary
Add macdoc convert --to tokens <file> that outputs exact token counts.
Depends on #27 (CLI migration to convert --to pattern).
CLI Interface
macdoc convert --to tokens file.md # both models
macdoc convert --to tokens --model gpt-4o file.md # OpenAI only
macdoc convert --to tokens --model claude-sonnet file.md # Claude onlyOutput Format
# Default (tabular)
Model Tokens
─────────────────────────
gpt-4o 1,234
claude-sonnet-4-6 1,198
# Single model (just the number for piping)
1234
Token Counting Methods
| Provider | Method | Cost |
|---|---|---|
| OpenAI | swift-tiktoken (offline BPE) |
Free, offline |
| Claude | Anthropic /v1/messages/count_tokens API |
Free, online (RPM-limited) |
Architecture
- New package:
token-counter-swiftunderpackages/ - Layer 2 utility (measurement tool, not format converter)
- Depends on:
swift-tiktoken,Foundation(URLSession for Anthropic API) - Needs
ANTHROPIC_API_KEYenv var for Claude counting
References
- Anthropic token counting docs — free, RPM-limited
- Blocked by refactor: unify CLI to
macdoc convert --topattern (textutil-compatible) #27
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels