How much of each model's context window does your text actually fill?
$ tokenbudget article.md
article.md — 47,428 characters
MODEL VENDOR TOKENS CONTEXT USED
Apple on-device Apple 12,641 4,096 308.81% — does not fit
GPT-4o OpenAI 10,866 128,000 8.49%
GPT-3.5 Turbo OpenAI 10,812 16,385 65.99%
Claude Opus 5 Anthropic — 1,000,000 no offline tokenizer — needs POST /v1/messages/count_tokens
Claude Sonnet 5 Anthropic — 1,000,000 no offline tokenizer — needs POST /v1/messages/count_tokens
Claude Haiku 4.5 Anthropic — 200,000 no offline tokenizer — needs POST /v1/messages/count_tokens
Apple on-device: TOKENS is your text alone. 8 more are charged for prompt scaffolding — an empty string costs 8 — and USED includes them.
context windows verified 2026-08-02
That is real output on a real article, and the first row is the reason the tool exists. The same 47k characters that fill 8% of GPT-4o's window are three times too big for the model already on the machine — and Apple's tokenizer needs 12,641 tokens where GPT-4o needs 10,866, about 16% more for identical text.
There is no single answer to "how many tokens is this?" There are three, and they differ by vendor:
| Vendor | Exact count | How |
|---|---|---|
| OpenAI | offline, free | tiktoken rank tables, BPE implemented here in Swift |
| Apple | offline, free | SystemLanguageModel.tokenCount(for:), needs Apple Intelligence on |
| Anthropic | network call | POST /v1/messages/count_tokens — no public offline tokenizer |
Most token counters paper over that by running everything through tiktoken.
For Claude that undercounts by roughly 15–20% on ordinary prose and considerably
more on code, so this tool prints nothing rather than a number it can't stand
behind.
The gap between tokenizers is not a rounding error. नमस्ते दुनिया is 3 tokens
on Apple's on-device model, 5 under o200k_base and 13 under
cl100k_base — one of those pairs is the same vendor, two encodings, 2.6× apart.
Note that the ranking inverts on English: the tokenizer that wins on Devanagari
is the one that costs 16% more on an English article.
swift build -c release
cp .build/release/tokenbudget /usr/local/bin/Requires macOS 14+. The Apple row additionally needs macOS 26.4+ with Apple Intelligence enabled; without it the row reports why rather than disappearing.
tokenbudget path/to/file.md
cat file.md | tokenbudgetThe BPE implementation is checked against Python tiktoken rather than against
intuition. Scripts/ground-truth.py prints reference counts for a set of awkward
cases — whitespace runs, repeated newlines, emoji, Devanagari, source code — and
those exact numbers are pinned in Tests/. Regenerate them with:
python3 -m venv .venv && .venv/bin/pip install tiktoken
.venv/bin/python Scripts/ground-truth.pyThe rank tables (o200k_base, cl100k_base) are OpenAI's published files,
bundled so the tool works offline. Loading them fails loudly if any of the 256
single-byte tokens is missing, since byte-level BPE silently undercounts without
full byte coverage.
Context windows are facts with an expiry date. ModelCatalog.verifiedOn records
when they were last checked against vendor documentation; treat a stale date as a
reason to re-check rather than as a guarantee.
Two traps worth knowing, both about Apple's numbers.
contextSize has a decoy value. It is @backDeployed(before: macOS 26.4)
with a hardcoded 4096 fallback body, so on an older OS it returns that literal
instead of asking the model. The real window on 26.4+ turns out to be 4096 as
well — so the placeholder and the measured value are indistinguishable by
inspection, and reading 4096 tells you nothing about which one you got. This
tool reports no window at all unless the model is genuinely available.
tokenCount(for:) does not count your string. It counts what your string
costs as a prompt: an empty string returns 8 tokens on this machine, and every
count carries that same fixed scaffolding. Measured, not assumed — the tool calls
it with "" at runtime and subtracts, rather than hardcoding 8, since it is
Apple's implementation detail to change. TOKENS is therefore the text alone and
comparable across vendors, while USED includes the overhead, because the window
is spent either way.