Foveance 0.5.1 — the lossless token codec (positioning)

Documentation and packaging metadata only; no code behaviour change. Foveance is repositioned around
its headline feature: it is now described everywhere as the lossless token-optimization codec
(up to 82% fewer input tokens, nothing dropped), with the anticipatory allocator presented as the
optional lossy-but-recoverable layer on top. README rebuilt to lead with compress() and the
codec benchmark (scorecard + 8-framework head-to-head), PyPI/npm summaries updated, and the social
card / share square / Twitter header re-cut to "Up to 82% fewer tokens. Losslessly."

Foveance 0.5.0 — the codec release

The first release since 0.2.0, and it ships everything from the 0.3 and 0.4 lines as well. Those two
versions were developed and are documented in full below, but were never published: 0.3.0 predates
its own vault handle-leak fix, and both predate the codec work that supersedes them. Rather than
put superseded builds on PyPI permanently, 0.5.0 folds them in. If you are coming from 0.2.0 you get
the 0.3 interactive-compression work, the 0.4 lossless codec, and the 0.5 codec-everywhere work in
one step.

Added

• Shared-prefix template pass (RedundancyCodec(template=True), compress(..., template=True)):
  factors the prefix shared by a run of near-identical lines out once
  ([fov:tpl 20 "src/service/module_"] + the suffixes), recovering the intra-line redundancy that
  line-level dedup structurally cannot see. Exactly invertible via the new
  foveance.expand_templates(), never inflates, and takes the head-to-head from 75.4% to 82.4%
  saved — past LLMLingua-2 matched (79.8%) while remaining lossless with 100% of facts preserved.
  Opt-in by design: it measured 0.90 vs 0.95 mean accuracy at 14% fewer tokens on the five-model
  benchmark (one task in twenty, within noise, but not measurably free), and the default must stay
  safe to enable unconditionally.
• RULER public benchmark arm (bench/fetch_ruler.py, bench/codec_ruler.py): 480 real examples
  across all task families at 4k/8k/16k. Serves as a negative control — RULER plants distinct
  needles, so the codec correctly saves ~0% on qa/niah_multikey and never inflates, while
  removing 91.4% on the one family that genuinely repeats. All 480 round-trip exactly.
• .foveance.toml now also carries the codec and agentic_codec switches, so the codec can be
  turned on once per machine/project instead of per invocation.

Fixed

• foveance --version / -V now print the version. Previously only the foveance version
  subcommand worked and the flags silently printed help and exited 0, which reads like a broken
  install. Regression-tested.
• The test suite no longer hard-fails without the optional ML extras. Four tests imported numpy
  (via foveance.learned) without a guard, so pip install "foveance[dev]" && pytest reported
  four ModuleNotFoundError failures on an otherwise healthy install. They now
  pytest.importorskip("numpy") and skip cleanly, matching how the integration tests already
  handle optional dependencies.

Changed (packaging)

• The source distribution no longer ships assets/ (README images are served from GitHub by
  absolute URL) or the manuscript's vector PDFs, halving it from 4.4 MB to 2.2 MB. The benchmark
  figures (bench/plots/*.png) and every results CSV are still included — they are the evidence
  behind the README's numbers. The installed wheel is unchanged at ~82 KB.
• Lossless codec on the agentic (tool-use) paths (foveance proxy --agentic-codec /
  FOVEANCE_AGENTIC_CODEC=1): on tool-using requests the codec now runs across the old tool
  payloads in place, collapsing cross-message repeats (re-listed dirs, retried stack traces,
  boilerplate) to legible pointers with the first occurrence kept verbatim — instead of lossily
  digesting each payload. Only free-text payload strings are rewritten; message count/order, roles,
  tool_use↔tool_result ids, cache_control blocks, and the last agentic_protect_last turns
  stay byte-identical, so the provider still validates the request and the prompt cache is never
  invalidated. Wired through all three dialects (Anthropic Messages, OpenAI Chat, OpenAI Responses)
  and covered by a full tool-pairing safety matrix. agentic_codec surfaced in /admin/stats.
• New agent adapters: foveance wrap/env now know Cursor, Windsurf, Roo Code, Zed, and the
  Gemini CLI (15 total), each with an honest note on GUI-vs-env configuration.
• Lossless codec in the proxy (foveance proxy --codec / FOVEANCE_CODEC=1): the 0.4
  cross-item redundancy codec now runs on the assembled plain-chat context, reference-encoding
  repeated line-runs across items losslessly (the first occurrence stays verbatim, so no fact
  is dropped). Off by default; a codec_saved_tokens counter reports the additional lossless
  saving. Zero accuracy risk, so it is safe to layer on top of any budget/policy.

Changed

• Codec default min_run=1 (was 2): because every reference is already gated by the token
  guard, dereferencing a single repeated line only when the pointer is strictly cheaper is
  cost-optimal — measured 76.1% vs 75.6% saved on the redundancy suite, never inflates, losslessness
  unchanged. Applied to compress/compress_anthropic/proxy.
• Vault storage codec now picks the strongest installed backend — Brotli (~50×) or Zstandard
  (~44×) when available, falling back to stdlib zlib — via a self-describing 3-byte header.
  Backward-compatible: legacy headerless-zlib blobs still decode.
• foveance.compress_anthropic(system, messages) — the Anthropic-shaped lossless codec
  one-liner, returning (new_system, new_messages, report) with the system string participating
  in the cross-message dedup.
• Entropy-coded vault storage (ItemVault(compress=True), default on): stored full texts are
  now zlib-compressed (the transport-codec storage saving — up to ~50–100× on redundant content),
  realising the paper's storage-side result. Reads transparently handle both compressed and legacy
  plaintext rows, so existing vaults keep working (a blob column is migrated in automatically).

v0.2.0 — Foveance Pro

Foveance 0.2.0 adds Foveance Pro (optional, offline-verified): a Pro license unlocks persistent savings accounting — token/$ totals survive restarts, the dashboard shows all-time and per-day history, and /admin/export.csv exports it. The open-source package remains fully functional without a license.

Keys are RSA-2048 signatures over a small JSON payload, verified in pure stdlib against a bundled public modulus — no network call, no phoning home, no new dependencies.

Also: foveance.integrations.llamaindex, and --token-encoding for exact-token accounting on newer tokenizers.

pip install --upgrade foveance

Foveance v0.1.3

Bundles the features merged from the maintenance PRs.

New

• shrink_anthropic(system, messages, budget) — Anthropic-shaped one-liner (#9)
• Config file: ~/.foveance.toml / ./.foveance.toml defaults (#2)
• Exact token counting via --exact-tokens (tiktoken) (#7)
• LiteLLM + LangChain integrations behind extras (#4, #5)
• Docker image + GHCR publish workflow — docker run ghcr.io/aimaghsoodi/foveance (#8, thanks @Hayathorium)

Fixed

• Clear ImportError from LogisticFutureRelevance.fit() when numpy is missing (#14)
• Cross-platform config-file test (Windows home dir)

pip install -U foveance · Apache-2.0

Foveance v0.1.2

Cut your LLM token bill 60%+ without changing your code or your answers.

Foveance keeps the parts of an agent's chat history that still matter and trims the rest, so you get the same answers for a fraction of the tokens. Nothing is deleted forever, and you don't change a line of your app.

Install & use

pip install foveance
foveance wrap claude          # or: foveance wrap -- codex "fix the tests"

Or from Python, one function:

from foveance import shrink
smaller = shrink(messages, budget=2000)

Highlights in 0.1.x

• One-command foveance wrap for any agent (Claude Code, Codex, aider, …) + live tokens-saved dashboard.
• shrink() one-liner — compress an OpenAI-style messages list from Python, no proxy, no setup.
• Drop-in proxy speaking OpenAI Chat, OpenAI Responses, and Anthropic Messages; prompt-cache aware.
• Real-model benchmark: matches full-replay accuracy at ~1/3 the tokens on Gemma/Llama/Qwen; every number traces to a CSV.

Links

• PyPI: https://pypi.org/project/foveance/
• npm: https://www.npmjs.com/package/foveance-proxy
• Docs: https://aimaghsoodi.github.io/foveance/
• Benchmark data: https://huggingface.co/datasets/AbteeXAILabs/foveance-benchmark

Apache-2.0.