LiteLLM × Headroom: Use 60-95% fewer tokens with Prompt Compression

[krrish-berri-2]

LiteLLM × Headroom: Use 60-95% fewer tokens with Prompt Compression

Use https://github.com/BerriAI/litellm/releases/tag/v1.92.0-dev.1 for testing

Headroom now runs as a native guardrail on the LiteLLM proxy, compressing tool outputs, RAG payloads, database results, and file reads before they reach the model.

Why this matters

Long-context agents burn most of their input budget on repeated tool output, retrieved chunks, and stale scratch state. Headroom intelligently rewrites that content so the model sees the same information at a fraction of the tokens.

If the model needs the full context, LiteLLM will also pass a retrieve_headroom tool to the model, to retrieve the full context from Headroom.

How is it deployed?

Headroom runs as a sidecar to LiteLLM. Client traffic still hits the LiteLLM gateway; LiteLLM invokes Headroom during the pre_call step, swaps in the compressed messages, and forwards the payload upstream. Clients and the LLM provider never talk to Headroom directly.

The benefit of this is two-fold:

• Convenience: Users get 1 API base regardless of if they use prompt compression or not.
• Reliability: If Headroom goes down, your LLM calls are unaffected.

Works with both OpenAI & Anthropic formats

Compression works on both /v1/chat/completions and /v1/messages (Anthropic format), which makes the Claude Code rollout a one-liner for the admin: attach headroom-compression to a virtual key, hand it to the developer, and every request they make through ANTHROPIC_BASE_URL gets compressed automatically. No client-side change, no code diff.

How to enable it

Turn it on per key, per request, or globally via default_on: true. Confirm it ran by checking the x-litellm-applied-guardrails response header or the Guardrails panel in the Logs UI.

📖 Full blog post: https://docs.litellm.ai/blog/headroom-integration

📘 Setup guide: https://docs.litellm.ai/docs/proxy/guardrails/headroom

[dennys]

It looks like the Setup Guide link is incorrect. I believe it should be https://docs.litellm.ai/docs/proxy/headroom

[krrish-berri-2]

what version are you on? this is on the latest nightly

[tnndclub]

v1.90.2
after following the setup guide, I got the error below

ValueError: Unsupported guardrail: headroom

ERROR:    Application startup failed. Exiting.

which version should I use to setup headroom properly?

[krrish-berri-2]

what version are you on? this is on the latest nightly

[tnndclub]

@krrish-berri-2 on docker deployment v1.90.2
thanks for the clarification
waiting for v1.92.x release

[krrish-berri-2]

it's live on a dev release if you want it - https://github.com/BerriAI/litellm/releases/tag/v1.92.0-dev.1

[krrish-berri-2]

updated posts to clarify it's on v1.92.x versions

[eyevanovich]

Wow, what timing. I was just looking into adding Headroom in a different way. Will this work for all coding agents, not just Claude Code? I guess it's whatever is calling the v1/chat/completions and /v1/messages routes right? I use Pi but others use OpenCode and Claude Code so want to make sure it's still going to work for everyone.

[krrish-berri-2]

yea. i'll add /v1/responses support too for codex.

[qschweitzer]

Hey,

Thanks for adding Headroom in LiteLLM !

EDIT 2026/07/02 06:00PM:
I don't know how, because a lot of shitt* tests I mean, but the issue disappeared.
More simpler and faster usage is the docker compose way.

---

I just test the python package version 1.92.0-dev.1 =>

.claude/.venv/lib/python3.12/site-packages/litellm/proxy/guardrails/guardrail_registry.py\",
 line 494, in initialize_guardrail\n raise ValueError(f\"Unsupported guardrail: {guardrail_type}\")\nValueError: 
Unsupported guardrail: headroom\n", "level": 
"ERROR", "timestamp": "2026-07-02T08:44:27.332080

Same error with the Docker container 1.92.0-dev.1 :

init_guardrails_v2(\n    ~~~~~~~~~~~~~~~~~~^\n        all_guardrails=guardrails_v2,\n        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
        config_file_path=config_file_path,\n        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n        llm_router=router,\n
        ^^^^^^^^^^^^^^^^^^\n    )\n    ^\n  File \"/app/.venv/lib/python3.13/site-packages/litellm/proxy/guardrails/init_guardrails.py\",
 line 29, in init_guardrails_v2\n    initialized_guardrail = IN_MEMORY_GUARDRAIL_HANDLER.initialize_guardrail(\n
        guardrail=cast(Guardrail, guardrail),\n    ...<2 lines>...\n        source=\"config\",\n    )\n  File \"/app/.venv/lib/python3.13/site-
packages/litellm/proxy/guardrails/guardrail_registry.py\", line 494, in initialize_guardrail\n    raise ValueError(f\"Unsupported 
guardrail: {guardrail_type}\")\nValueError: Unsupported guardrail: headroom\n", "level": "ERROR", "timestamp": 
"2026-07-02T06:50:32.845508", "component": "uvicorn.error", "logger": "on.py:121"}
{"message": "Application startup failed. Exiting.", "level": "ERROR", "timestamp": "2026-07-02T06:50:32.845672", "component": 
"uvicorn.error", "logger": "on.py:59"}
{"message": "[LoggingWorker] atexit: No queue initialized", "level": "DEBUG", "timestamp": "2026-07-02T06:50:32.846095",
 "component": "LiteLLM", "logger": "logging_worker.py:453"}

# Config.yml
...

guardrails:
  - guardrail_name: headroom-compression
    litellm_params:
      guardrail: headroom
      mode: pre_call
      api_base: http://127.0.0.1:8787
      # api_key: os.environ/HEADROOM_API_KEY
      default_on: true

[krrish-berri-2]

so this is working?

[qschweitzer]

Yep on Docker Compose. With database.

I use it with Ollama, Mistral and Deepseek.
I don't see headroom Guardrails on every requests. Is it only visible on the first one ?
I'm not sure that headroom is really used even if I set 'default_on' to "true" and some of stats are visibles in headroom dashboard.

Thanks for your work 🙏🏼

[krrish-berri-2]

How do I repro this? *Krrish Dholakia | *CEO Book a meeting with me <https://calendly.com/d/cx9p-5yf-2nm/litellm-introductions> LinkedIn <https://www.linkedin.com/in/krish-d/> | (770) 878 - 3106 ***@***.*** P.S. See how LiteLLM helps LLM Platform teams move fast and stay in control <https://www.litellm.ai/#features>

…

On Thu, Jul 2, 2026 at 11:16 PM Quentin (b3ard0x) Schweitzer < ***@***.***> wrote: Yep. I use it with Ollama, Mistral and Deepseek. I don't see headroom Guardrails on every requests. Is it only visible on the first one ? I'm not sure that headroom is really used even if I set 'default_on' to "true" and some of stats are visibles in headroom dashboard. Thanks for your work 🙏🏼 — Reply to this email directly, view it on GitHub <#31816?email_source=notifications&email_token=CARFWGE32TJ5CNC5IPE7CPL5C5FVJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZVGE4TKOJUUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17519594>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/CARFWGEH442YAHHGFFSG3JD5C5FVJAVCNFSNUABIKJSXA33TNF2G64TZHM3DOMJSGY4TKMBVHNCGS43DOVZXG2LPNY5TCMBTGUYDSMBXUF3AE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[krrish-berri-2]

fyi improved logging so it's easy to see how headroom ran and what it did

would there interest in a cost saving dashboard? to see how much money you saved with headroom + other strategies (e.g. budget based fallbacks)

[D0wn10ad]

hi,

I am having problem with retrieving compressed ccr via retrieve_headroom (as mentioned in the blog and also this text). I have followed the setup guide and wondering did i miss any step(s)?

in opencode, the model responded as the tool is not exposed to it:

> call headroom_retrieve with 94b5c2fe9d61 
I can’t call headroom_retrieve in this session because that tool is not exposed to me.
What I do know:
- the placeholder ID is 94b5c2fe9d61
- the blocked page is wikilink/
- phabricator_document_info is still returning:
- <<ccr:94b5c2fe9d61,string,9.0KB>>
If you expose headroom_retrieve or retrieve_headroom to this session, I can use it immediately on 94b5c2fe9d61. Right now, the missing blocker is tool access, not the identifier.

I am using headroom with default:on.

Regards,
Titus