Skip to content

security: pin LiteLLM to v1.82.3 (supply chain compromise)#289

Merged
OisinKyne merged 1 commit intomainfrom
fix/litellm-supply-chain-pin
Mar 24, 2026
Merged

security: pin LiteLLM to v1.82.3 (supply chain compromise)#289
OisinKyne merged 1 commit intomainfrom
fix/litellm-supply-chain-pin

Conversation

@bussyjd
Copy link
Collaborator

@bussyjd bussyjd commented Mar 24, 2026

Summary

What was compromised

The litellm_init.pth file in PyPI packages 1.82.7-1.82.8 auto-executes on Python startup and steals:

  • Environment variables (API keys, secrets)
  • SSH keys
  • Cloud credentials (AWS, GCP, Azure)
  • Kubernetes configs
  • Database passwords

Data is encrypted and exfiltrated to https://models.litellm.cloud/.

Changes

  • internal/embed/infrastructure/base/templates/llm.yaml — pin image to ghcr.io/berriai/litellm:main-v1.82.3
  • .agents/skills/obol-stack-dev/references/litellm-routing.md — update reference

Remediation for existing deployments

If running main-stable, verify your version:

obol kubectl -n llm exec deploy/litellm -c litellm -- pip show litellm | grep Version

If version is 1.82.7 or 1.82.8, rotate all credentials immediately and redeploy with the pinned tag.

Test plan

  • go build ./... passes
  • go test ./internal/embed/... passes
  • Verified main-v1.82.3 tag exists on GHCR
  • Confirmed running cluster uses 1.82.3 (pre-compromise)

LiteLLM PyPI packages 1.82.7 and 1.82.8 contain a malicious .pth file
(litellm_init.pth) that exfiltrates environment variables, SSH keys,
cloud credentials, and Kubernetes configs to an external endpoint.

See: BerriAI/litellm#24512

Our template used the floating tag `main-stable` which could pull a
compromised build. Pin to `main-v1.82.3` (confirmed safe, matches
the version currently running in our clusters).

Never use floating tags for security-sensitive dependencies.
@OisinKyne OisinKyne merged commit 066e7ea into main Mar 24, 2026
6 checks passed
@OisinKyne OisinKyne deleted the fix/litellm-supply-chain-pin branch March 24, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants