Skip to content

feat: add native Gemini passthrough to LiteLLM sidecar - #29

Merged
VickyXAI merged 2 commits into
mainfrom
fix/gemini-native-proxy
Jul 22, 2026
Merged

feat: add native Gemini passthrough to LiteLLM sidecar#29
VickyXAI merged 2 commits into
mainfrom
fix/gemini-native-proxy

Conversation

@KillerQueen-Z

Copy link
Copy Markdown
Collaborator

What changed

  • add native Gemini generateContent and streamGenerateContent routes to the BlockRun LiteLLM sidecar
  • preserve Gemini request/response JSON, native error envelopes, and SSE frames while the sidecar handles Base or Solana x402 payment
  • derive streaming from the Gemini URL method instead of an OpenAI-style JSON stream field
  • strip client Google credentials (x-goog-api-key, Authorization, and query parameters) before forwarding
  • record the native Gemini model and /v1beta/models/... path in local usage telemetry
  • document raw REST and official google-genai client setup
  • bump the package version to 0.8.0

Why

The Solana gateway already supports Gemini's native protocol, but blockrun-litellm only exposed OpenAI-shaped chat and native Anthropic HTTP routes. Customers using the sidecar therefore could not point a Gemini-native client at it without translating the request themselves.

Customer impact

After upgrading blockrun-litellm, customers can point the Google Gen AI SDK custom base URL at the sidecar root and keep their existing native Gemini request/response shapes. The in-process litellm.completion(...) provider remains OpenAI-compatible and is unchanged.

Validation

  • full suite: 299 passed, 2 skipped
  • raw local sidecar -> Solana production E2E:
    • non-streaming native JSON returned HTTP 200 and native-ok
    • streaming native SSE returned HTTP 200 and stream-ok
  • official google-genai 2.13.0 -> local sidecar -> Solana production E2E:
    • non-streaming returned sdk-native-ok
    • streaming returned sdk-stream-ok
  • verified the local Solana wallet signed and settled all paid requests
  • git diff --check and Python bytecode compilation pass

@KillerQueen-Z
KillerQueen-Z marked this pull request as ready for review July 22, 2026 13:27
- sanitize the model id (strip models//google/ prefixes, reject slashes) and
  rebuild the forwarded path from clean parts, so no crafted model name can move
  the signed upstream target outside /v1beta/models/ (defense-in-depth; gateway
  already backstops, but the wallet shouldn't rely on it)
- log every local 400 reject, honoring the 0.7.6 every-exit-logs invariant
- document the Base enterprise/allowlist gate (403 for retail) in README + CHANGELOG,
  point retail callers to Solana or /v1/chat/completions
- de-duplicate _gemini_fwd_headers, refresh _forward_passthrough docstring for the
  new kwargs, simplify the stream_override ternary
- tests: isolate BLOCKRUN_API_URL, add slash-rejection, blank-model, dot-segment,
  and proxy-token denial coverage; assert the reject logs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants