Skip to content

[Bug]: Vector Store Search Defaults to OpenAI in Air-Gapped Environment Despite Local embedding_model Configuration #15726

Description

@Badlybear

What happened?

In a completely air-gapped environment, the /vector_stores/VS_ID/search endpoint ignores the embedding_model setting in config.yaml when the specified model uses a custom provider (in my case, hosted_vllm).

Instead of using the configured local model, LiteLLM falls back to its default behavior and attempts to connect to api.openai.com to generate embeddings. This call is impossible in an air-gapped setup, causing the request to fail.

This suggests that the vector store logic may not correctly recognize or handle custom model providers, causing it to ignore the configuration and revert to a hardcoded default.

How to Reproduce

Set up LiteLLM Proxy in an air-gapped Kubernetes or OpenShift environment.

Configure a local embedding model served via a custom OpenAI-compatible endpoint (e.g., vLLM). In the config.yaml, define this model using the hosted_vllm prefix.

Configure a vector store (e.g., pg_vector) and link it to the local model using the embedding_model parameter, referencing the model_name from step 2.

Send a POST request to the /vector_stores/VS_ID/search endpoint.

Expected Behavior

LiteLLM should honor the embedding_model: "local-embedding" configuration.

It should make a network request to the internal api_base (https://qwen3-embedding-8b...) to generate embeddings for the search query.

It should not make any external network calls to api.openai.com.

Actual Behavior

LiteLLM ignores the configuration for the hosted_vllm model.

It attempts to connect externally to api.openai.com:443.

The request fails with the error: litellm.APIConnectionError: openai - Cannot connect to host api.openai.com:443 ssl:default [Name or service not known].

Environment

Deployment: Air-gapped OpenShift Cluster

LiteLLM Version: Latest Docker Image (as of Oct 2025)

Vector Store: pg_vector -- litellm_pgvector

Embedding Model Provider: Custom hosted_vllm endpoint (OpenAI-compatible vLLM service)

Debugging Steps Taken

Confirmed the LiteLLM pod had restarted and loaded the latest ConfigMap multiple times.

Verified that the LiteLLM pod has direct network connectivity to the embedding model's internal service URL (https://qwen3-embedding-8b...). A python urllib script run from within the LiteLLM pod successfully connects to the endpoint, proving it's not a network or SSL issue.

Setting the OPENAI_API_VERIFY_SSL_CERTS=False environment variable had no effect, which is expected since the call is being incorrectly routed to the public OpenAI URL, not the internal one.

Potential Workaround Found: If I "disguise" the model as an openai model (e.g., model: openai/my-model) while keeping the local api_base, LiteLLM appears to respect the configuration. This strongly suggests the issue is tied to how LiteLLM's vector store feature handles non-standard provider names like hosted_vllm.

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

main-v1.77.5-stable

Twitter / LinkedIn details

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions