Check for existing issues
The Feature
LiteLLM currently loads the “local fallback” model cost map from a JSON bundled inside the Python package (via importlib.resources). In packaged/bundled deployments (e.g., PyInstaller, slim Docker images), that bundled JSON may be missing, which can break offline fallback behavior.
Proposal: support user-provided local sources for the model cost map:
LITELLM_MODEL_COST_MAP_PATH=/path/to/model_cost_map.json (load JSON from file)
Pseudocode
@staticmethod
def load_local_model_cost_map() -> dict:
path = os.getenv("LITELLM_MODEL_COST_MAP_PATH", "").strip()
if path:
with open(path, "r", encoding="utf-8") as f:
return json.load(f)
# existing fallback
return json.loads(
files("litellm")
.joinpath("model_prices_and_context_window_backup.json")
.read_text(encoding="utf-8")
)Motivation, pitch
This is mainly for offline/firewalled and packaged environments where:
- remote fetch (GitHub raw) is blocked/unavailable, and
- package data files (backup JSON) may be excluded by packaging/build tooling.
Related but not duplicates:
This change would make LiteLLM more reliable in air-gapped deployments and bundled binaries, while keeping current default behavior unchanged.
What part of LiteLLM is this about?
SDK (litellm Python package)
LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?
No
Twitter / LinkedIn details
No response
Check for existing issues
The Feature
LiteLLM currently loads the “local fallback” model cost map from a JSON bundled inside the Python package (via
importlib.resources). In packaged/bundled deployments (e.g., PyInstaller, slim Docker images), that bundled JSON may be missing, which can break offline fallback behavior.Proposal: support user-provided local sources for the model cost map:
LITELLM_MODEL_COST_MAP_PATH=/path/to/model_cost_map.json(load JSON from file)Pseudocode
Motivation, pitch
This is mainly for offline/firewalled and packaged environments where:
Related but not duplicates:
LITELLM_LOCAL_MODEL_COST_MAP=True) still assumes the bundled backup JSON exists.This change would make LiteLLM more reliable in air-gapped deployments and bundled binaries, while keeping current default behavior unchanged.
What part of LiteLLM is this about?
SDK (litellm Python package)
LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?
No
Twitter / LinkedIn details
No response