[Bug]: Errors after activating redis semantic caching

[BrDewit]

What happened?

I've deployed LiteLLM using docker compose with version litellm-database:main-stable (which at the time of writing this is v1.77.2-stable). Initially after trying to enable redis semantic caching, I got an error that redisvl is not installed, but extending the docker image with the installation of redisvl 0.4.1 fixed this one.

I have a model in my model_list which I called infinity-granite-embedding-multilingual and it's the ibm-granite/granite-embedding-107m-multilingual hosted using Infinity, using the infinity provider. When I run LiteLLM without the semantic cache, I can call this model through the proxy to embed text without any issues, but if I specify this model as "redis_semantic_cache_embedding_model" in my config.yaml, I get the error "LLM Provider NOT provided. Pass in the LLM provider you are trying to call". In the documentation for the redis semantic cache, they also just use the model name, but it doesn't work for me.

Below you can find my cache_params config and the stack trace of the error.

cache_params in config.yaml

 cache_params:
    type: "redis-semantic"
    host: os.environ/REDIS_HOST
    port: os.environ/REDIS_PORT
    password: os.environ/REDIS_PASSWORD
    namespace: "litellm_cache"
    ttl: 600
    similarity_threshold: 0.85
    redis_semantic_cache_embedding_model: infinity-granite-embedding-multilingual

The stack trace

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/redisvl/utils/vectorize/text/custom.py", line 142, in _validate_sync_callablesredis_semantic_cache_embedding_model
    test_single = self._embed("dimension test")
  File "/usr/lib/python3.13/site-packages/litellm/caching/redis_semantic_cache.py", line 148, in _get_embedding
    litellm.embedding(
    ~~~~~~~~~~~~~~~~~^
        model=self.embedding_model,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        input=prompt,
        ^^^^^^^^^^^^^
        cache={"no-store": True, "no-cache": True},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ),
    ^
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1345, in wrapper
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1220, in wrapper
    result = original_function(*args, **kwargs)
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 3860, in embedding
    model, custom_llm_provider, dynamic_api_key, api_base = get_llm_provider(
                                                            ~~~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<2 lines>...
        api_key=api_key,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/get_llm_provider_logic.py", line 407, in get_llm_provider
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/get_llm_provider_logic.py", line 384, in get_llm_provider
    raise litellm.exceptions.BadRequestError(  # type: ignore
    ...<8 lines>...
    )
litellm.exceptions.BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=infinity-granite-embedding-multilingual
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/starlette/routing.py", line 694, in lifespan
    async with self.lifespan_context(app) as maybe_state:
               ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^that 
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 612, in proxy_startup_event
    await initialize(**worker_config)
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 3325, in initialize
    ) = await proxy_config.load_config(router=llm_router, config_file_path=config)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 1828, in load_config
    self._init_cache(cache_params=cache_params)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 1608, in _init_cache
    litellm.cache = Cache(**cache_params)
                    ~~~~~^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/caching/caching.py", line 191, in __init__
    self.cache = RedisSemanticCache(
                 ~~~~~~~~~~~~~~~~~~^
        host=host,
        ^^^^^^^^^^
    ...<5 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/caching/redis_semantic_cache.py", line 110, in __init__
    cache_vectorizer = CustomTextVectorizer(self._get_embedding)
  File "/usr/lib/python3.13/site-packages/redisvl/utils/vectorize/text/custom.py", line 126, in __init__
    self.dims = self._validate_sync_callables()
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/site-packages/redisvl/utils/vectorize/text/custom.py", line 146, in _validate_sync_callables
    raise ValueError(f"Invalid embedding method: {e}")
ValueError: Invalid embedding method: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=infinity-granite-embedding-multilingual
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers

Application startup failed. Exiting.

I've tried to pass the provider and the model directly instead of using the model name, but then it said "api_base is required for Infinity embeddings", but there's no way to specify this for the semantic cache, and like mentioned before, this is not how it's done in the documentation. I've tried many things, like using the OpenAI provider, as Infinity follows the OpenAI API specification, but I get the exact same error as with Infinity, that I have to specify a provider.

Relevant log output

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/redisvl/utils/vectorize/text/custom.py", line 142, in _validate_sync_callablesredis_semantic_cache_embedding_model
    test_single = self._embed("dimension test")
  File "/usr/lib/python3.13/site-packages/litellm/caching/redis_semantic_cache.py", line 148, in _get_embedding
    litellm.embedding(
    ~~~~~~~~~~~~~~~~~^
        model=self.embedding_model,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        input=prompt,
        ^^^^^^^^^^^^^
        cache={"no-store": True, "no-cache": True},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ),
    ^
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1345, in wrapper
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1220, in wrapper
    result = original_function(*args, **kwargs)
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 3860, in embedding
    model, custom_llm_provider, dynamic_api_key, api_base = get_llm_provider(
                                                            ~~~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<2 lines>...
        api_key=api_key,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/get_llm_provider_logic.py", line 407, in get_llm_provider
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/get_llm_provider_logic.py", line 384, in get_llm_provider
    raise litellm.exceptions.BadRequestError(  # type: ignore
    ...<8 lines>...
    )
litellm.exceptions.BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=infinity-granite-embedding-multilingual
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/starlette/routing.py", line 694, in lifespan
    async with self.lifespan_context(app) as maybe_state:
               ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^that 
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/fastapi/routing.py", line 134, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/contextlib.py", line 214, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 612, in proxy_startup_event
    await initialize(**worker_config)
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 3325, in initialize
    ) = await proxy_config.load_config(router=llm_router, config_file_path=config)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 1828, in load_config
    self._init_cache(cache_params=cache_params)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 1608, in _init_cache
    litellm.cache = Cache(**cache_params)
                    ~~~~~^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/caching/caching.py", line 191, in __init__
    self.cache = RedisSemanticCache(
                 ~~~~~~~~~~~~~~~~~~^
        host=host,
        ^^^^^^^^^^
    ...<5 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/caching/redis_semantic_cache.py", line 110, in __init__
    cache_vectorizer = CustomTextVectorizer(self._get_embedding)
  File "/usr/lib/python3.13/site-packages/redisvl/utils/vectorize/text/custom.py", line 126, in __init__
    self.dims = self._validate_sync_callables()
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/site-packages/redisvl/utils/vectorize/text/custom.py", line 146, in _validate_sync_callables
    raise ValueError(f"Invalid embedding method: {e}")
ValueError: Invalid embedding method: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=infinity-granite-embedding-multilingual
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers

Application startup failed. Exiting.

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.77.2

Twitter / LinkedIn details

https://www.linkedin.com/in/bram-dewit-339000245/

[luizhsalazar]

I'm facing the error:

File "/usr/lib/python3.13/site-packages/litellm/caching/redis_semantic_cache.py", line 69, in init │
│ from redisvl.extensions.llmcache import SemanticCache │
│ ModuleNotFoundError: No module named 'redisvl'

in version main-v1.78.5-stable (docker image: ghcr.io/berriai/litellm-database:main-v1.78.5-stable)

Any update on that?

[BrDewit]

@luizhsalazar I can't remember exactly what we did, because we're currently using another setup but, as mentioned in the original post, I extended the existing docker image, installed redisvl, and then started litellm from the entrypoint.

Like I said, I'm not sure how exactly I installed redisvl at the time, maybe it was just pip install redisvl==0.4.1, not 100% sure, but if you're still struggling with it, you can try it out.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.