Connect AI Provider to local Ollama

[cgspeck]

Documentation link

https://docs.tandoor.dev/features/ai/

Affected section

Does not exist yet

Other

No response

Issue description

While it is great that there is now AI import, there does not appear to be any clear / step by step instructions about how to integrate this new function with self hosted ollama server, which would be a very common setup for self-hosters / home lab.

[vabene1111]

go to Database> ai provider and create a new provider. URL, key and model name.

[cgspeck]

Does AI Import actually work for self-hosted Tandoor with self-hosted models/inference?

If it does then the instructions are unclear / incomplete and this should be reopened.

1. If you fill all the fields in, and then try it, an error pops up with reference to "LiteLLM"
2. If you jump through the hoops to set up LiteLLM, then point Tandoor at that, you get a generic "Error while fetching" message when attempting to AI import a PDF

Upon further inspection, it looks like Tandoor is attempting to call a LiteLLM route that does not exist:

recipes-1  |
recipes-1  | Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
recipes-1  | LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
recipes-1  |
recipes-1  | ThreadPoolExecutor-0_1 ERROR 2025-10-06 21:03:43,234 django.request Internal Server Error: /api/ai-import/
recipes-1  | Traceback (most recent call last):
recipes-1  |   File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 119, in _make_common_sync_call
recipes-1  |     response = sync_httpx_client.post(
recipes-1  |         url=api_base,
recipes-1  |     ...<4 lines>...
recipes-1  |         logging_obj=logging_obj,
recipes-1  |     )
recipes-1  |   File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/http_handler.py", line 578, in post
recipes-1  |     raise e
recipes-1  |   File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/http_handler.py", line 559, in post
recipes-1  |     response.raise_for_status()
recipes-1  |     ~~~~~~~~~~~~~~~~~~~~~~~~~^^
recipes-1  |   File "/opt/recipes/venv/lib/python3.13/site-packages/httpx/_models.py", line 829, in raise_for_status
recipes-1  |     raise HTTPStatusError(message, request=request, response=self)
recipes-1  | httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://litellm:4000/api/generate'

If it works with Ollama then there are apparently more requirements and steps to get this working in a self-hosted environment. Can you share what is missing?

[yacob841]

Ever figure it out? I’m gettin a different error but trying to accomplish the same thing.

[nprime496]

Hi, trying to achieve the same thing but the interface is not even the same, I cannot input the model that I want :

[rjanderson9]

Unsure why this issue was closed, issue persists.

[achurak]

Fwiw I was able to add a local ollama instance as a provider and I even see the requests/prompts coming in from the tandoor instance however the replies don't seem to be processed correctly for some reason and tandoor throws KeyError exception when trying to access some dict["name"] value. I can share the full trace if it's of any interest.

[yacob841]

Fwiw I was able to add a local ollama instance as a provider and I even see the requests/prompts coming in from the tandoor instance however the replies don't seem to be processed correctly for some reason and tandoor throws KeyError exception when trying to access some dict["name"] value. I can share the full trace if it's of any interest.

Can you give steps taken to add the local Ollama? I know it doesn’t work but you’re still a lot further than me

[rjanderson9]

I was also able to get them talking, and it even sometimes works. I had to increase the GUNICORN_TIMEOUT and use some interesting settings for ollama. I used sk-1234567890abcdef for the api key, openai/llama3.2:1b as the model, and http://ipAddress:11434/v1 as the url. It's not perfect, I'm guessing due to my ollama server being underpowered, it's an NVIDIA P2200, so it's only so powerful...

[achurak]

Yeah, it's quite literally just this:

Other models also work fine, but give the same error when tandoor is trying to process the response.

[achurak]

And here's the exception it throws every time for me:

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
ThreadPoolExecutor-0_1 ERROR 2025-12-27 13:23:26,065 django.request Internal Server Error: /api/ai-import/
Traceback (most recent call last):
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/main.py", line 2836, in completion
    response = base_llm_http_handler.completion(
        model=model,
    ...<13 lines>...
        client=client,
    )
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 388, in completion
    return provider_config.transform_response(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<9 lines>...
        json_mode=json_mode,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/llms/ollama/completion/transformation.py", line 266, in transform_response
    "name": function_call["name"],
            ~~~~~~~~~~~~~^^^^^^^^
KeyError: 'name'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/recipes/venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/recipes/venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/recipes/venv/lib/python3.13/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
  File "/opt/recipes/venv/lib/python3.13/site-packages/django/views/generic/base.py", line 105, in view
    return self.dispatch(request, *args, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/recipes/venv/lib/python3.13/site-packages/rest_framework/views.py", line 515, in dispatch
    response = self.handle_exception(exc)
  File "/opt/recipes/venv/lib/python3.13/site-packages/rest_framework/views.py", line 475, in handle_exception
    self.raise_uncaught_exception(exc)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/opt/recipes/venv/lib/python3.13/site-packages/rest_framework/views.py", line 486, in raise_uncaught_exception
    raise exc
  File "/opt/recipes/venv/lib/python3.13/site-packages/rest_framework/views.py", line 512, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/recipes/cookbook/views/api.py", line 2535, in post
    ai_response = completion(**ai_request)
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/utils.py", line 1235, in wrapper
    raise e
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/utils.py", line 1113, in wrapper
    result = original_function(*args, **kwargs)
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/main.py", line 3148, in completion
    raise exception_type(
          ~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<3 lines>...
        extra_kwargs=kwargs,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2214, in exception_type
    raise e
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2190, in exception_type
    raise APIConnectionError(
    ...<8 lines>...
    )
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: 'name'
Traceback (most recent call last):
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/main.py", line 2836, in completion
    response = base_llm_http_handler.completion(
        model=model,
    ...<13 lines>...
        client=client,
    )
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 388, in completion
    return provider_config.transform_response(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<9 lines>...
        json_mode=json_mode,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/recipes/venv/lib/python3.13/site-packages/litellm/llms/ollama/completion/transformation.py", line 266, in transform_response
    "name": function_call["name"],
            ~~~~~~~~~~~~~^^^^^^^^
KeyError: 'name'

[yacob841]

I get error 404 showing up in Ollama logs, Kitshn gives me a 500 error when trying a social media import, tandoor gives an “Error while fetching” doing an AI text recipe

[Gorillix]

I am experiencing the same issue and cannot get past the "invalid image input" error.

I have attempted this with a self-hosted Tandoor instance (LXC) pointing to a separate local Ollama instance (LAN). While basic connectivity works, Image Import fails consistently across multiple configurations.

Environment:

• Backend: Tandoor (Docker)
• AI Host: Ollama (Latest, running on RTX 4090)
• Models Pulled: llama3.2-vision, llava, llama3.2

Configuration Tested (OpenAI Compatible):
I configured the provider to force the generic OpenAI driver to bypass the KeyError: 'name' issue seen with the native driver.

• Provider: OpenAI
• Base URL: http://[LAN_IP]:11434/v1
• API Key: (Blank/Arbitrary)
• Model Names Tested: * openai/llama3.2
  • openai/llama3.2-vision
  • openai/llava

Result:
The connection is established, but the request fails immediately upon image upload with the following error:

The AI could not process your request.
litellm.BadRequestError: OpenAIException - invalid image input

Note:

• Text-only requests (using the "Text" import tab) appear to work with this config, suggesting the issue is specifically how the image payload is formatted/encoded when sent to Ollama's /v1 compatibility endpoint.
• I also attempted the native ollama/ driver configuration (without /v1), but that results in the KeyError: 'name' server error previously mentioned in this thread.

It seems we are blocked on both fronts: the native driver expects a function-call response structure that local vision models aren't providing, and the OpenAI driver path is sending an image format that Ollama rejects.

[aestheticjmack]

This needs to be reopened, OLLAMA does not work