fix: support remote Ollama server for function calling and pydantic output (#4694)

[devin-ai-integration]

fix: support remote Ollama server for function calling and pydantic output

Fixes #4694

Summary

Two bugs prevented CrewAI from working with remote (non-localhost) Ollama servers:

Bug 1 — InternalInstructor.to_pydantic() loses remote URL: The instructor's completion call to litellm did not forward api_base/base_url/api_key from the LLM object, so litellm defaulted to localhost and failed to reach the remote Ollama server during structured output parsing. Fixed by adding _get_llm_extra_kwargs() which extracts these params and splats them into the create() call.

Bug 2 — LLM.supports_function_calling() always returns False for remote Ollama: litellm.utils.supports_function_calling() internally tries to query localhost:11434 to check model capabilities, which fails when Ollama runs on a different host. Fixed by adding a fallback that queries the remote Ollama /api/show endpoint directly, inspecting model_info keys and the template string for tool support indicators.

Files changed

• lib/crewai/src/crewai/utilities/internal_instructor.py — new _get_llm_extra_kwargs() helper + forwarding in to_pydantic()
• lib/crewai/src/crewai/llm.py — new _is_ollama_model(), _get_ollama_base_url(), _check_ollama_function_calling() helpers + fallback in supports_function_calling()
• lib/crewai/tests/test_remote_ollama.py — 31 new unit tests covering both bugs

Review & Testing Checklist for Human

• Verify tool-detection heuristics against real Ollama responses: _check_ollama_function_calling() checks for "tool" in key.lower() and value is True in model_info and "tools"/".ToolCalls" in the template string. Confirm these patterns match actual /api/show responses for tool-capable models (e.g., mistral-small3.2, qwen2.5) and don't false-positive on non-tool models.
• Synchronous HTTP call in hot path: _check_ollama_function_calling() makes a blocking httpx.post with 5s timeout. This only fires when litellm returns False for an Ollama model with a remote URL, but verify this doesn't introduce unacceptable latency in your workflows.
• Test with a real remote Ollama server: All tests use mocks. Spin up an Ollama server on a different host (or Docker container with exposed port) and verify:
  • LLM.supports_function_calling() returns True for tool-capable models like ollama_chat/mistral-small3.2:24b
  • InternalInstructor.to_pydantic() successfully parses structured output from a remote Ollama model
  • Non-tool models (e.g., ollama/llama2) correctly return False
• Edge cases: Test with trailing slashes in api_base (should be stripped), unreachable servers (should return False without crashing), and localhost URLs (should behave as before).

Notes

• Test file uses object.__new__() to bypass InternalInstructor.__init__ and avoid the dynamic instructor import; this doesn't test the real initialization path but allows testing the helper methods in isolation.
• The unused pytest import can be removed if desired (no parametrize or raises used).
• Session: https://app.devin.ai/sessions/1aecc957ee64482b984d1bb06283ac54
• Requested by: João

---

Note

Medium Risk
Adds a new synchronous HTTP capability probe to LLM.supports_function_calling() for remote Ollama and changes instructor completion kwargs forwarding; risk is moderate due to potential latency/false positives and altered request parameters.

Overview
Fixes remote Ollama usage by ensuring structured-output (InternalInstructor.to_pydantic) forwards provider connection kwargs (api_base, base_url, api_key) into the litellm create() call.

Adds an Ollama-specific fallback in LLM.supports_function_calling() that, when litellm reports no tool support and a remote URL is configured, queries the remote Ollama /api/show endpoint to infer tool/function-calling capability.

Introduces a new test_remote_ollama.py suite covering both behaviors (kwargs forwarding and remote capability probing) with mocked httpx/litellm interactions.

^{Written by Cursor Bugbot for commit 9bdc7b9. This will update automatically on new commits. Configure here.}

[devin-ai-integration]

Prompt hidden (unlisted session)

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[greysonlalonde]

Closing as stale — no activity in 30+ days.