[Bug]: Ollama API openai.APIConnectionError: Connection error. #1893
Labels
bug
Something isn't working
waiting for input
Need more information from the author to proceed further.
Is there an existing issue for the same bug?
Describe the bug
Request sent to Ollama though openai compatible API loads the model in ollama and then errors out in opendevin:
==============
STEP 0
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.
Provider List: https://docs.litellm.ai/docs/providers
20:55:15 - opendevin:ERROR: agent_controller.py:109 - Error while running the agent: OpenAIException - Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
yield
File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 113, in iter
for part in self._httpcore_stream:
File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 367, in iter
raise exc from None
File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 363, in iter
for part in self._stream:
File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 349, in iter
raise exc
File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 341, in iter
for chunk in self._connection._receive_response_body(**kwargs):
File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 210, in _receive_response_body
event = self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/http11.py", line 220, in _receive_event
with map_exceptions({h11.RemoteProtocolError: RemoteProtocolError}):
File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit
self.gen.throw(value)
File "/app/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.RemoteProtocolError: peer closed connection without sending complete message body (received 0 bytes, expected 407)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 952, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 928, in send
raise exc
File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 922, in send
response.read()
File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 813, in read
self._content = b"".join(self.iter_bytes())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 829, in iter_bytes
for raw_bytes in self.iter_raw():
File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 883, in iter_raw
for raw_stream_bytes in self.stream:
File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 126, in iter
for chunk in self._stream:
File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 112, in iter
with map_httpcore_exceptions():
File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit
self.gen.throw(value)
File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.RemoteProtocolError: peer closed connection without sending complete message body (received 0 bytes, expected 407)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai.py", line 533, in completion
raise e
File "/app/.venv/lib/python3.12/site-packages/litellm/llms/openai.py", line 492, in completion
response = openai_client.chat.completions.create(**data, timeout=timeout) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_utils/_utils.py", line 277, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/resources/chat/completions.py", line 590, in create
return self._post(
^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1240, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 921, in request
return self._request(
^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 976, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1053, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 976, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1053, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 986, in _request
raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
20:55:15 - opendevin:INFO: agent_controller.py:150 - Setting agent(CodeActAgent) state from AgentState.RUNNING to AgentState.ERROR
Current Version
Installation and Configuration
Model and Agent
Does it with all models and agents I have tried.
Reproduction Steps
No response
Logs, Errors, Screenshots, and Additional Context
No response
The text was updated successfully, but these errors were encountered: