Run with litellm #10
Comments
|
@superuben From the error messages you posted, it looks like an empty api key was configured. Could you please check if you have successfully configured your OpenAI api key? |
|
look all the way at the top of my message "llm.api_key": "" that config is my own input. i run litellm at port 8000 with an openai compatible endpoint so no key is needed. curling it works fine |
|
from another terminal: python -m taskweaver -p ./project/ |
|
We haven't tested under litellm, so not sure if it would work. Previously, we actually ignored the api_base configuration if api_type is |
|
Fixed api_base issue for openai api_type. Please have a try. |
ShilinHe
added
the
solution provided
|
TaskWeaver: I am TaskWeaver, an AI assistant. To get started, could you please enter your request? =========== litellm debug: Debug this by setting Custom Logger - final response object: None |
The issue appears to be associated with the incompatibility of litellm and OpenAI's 'response_format' argument. The latest OpenAI API supports both 'response_object' and 'text' modes, but it is unclear whether it can adapt to the litellm API. If you are experiencing a similar problem, feel free to submit a new issue for tracking purposes. |
|
LiteLLM is now supported in TaskWeaver, please follow the docs for more information. |
/home/user/TaskWeaver/project/taskweaver_config.json
{
"llm.api_base": "http://0.0.0.0:8000/v1",
"llm.api_key": "",
"llm.model": "gpt-3.5-turbo"
}
TaskWeaver$ python -m taskweaver -p ./project/
|_ | _ | | _ | | / / ____ __ _____ _____
| |/
/ __| |/ /| | /| / / _ \/ __/ | / / _ / /| | (| _ \ < | |/ |/ / __/ // /| |/ / __/ /
||_,|/|_|/|/_/_,/ |/_/_/
TaskWeaver: I am TaskWeaver, an AI assistant. To get started, could you please enter your request?
Human: heya
Error: Cannot process your request due to Exception: OpenAI API request failed to connect: Connection error.
Traceback (most recent call last):
File "/home/user/TaskWeaver/taskweaver/llm/init.py", line 146, in wrapper
raise Exception("cassette_mode or cassette_path is not set")
Exception: cassette_mode or cassette_path is not set
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
yield
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 142, in _send_request_headers
event = h11.Request(
^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/h11/_events.py", line 96, in init
self, "headers", normalize_and_validate(headers, _parsed=_parsed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/h11/_headers.py", line 164, in normalize_and_validate
validate(_field_value_re, value, "Illegal header value {!r}", value)
File "/home/user/.local/lib/python3.11/site-packages/h11/_util.py", line 91, in validate
raise LocalProtocolError(msg)
h11._util.LocalProtocolError: Illegal header value b'Bearer '
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
yield
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 228, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 268, in handle_request
raise exc
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 251, in handle_request
response = connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 103, in handle_request
return self._connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 133, in handle_request
raise exc
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 92, in handle_request
self._send_request_headers(**kwargs)
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 141, in _send_request_headers
with map_exceptions({h11.LocalProtocolError: LocalProtocolError}):
File "/home/user/miniconda3/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.LocalProtocolError: Illegal header value b'Bearer '
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 882, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 901, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 929, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 966, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 1002, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 227, in handle_request
with map_httpcore_exceptions():
File "/home/user/miniconda3/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.LocalProtocolError: Illegal header value b'Bearer '
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
yield
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 142, in _send_request_headers
event = h11.Request(
^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/h11/_events.py", line 96, in init
self, "headers", normalize_and_validate(headers, _parsed=_parsed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/h11/_headers.py", line 164, in normalize_and_validate
validate(_field_value_re, value, "Illegal header value {!r}", value)
File "/home/user/.local/lib/python3.11/site-packages/h11/_util.py", line 91, in validate
raise LocalProtocolError(msg)
h11._util.LocalProtocolError: Illegal header value b'Bearer '
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
yield
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 228, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 268, in handle_request
raise exc
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 251, in handle_request
response = connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 103, in handle_request
return self._connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 133, in handle_request
raise exc
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 92, in handle_request
self._send_request_headers(**kwargs)
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 141, in _send_request_headers
with map_exceptions({h11.LocalProtocolError: LocalProtocolError}):
File "/home/user/miniconda3/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.LocalProtocolError: Illegal header value b'Bearer '
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 882, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 901, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 929, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 966, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 1002, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 227, in handle_request
with map_httpcore_exceptions():
File "/home/user/miniconda3/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.LocalProtocolError: Illegal header value b'Bearer '
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
yield
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 142, in _send_request_headers
event = h11.Request(
^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/h11/_events.py", line 96, in init
self, "headers", normalize_and_validate(headers, _parsed=_parsed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/h11/_headers.py", line 164, in normalize_and_validate
validate(_field_value_re, value, "Illegal header value {!r}", value)
File "/home/user/.local/lib/python3.11/site-packages/h11/_util.py", line 91, in validate
raise LocalProtocolError(msg)
h11._util.LocalProtocolError: Illegal header value b'Bearer '
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
yield
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 228, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 268, in handle_request
raise exc
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 251, in handle_request
response = connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 103, in handle_request
return self._connection.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 133, in handle_request
raise exc
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 92, in handle_request
self._send_request_headers(**kwargs)
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 141, in _send_request_headers
with map_exceptions({h11.LocalProtocolError: LocalProtocolError}):
File "/home/user/miniconda3/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/home/user/.local/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.LocalProtocolError: Illegal header value b'Bearer '
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 882, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 901, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 929, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 966, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_client.py", line 1002, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 227, in handle_request
with map_httpcore_exceptions():
File "/home/user/miniconda3/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/home/user/.local/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.LocalProtocolError: Illegal header value b'Bearer '
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/TaskWeaver/taskweaver/llm/init.py", line 423, in chat_completion
res: Any = client.chat.completions.create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_utils/_utils.py", line 301, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 598, in create
return self._post(
^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 1096, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 856, in request
return self._request(
^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 929, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 966, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 929, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 966, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 938, in _request
raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/TaskWeaver/taskweaver/session/session.py", line 124, in send_message
post = _send_message(post.send_to, post)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/TaskWeaver/taskweaver/session/session.py", line 96, in _send_message
reply_post = self.planner.reply(
^^^^^^^^^^^^^^^^^^^
File "/home/user/TaskWeaver/taskweaver/planner/planner.py", line 179, in reply
llm_output = self.llm_api.chat_completion(chat_history, use_backup_engine=use_back_up_engine)["content"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/TaskWeaver/taskweaver/llm/init.py", line 160, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/user/TaskWeaver/taskweaver/llm/init.py", line 453, in chat_completion
raise Exception(f"OpenAI API request failed to connect: {e}")
Exception: OpenAI API request failed to connect: Connection error.
Human:
The text was updated successfully, but these errors were encountered: