We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
litellm --model mistral/mistral-medium --drop_params
litellm --test
LiteLLM: Making a test ChatCompletions + streaming request to proxy. Model=gpt-3.5-turbo LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='In the', function_call=None, role='assistant', tool_calls=None), finish_reason=None, index=0)], created=1704731505, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' quiet of the night,', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731505, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=" under the moon'", function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731505, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='s gentle light,\n', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731505, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='A whispering bree', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='ze stirs the', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' leaves', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' with delight.\n', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='The stars twinkle', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' bright, casting a', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' silver light,\nIn', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' this peaceful moment,', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' all feels just right', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='.\n\nSo take', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731506, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' a deep breath, and', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' let go of your', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' sight,\nClose your', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' eyes and listen, to', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' the stillness of the', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' night.\nLet', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' your worries drift', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' away, like a fe', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='ather in flight,', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='\nAnd let the night', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731507, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content="'s calmness,", function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731508, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content=' bring you pure delight', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731508, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='cmpl-04a22dde86c9405b82032666b4fb02d0', choices=[Choice(delta=ChoiceDelta(content='.', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0)], created=1704731508, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) LiteLLM: streaming response from proxy ChatCompletionChunk(id='chatcmpl-1c0b378c-7a41-49c3-9090-222dcd47afcd', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, role=None, tool_calls=None), finish_reason='stop', index=0)], created=1704731508, model='mistral-medium', object='chat.completion.chunk', system_fingerprint=None, usage={}) making completion request to proxy Traceback (most recent call last): File "/Users/jakekoenig/mentat/.venv/bin/litellm", line 8, in <module> sys.exit(run_server()) ^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/proxy/proxy_cli.py", line 365, in run_server response = client.completions.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/_utils/_utils.py", line 299, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/resources/completions.py", line 559, in create return self._post( ^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 1055, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 834, in request return self._request( ^^^^^^^^^^^^^^ File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 877, in _request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: Error code: 400 - {'detail': 'OpenAIException - Error code: 400 - {\'object\': \'error\', \'message\': \'Expected last role to be user but got RoleEnum.system\', \'type\': \'internal_error_proxy\', \'param\': None, \'code\': \'1000\'}\n\nTraceback (most recent call last):\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/main.py", line 2510, in atext_completion\n response = await response\n ^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/llms/openai.py", line 402, in acompletion\n raise e\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/llms/openai.py", line 387, in acompletion\n response = await openai_aclient.chat.completions.create(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1199, in create\n return await self._post(\n ^^^^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 1474, in post\n return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 1275, in request\n return await self._request(\n ^^^^^^^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/openai/_base_client.py", line 1318, in _request\n raise self._make_status_error_from_response(err.response) from None\nopenai.BadRequestError: Error code: 400 - {\'object\': \'error\', \'message\': \'Expected last role to be user but got RoleEnum.system\', \'type\': \'internal_error_proxy\', \'param\': None, \'code\': \'1000\'}\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 1267, in completion\n response = await litellm.atext_completion(**data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/utils.py", line 2363, in wrapper_async\n raise e\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/utils.py", line 2255, in wrapper_async\n result = await original_function(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/main.py", line 2528, in atext_completion\n raise exception_type(\n ^^^^^^^^^^^^^^^\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/utils.py", line 6594, in exception_type\n raise e\n File "/Users/jakekoenig/mentat/.venv/lib/python3.11/site-packages/litellm/utils.py", line 5616, in exception_type\n raise APIError(\nlitellm.exceptions.APIError: OpenAIException - Error code: 400 - {\'object\': \'error\', \'message\': \'Expected last role to be user but got RoleEnum.system\', \'type\': \'internal_error_proxy\', \'param\': None, \'code\': \'1000\'}\n'}
### Twitter / LinkedIn details _No response_
The text was updated successfully, but these errors were encountered:
Should be live in the next release
Sorry, something went wrong.
No branches or pull requests
What happened?
litellm --model mistral/mistral-medium --drop_params
litellm --test
Relevant log output
The text was updated successfully, but these errors were encountered: