Message formatting error when using OpenAI compatible server

[Moonlight-YS]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:Windows 11
- Continue:v0.8.26
- IDE:vscode 1.89.0
- Model:gpt-3.5-turbo
- Language environment : Chinese

Description

I'm using a local openai format api service that I built myself, and when using crtl + i and tabauto, I can read this ERROR from the log log:

[warning][index<837,22>] Params body.messages invalid: [Mismatch] -> function isArray() { [native code] }
[error][index<1051,28>] APIException [Error]: Params body.messages invalid

Using the sidebar dialog works fine without any errors

To reproduce

This happens with crtl + i and tabauto

Log output

No errors were found in vscode's developer console

[sestinj]

@Moonlight-YS Do you have a way of capturing the request body? It seems that this is happening for requests to the /completions endpoint (used for autocomplete and ctrl/cmd+I), but is working with /chat/completions (which is used in the sidebar).

For the completions endpoint, the OpenAI format should accept the "prompt" attribute rather than "messages", and it is just a string: https://platform.openai.com/docs/api-reference/completions

[Moonlight-YS]

Oh, thanks for the reply, I should have thought of that reason, the situation is exactly what you said, my local service doesn't support the old endpoints format.So do a lot of other self-local services, are there any plans for future adaptations for this? If not please close this issue, sorry for taking up your time, have a nice day!

[sestinj]

@Moonlight-YS there are a couple of other services that don't support /completions, and (I should have mentioned this before), you can set "useLegacyCompletionsEndpoint": false for your model in config.json. This will force Continue to go through the /chat/completions endpoint, even though it means that we can't use the best formatting tricks.

For autocomplete unfortunately, the /chat/completions endpoint will probably not work at all

[Moonlight-YS]

@Moonlight-YS there are a couple of other services that don't support /completions, and (I should have mentioned this before), you can set "useLegacyCompletionsEndpoint": false for your model in config.json. This will force Continue to go through the /chat/completions endpoint, even though it means that we can't use the best formatting tricks.

For autocomplete unfortunately, the /chat/completions endpoint will probably not work at all

Thank you for your reply.I will close this issue

On the other hand, some official apis don't support /completions in any model, such as deepseek，they also use the openai format, which recently launched an API, but only supports new formats.