-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add transparent conversion for "tools" to "functions" in v1/chat/completions endpoint #1712
Comments
Hey @stippi2 thanks for the heads up and the initial work! To get more debugging informations you could comment out this line: Line 149 in 5e155fb
--debug flag, currently it's not the case).
It will avoid the server to recovery so you should be able to see the full stack trace of the error |
Thanks for the hint! |
The panic is because at some point the |
closing as handled in #1715 |
Is your feature request related to a problem? Please describe.
Yes. I got LocalAI running, both with Docker and building on my Mac. I tested with the Mistral model, but noticed that it didn't use functions. The reason is that 'functions' is actually already deprecated and there is now a 'tools' array instead. And my client is already using that.
Describe the solution you'd like
The
tools
array is a layer abovefunctions
. Eachtool
has a 'type', and the only supported type is 'function'. So if the type isfunction
, there is a second field namedfunction
, which is then the Function object that used to be directly contained in the oldfunctions
array. So I imaging a simple conversion can be done, and I've actually tried to implement that:... but it crashes. Unfortunately, I only get a 500 and a nil panic in the log, but not actually where it happens. With some guidance, maybe I can provide a PR.
The text was updated successfully, but these errors were encountered: