Added tool calls support for LLM clients #1694
Conversation
…m-functions-to-tools
…m-functions-to-tools
…api-call-from-functions-to-tools' into feature/gsk-2367-migrate-openai-api-call-from-functions-to-tools
giskard/llm/evaluators/base.py
Outdated
| temperature=self.llm_temperature, | ||
| caller_id=self.__class__.__name__, | ||
| ) | ||
| if out.function_call is None or "passed_test" not in out.function_call.args: | ||
| if len(out.tool_calls) != 1 or "passed_test" not in out.tool_calls[0].args: |
There was a problem hiding this comment.
In this specific case, we do not expect, that we can make multiple tool calls, right?
There was a problem hiding this comment.
Yes since we are evaluating a single case we expect it to be evaluated once
giskard/llm/client/base.py
Outdated
| @@ -16,6 +16,7 @@ class LLMFunctionCall: | |||
| class LLMOutput: | |||
| message: Optional[str] = None | |||
| function_call: Optional[LLMFunctionCall] = None | |||
| tool_calls: Sequence[LLMFunctionCall] = field(default_factory=list) | |||
There was a problem hiding this comment.
For this moment it is ok for a backward compatibility, but I believe, that by the end of the day we need to keep only "tool" and not "function" naming convention and thus related variables like "function_call", as well as we may want change "LLMFunctionCall" to "LLMToolCall"
There was a problem hiding this comment.
Yes I'm not sure if function_call are being used in other PR's and might be used by some users having developed there own test.
I guess it's okay to keep LLMFunctionCall for now since only tool supported for now is of type function. I guess we will need to update it once we have other tools supported by OpenAI.
|
@kevinmessiaen does this work with openai<1? We might need to add some version constraint in |
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
Yes, I tried with |
…m-functions-to-tools # Conflicts: # giskard/llm/generators/base.py
…m-functions-to-tools # Conflicts: # giskard/llm/evaluators/base.py
This reverts commit 3b60c31.
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
…m-functions-to-tools
|
kevinmessiaen
deleted the
feature/gsk-2367-migrate-openai-api-call-from-functions-to-tools
branch
Description
Added tool calls support for LLM clients, migrate function call to tool calls
Type of Change