Setting temperature for evaluation #2537
Replies: 1 comment
|
@staehlmich the seam is the
The subclass route, on the last line, is the way through: from giskard.llm.client.litellm import LiteLLMClient
class FixedTemperature(LiteLLMClient):
def complete(self, messages, temperature=1.0, **kwargs):
return super().complete(messages, temperature=0.0, **kwargs)
client = FixedTemperature("azure/gpt-4o", completion_params={"api_base": "..."})
report = evaluate(answer_fn, testset=testset, llm_client=client)Nothing but temperature has to move. The constructor ignores what Are you staying on v2, or moving to |
Uh oh!
There was an error while loading. Please reload this page.
Hi everyone,
I wanted to set the temperature of the model when running giskard.rag.evaluate. I've been trying to find an answer in the docs, but couldn't find one. Thanks for your help.
All reactions