Differents results with the same prompt and same documents

[be-swarm]

Hi,
I am developing an application with Semantic Kernel and Memory Kernel.
In my prompt I ask questions about the content of the documents. In particular in order to know if, for example, a date in document A is later than a date in document B.
I sometimes get different results when I run the same prompt on the same data. Sometimes he answers yes sometimes he answers no Why?
My prompt is executed with this parameters:
var result = await _kernel.InvokePromptAsync(
prompt,
new(new OpenAIPromptExecutionSettings()
{
FunctionChoiceBehavior = FunctionChoiceBehavior.Auto(),
Temperature = 0.0
}));

Thanks for ideas.

[moonbox3]

Hi @be-swarm, even though your temp is at 0.0, these models may still not produce deterministic results because they are non-deterministic. With that said, if you are using the documents as the source of truth, you should be able to get more deterministic answers if things are working correctly. Tagging @westey-m or @dluc for some more thoughts on this.

[westey-m]

@be-swarm, there may be other ways to approach the same problem.
E.g. you could try and use Structured Outputs (see here) to extract the dates from the two documents, and then use .net code to compare the two dates. This should make the date comparison more deterministic and make it easier to troubleshoot e.g. cases where the model misinterprets the date.

Or you could supply a plugin to do the date comparison, although the model may not necessarily always choose to invoke it, but it might increase reliability.

Also, newer models are usually better at getting these things right every time, so if you are not using the latest ones already, consider upgrading.

[be-swarm]

thank you for your answers.
I added the seed parameter and the responses seem more deterministic