MultiChainComparison input from ChainOfThought

[Bobsimonoff]

Maybe I am missing the short cut:

ChainOf thought, where n=4, returns a Prediction, which has a single rationale, a single answer, and a Completions object. The completions object has an array of 3 strings for the rationale and an array of 3 strings for the answer.

The MultiChainComparison however takes an array of Predictions.

Wouldn't it be more natural for MultiChainComparison to accept the output Predictions object or the Completions object rather than forcing the caller to reformat ChainOfThought's output to comply with MultiChainComparison's input?

Also, if n=4, shouldn't there be 4 completions not 3?

[arnavsinghvi11]

Hi @Bobsimonoff ,

I believe the current logic just requires that additional step of creating dspy.Prediction objects, but this makes sense and would be great to add to MultiChainComparison via a PR.

As for the number of outputted completions, I believe it has to do with the generation extension logic where if the produced completion is deemed incomplete (missing the corresponding output fields), it is not included in the final set of completions - relevant PR #920.

[Bobsimonoff]

Thanks @arnavsinghvi11 for your response ... so n really means up to the requested number of responses, not n responses. Didn't realize that,

[arnavsinghvi11]

Tagging the discussion here if you'd like to follow. Closing this for now.