Add Tests for extend generation logic

[mikeedjones]

Added tests for extend generation logic to avoid any further breaking changes (#920) which must then be reverted (#1169)

I'm not sure the behaviour currently exhbitied is intended? I have included the generation logs as a comment in each test to illustrate how the lm is being prompted to show the DummyLM responses in context.

Adds tests to illustrate issue #1232 and to test against any solution for that issue.

Cheers

[arnavsinghvi11]

Hi @mikeedjones , thanks for the PR. can you rebase to main and push to retrieve the latest tests?

[mikeedjones]

Hi @arnavsinghvi11 - updated - thanks!

[mikeedjones]

Hi @arnavsinghvi11 Is there anything else which needs to be added to this?

[mikeedjones]

@arnavsinghvi11 - Anything which needs to be done to merge these tests?

[mikeedjones]

Hi, @arnavsinghvi11, it looks like the use case these tests apply to is no longer supported in dspy 2.5? A ValueError is thrown by ChatAdapter if the LM doesn't reproduce every output field. Is there a plan to put support for that back in to a future release? I know that extend-generation capability has been used by a few of the optimizers in the past.