Can this generate answer after max turns? Is there any settings to this?

[quanqigu]

Question

Can this generate answer after max turns? Is there any settings to this?

[seratch]

Yes, the exception holds the items used for the last turn. So you can catch the exception and continue with a new turn like #2052 does (we haven't yet decided to have this helper). Also, if 10 turns (the default number) are not really sufficient for your use case, you can customize the upper limit by passing a runner config.

[quanqigu]

@seratch We can catch the exception and handle it, but that requires more code. I'm thinking about the agent's autonomous planning, so no matter how many turns are set, it's possible to exceed them.

Therefore, the first approach is a fallback strategy like the one in the PR, where the output is generated once the maximum number of turns is reached.

For the second approach, is it possible to make it smarter? For instance, by making the agent realize that it has limited opportunities left and should produce an output promptly, such as when it reaches the maximum turns minus one. At that point, the model should refrain from attempting further tool calls and instead generate a response directly. Do you think for the second approach, we could modify the prompt at the last moment by adding something like this?

[seratch]

For the second point, I don't think it’s feasible to achieve a stable solution through prompt engineering alone, since each turn involves a different LLM call.

Generally speaking, if it takes more than 10 turns, it may indicate that the instructions could be clearer to help the model make better progress in each turn. So, you may want to tweak instructions and app design for better performance. That said, for apps that genuinely require more than 10 turns, you can adjust the max number based on the results of behavior evaluations.