3.3.0
Chat History improvements
In the process of fixing a bug ( #523 ) we worked on a more efficient algorithm to manage the cut of the message history if it goes out of the context window. The intent was to minimize the context loss when cutting.
The context window cut must be considered the last resort to prevent fatal errors with provider APIs. Before reaching the limit of the context window you should use summarization to compact the context into a single message as a new starting point.
In order to handle all edge cases, the internal trimmer can identify a cutting point slightly less aggressive than the initially identified. To make sure the agent conversation stays in the limit, you should configure the context window limit in the agent chat history with a margin of 5%-10% from the actual limit of the underlying model.
If your model works with a 200K context window, you should instantiate your chat history with 190K for example.
Evaluation improvements
We introduced a couple of built-in rules for Ai as a judge assertions: