Abstract message sequence to handlers that can be overriden.

[kiranandcode]

This PR updates LiteLLMProvider to handle message sequences through a handler. The agent.py has been updated to reuse this mechanism. The tests have been updated to enforce the respective invariants for call_tool and call_assistant, that is:

call_tool: fresh message sequence
call_assistant: does not duplicate messages, only saves messages on successful execution of fwd

[kiranandcode]

Depends on #527

[eb8680]

Let's see if we can simplify this a bit

[eb8680]

Another invariant we should be enforcing and testing is that Messages are always ordered correctly in the input to completion. I'm a little concerned that this will reorder histories by ordering Messages by handler context, rather than actual global time.

[kiranandcode]

Another invariant we should be enforcing and testing is that Messages are always ordered correctly in the input to completion. I'm a little concerned that this will reorder histories by ordering Messages by handler context, rather than actual global time.

yes, that is a problem,

        msg_list = list(messages)
        seen_ids = {m["id"] for m in msg_list}

        prefix = [
            m for msg_id, m in self.message_sequence.items() if msg_id not in seen_ids
        ]

        message, tool_calls, result = fwd(prefix + msg_list, *args, **kwargs)
        self.message_sequence[message["id"]] = message
        return message, tool_calls, result

This implementation of MessageSequence.call_assistant was specifically written because messages were not being inserted in the right order. Maybe messagesequence shouldn't be an ordered dict, but a dict, and message_sequence should order by id, or timestamp?

[kiranandcode]

@eb8680 updated with a new design that does enforce all the invariants + tests pass

Key interface is a new operation get_message_sequence() -> collections.OrderedDict[str, Message]. Its default rule creates a new instance each time. Template.__apply__ grabs the current message_sequence and installs a local handler that persists the message_sequence for the duration of Template.__apply__. `

call_tool installs a fresh collections.OrderedDict[str, Message] before calling calling tools, enforcing that message sequences for tools are fresh, maintaining its invariant

call_assistant grabs the current message sequence using get_message_sequence, and calls .values() to create the message list, enforcing that messages are not duplicated.