feat(streaming): initial implementation of streaming agentservice execution

[douglas-reid]

This PR establishes an initial implementation of streaming utilities for AgentService endpoints, based on FunctionsBasedAgent runs. This will allow clients to invoke an agent run through an async endpoint and stream, via SSE, block creation events related to the agent's execution. These events can include status messages (from Agents and Tools) as well as generated blocks (from LLMs and Tools).

With this code, new async endpoints can be exposed with code like the following:

    @post("async_prompt")
    def async_prompt(
        self, prompt: Optional[str] = None, context_id: Optional[str] = None, **kwargs
    ) -> StreamingResponse:
        ctx_id, history_file = self._streaming_context_id_and_file(context_id=context_id, **kwargs)
        task = self.invoke_later(
            "/prompt", arguments={"prompt": prompt, "context_id": ctx_id, **kwargs}
        )
        return StreamingResponse(task=task, file=history_file)

The PR ensures all Blocks created in the service of a run_agent call are tagged with the proper request-id, and that all calls to the LLM are streaming-compatible.

A test is provided that demonstrates an approach to consuming a stream of events, based on the sseclient-py library. A Generator is constructed that emits Blocks until a terminal block for a request is found in the stream.

[eob]

@douglas-reid if you extend BaseModel instead of CamelModel then the serialization is underscored rather than the camelcase that we use for engine comms.

It sneaks through in the case since it's internal to an object's response, but it breaks the parsing of the task in the Typescript client.

Would it be possible to add a test that, e.g., response.task.requestId is serialized as requestId instead of request_id just to make sure that the clients demarshalling it won't explode?

[douglas-reid]

sure

[eob]

A few non-blocking thoughts in there, but this looks fantastic. Excited for it to be merged!!

[eob]

These error messages are awesome -- thank you for adding them

[eob]

Is this the situation where in practice we've just been using context_id whereas in theory we're hashing the whole context_keys object?

[douglas-reid]

yeah. we have a mismatch in our exposed args in /prompt vs. our actual capability. not sure if that matter practically yet.

[eob]

I think in a V2 we'll want to follow up and see if we can have the things that are later-invoked auto-block this task from completing (thinking about the comment string about the semantics of the Task below.. we could maybe try to make conformance to that automatic)