🚀 Feature Request: [CoAgents] Stream Direct LLM Responses to State Instead of Using Function Calls

[yuvalkarmi]

🎤 Tell us your idea

I've run into an issue with LLM function calling being unreliable for content streaming. Even state-of-the-art LLMs (like Anthropic's) are inconsistent - sometimes they call functions with empty parameters, other times they pass content correctly.

Here's my current setup:

config = copilotkit_customize_config(
    config,
    emit_intermediate_state=[
        {
            "state_key": "content",
            "tool": "SetContent",
            "tool_argument": "content",
        }
    ],
)

# Forcing function calls in the prompt
SystemMessage(
    content="[A bunch of instructions omitted for brevity] You must respond ONLY with a SetContent function call..."
)

This creates a frustrating user experience where the agent looks like it's working, but nothing actually streams because of empty function arguments.

Proposed Solution

Rather than forcing responses function calls, I want to:

1. Let the LLM respond naturally (i.e. directly)
2. Stream its direct response
3. Route this stream to a different state variable (not the chat window)

The key question: Can I modify emit_intermediate_state to work with direct LLM responses rather than just function call results? This would make the system more reliable and remove the need for brittle function call patterns.

[jpr5]

The emit_intermediate_state mechanism has been significantly improved and the SDK now supports more flexible state streaming patterns via useAgent. Closing as outdated -- please check the current agent documentation for the latest approach to content streaming.