EN Course 02 Streaming Transport

Streaming Transport

The transport layer sends a provider request and converts the streaming response into one internal message stream.

llm_call() {
    local messages="$1" max_tokens="${2:-$MAX_TOKENS}" use_thinking="${3:-$THINKING}" body system_prompt
    system_prompt=$(agent_build_prompt)
    # Field order follows alphabetical (aligned with Go/Rust map serialization)
    body="{\"max_tokens\":${max_tokens},\"messages\":${messages},\"model\":\"${MODEL}\""
    body+=",\"stream\":true"
    [[ -n "$system_prompt" ]] && body+=",\"system\":\"$(util_json_escape "$system_prompt")\""
    [[ -n "$TOOL_DEF_JSON" ]] && body+=",\"tools\":${TOOL_DEF_JSON}"
    body+="}"
    printf '%s' "$body" | util_body_convert | llm_stream_curl | sse_convert | sse_parse
}

The important design is the pipeline:

Claude-shaped request
  -> provider body conversion
  -> HTTP stream
  -> provider SSE conversion
  -> internal SSE parser
  -> RESP-like runtime messages

Provider Compatibility Boundary

The rest of the runtime speaks the Claude-style internal model:

• system
• messages
• tools
• tool_use
• tool_result

OpenAI Chat compatibility belongs at the transport boundary. On request, the body is converted from the internal shape to OpenAI Chat. On response, OpenAI deltas are converted back into the same internal event stream.

This keeps agent_loop, tool dispatch, display, and store code independent from provider-specific formats.

The provider switch is configured before the loop starts:

case "$PROVIDER" in
    claude)
        util_body_convert() { cat; }
        sse_convert()  { cat; }
        ;;
    openai)
        util_body_convert() { util_awk_run -f "$AWK_DIR/json.awk" -f "$AWK_DIR/transport_openai_body.awk"; }
        sse_convert()  { util_awk_run -f "$AWK_DIR/json.awk" -f "$AWK_DIR/transport_openai_sse.awk"; }
        ;;
esac

After conversion, all providers pass through the same parser:

sse_parse() {
    util_awk_run -v verbose="${VERBOSE:-false}" \
        -f "$AWK_DIR/json.awk" \
        -f "$AWK_DIR/protocol.awk" \
        -f "$AWK_DIR/todo_protocol.awk" \
        -f "$AWK_DIR/claude_sse.awk"
}

This makes OpenAI compatibility a boundary concern, not an agent-loop concern.

Streaming Matters

The model may emit:

• text
• thinking
• usage
• tool call start
• tool input deltas
• stop reason
• error

The loop should not wait for a complete JSON response before showing useful output. It streams text and thinking immediately while accumulating tool calls until they are complete.

AWK as Parser and Compute Layer

The Bash runtime keeps heavier parsing and computation in AWK:

• HTTP stream framing
• JSON extraction
• Claude/OpenAI SSE normalization
• tool protocol parsing
• event replay
• stats update
• compact DP decision
• terminal title formatting

That keeps shell functions focused on orchestration while AWK handles structured text processing.

Next

Message Protocol and Display explains how internal messages are rendered without mixing transport logic with terminal output.