Add callModel API with multiple consumption patterns

[mattapperson]

Summary

Adds a new client.callModel() API that provides multiple flexible ways to consume streaming responses from the OpenRouter API with automatic tool execution support.

TODO

• Rename generated Tool type - The Speakeasy-generated Tool type (from src/models/tool.ts) conflicts with our new flagship Tool type. The generated type should be renamed (e.g., to ChatTool or LegacyTool) in a future PR to avoid naming conflicts.

New Types, Methods, and Enums

Enums

• ToolType - Enum for tool types (currently supports Function = "function")

Core Types

Tool Definition Types

• Tool - Union type encompassing all tool types with automatic execution (ToolWithExecution | ToolWithStreamingExecution). Represents tools with Zod schemas that are automatically executed by the SDK
• ToolWithExecution - Tool with a regular synchronous or asynchronous execute function
• ToolWithStreamingExecution - Tool with an async generator execute function that emits progress events during execution. Important: eventSchema events are for your app (e.g., UI updates) and are NOT sent to the model. Only the last yielded value (outputSchema) is sent to the model.
• TurnContext - Context object passed to tool execute functions containing: numberOfTurns, messageHistory, model/models
• ParsedToolCall - Represents a parsed tool call from the API response with id, name, and arguments
• ToolResult - Result of executing a tool, including toolCallId, toolName, result, preliminaryResults (for streaming tools), and optional error

Configuration Types

• MaxToolRounds - Configuration for tool execution loop limits. Can be a number (max rounds) or a function (context: TurnContext) => boolean that returns true to continue or false to stop
• CallModelOptions - Options for creating a ModelResponse, including request, client, options, tools, and maxToolRounds

Stream Event Types

• ResponsesStreamEvent - Stream event type that extends OpenResponsesStreamEvent with tool progress events
• ToolProgressEvent - Event emitted during streaming tool execution containing type, toolCallId, result, and timestamp
• ToolStreamEvent - Stream events for tool execution, including delta (argument streaming) and progress
• ChatStreamEvent - Stream events for chat format including content.delta, message.complete, tool.progress, and pass-through events

Classes

• ModelResponse - Main class for consuming API responses with multiple patterns. Provides methods for streaming and awaiting completion

Methods (ModelResponse)

• getMessage() - Returns a Promise that resolves to the complete AssistantMessage (tools auto-executed if provided)
• getText() - Returns a Promise that resolves to just the text content from the response (tools auto-executed if provided)
• getFullResponsesStream() - Returns an AsyncIterableIterator of ResponsesStreamEvent for all response events including tool progress
• getTextStream() - Returns an AsyncIterableIterator of string for streaming text content deltas
• getNewMessagesStream() - Returns an AsyncIterableIterator of AssistantMessage for streaming incremental message updates
• getReasoningStream() - Returns an AsyncIterableIterator of string for streaming reasoning deltas (for models that support reasoning)
• getToolStream() - Returns an AsyncIterableIterator of ToolStreamEvent for streaming tool call arguments and progress results
• getFullChatStream() - Returns an AsyncIterableIterator of ChatStreamEvent for streaming in a chat-friendly format with content deltas, completion events, and tool progress
• getToolCalls() - Returns a Promise that resolves to an array of ParsedToolCall from the completed response
• getToolCallsStream() - Returns an AsyncIterableIterator of ParsedToolCall for streaming structured tool calls as they complete
• cancel() - Cancels the underlying stream and cleans up resources

Utility Functions

• isToolProgressEvent(event) - Type guard to check if an event is a ToolProgressEvent
• hasExecuteFunction(tool) - Type guard to check if a tool has an execute function
• isStreamingTool(tool) - Type guard to check if a tool uses streaming execution (has eventSchema)
• isExecutionTool(tool) - Type guard to check if a tool is a regular (non-streaming) execution tool

SDK Method

• openrouter.callModel(request, options) - New method on OpenRouter client that returns a ModelResponse for consuming responses with automatic tool execution support

Features

Tool Support with Context

Tools can now access conversation context during execution:

import { ToolType } from "@openrouter/sdk";

const weatherTool = {
  type: ToolType.Function,
  function: {
    name: "get_weather",
    description: "Get current weather",
    inputSchema: z.object({ location: z.string() }),
    outputSchema: z.object({ 
      temperature: z.number(),
      description: z.string() 
    }),
    execute: async (params, context) => {
      // Context is optional - access conversation state if needed
      console.log(`Turn ${context?.numberOfTurns}`);
      console.log(`Model: ${context?.model}`);
      return { temperature: 72, description: "Sunny" };
    }
  }
};

const response = client.callModel({
  model: "openai/gpt-4o",
  input: "What's the weather in SF?",
  tools: [weatherTool],
  // Optionally, control tool rounds dynamically with context
  maxToolRounds: (context) => {
    return context.numberOfTurns < 3; // Allow up to 3 turns
  }
  // or by a hard coded number
  // maxToolRounds: 5
});

Streaming Tools with Progress Events

Tools can emit progress events using async generators. Important: These progress events are for your application (e.g., UI updates) and are NOT sent to the model. Only the final result (last yielded value) is sent to the model.

const processingTool = {
  type: ToolType.Function,
  function: {
    name: "process_data",
    inputSchema: z.object({
      data: z.string()
    }),
    // Progress events - for your app only, NOT sent to model
    eventSchema: z.object({
      type: z.enum(["start", "progress", "complete"]),
      message: z.string(),
      progress: z.number().optional()
    }),
    // Final output - this IS sent to the model
    outputSchema: z.object({
      type: z.literal("complete"),
      message: z.string(),
      progress: z.number()
    }),
    execute: async function* (params, context) {
      // Yield progress events (for your app UI, NOT sent to model)
      yield {
        type: "start",
        message: `Started processing: ${params.data}`,
        progress: 0
      };

      // Simulate work
      await new Promise(resolve => setTimeout(resolve, 1000));

      yield {
        type: "progress",
        message: "Processing halfway done",
        progress: 50
      };

      await new Promise(resolve => setTimeout(resolve, 1000));

      // Last yield is the final result (sent to model - must match outputSchema)
      yield {
        type: "complete",
        message: `Completed: ${params.data.toUpperCase()}`,
        progress: 100
      };
    }
  }
};

// Stream progress results as they arrive (in your app)
for await (const event of response.getToolStream()) {
  if (event.type === "progress") {
    console.log("Progress:", event.result);
  }
}

Consumption Patterns

Users can consume responses in any combination they prefer:

const response = client.callModel({
  model: "meta-llama/llama-3.2-1b-instruct",
  input: [{ role: "user", content: "Hello!" }]
});

// Get complete text
const text = await response.getText();

// Get complete message
const message = await response.getMessage();

// Stream text deltas
for await (const delta of response.getTextStream()) {
  console.log(delta);
}

// Stream incremental message updates
for await (const msg of response.getNewMessagesStream()) {
  console.log(msg);
}

// Stream reasoning deltas (for reasoning models)
for await (const delta of response.getReasoningStream()) {
  console.log(delta);
}

// Stream tool call deltas with progress results
for await (const event of response.getToolStream()) {
  if (event.type === "progress") {
    console.log("Tool progress:", event.result);
  }
}

// Stream all raw events
for await (const event of response.getFullResponsesStream()) {
  console.log(event);
}

// Stream in chat-compatible format
for await (const chunk of response.getFullChatStream()) {
  console.log(chunk);
}

Key Features

• ToolType Enum: Type-safe tool type definitions exported from SDK
• Tool Context: Tools receive conversation state (turn number, history, model)
• Streaming Tools: Emit progress events using async generators with eventSchema (for app UI, not sent to model)
• Dynamic Control: maxToolRounds can be a function for smart termination
• Backward Compatible: Context parameter is optional
• Concurrent Access: Multiple consumption patterns can be used simultaneously
• Sequential Access: Can consume same response multiple times
• Lazy Initialization: Stream only starts when first accessed
• Type Consistency: Returns AssistantMessage type consistent with chat API
• Clear API: Explicit method names make the API more discoverable and intuitive
• Progress vs Output: eventSchema for UI progress (not sent to model), outputSchema for final result (sent to model)

[louisgv]

nit: I think we should use pnpm + pnpm dlx for these actions still, unless we think there's no gain using pnpm -- at that point I'd want to move all to npm

[louisgv]

add bracket

[louisgv]

This reads like a weird pattern -- would like to improve the code quaity.

[louisgv]

these type cast didn't feel safe

[louisgv]

There's an iife here and I'm a bit iffy about them

[subtleGradient]

Code Review: PR #49 - feat: expose callModel from SDK client

Summary

This PR adds a comprehensive callModel API to the SDK that supports automatic tool execution with Zod schema validation and multiple response consumption patterns.

Good

1. Excellent test coverage - Comprehensive tests for all major functionality including edge cases, error handling, and type inference (52+ tool-related tests)
2. Strong type safety - Effective use of TypeScript generics and Zod for compile-time and runtime validation
3. Multiple consumption patterns - Flexible API supporting getText(), getMessage(), streaming, and concurrent consumers via ReusableReadableStream

Questions & Suggestions

[question] Implementation complexity

The implementation adds ~2000 lines across 6 new files. While each file has clear responsibilities, is this level of complexity necessary? Could some utilities be simplified or consolidated?

• Evidence: [CODE] 6 new files totaling ~1886 lines for the feature

[question] Tool orchestration limitation

The tool orchestrator mentions using previousResponseId for continuation but doesn't implement it (src/lib/tool-orchestrator.ts:129-134). Is this a known limitation with the current API?

• Evidence: [CODE] Comment indicates planned functionality not yet implemented

[nit] Examples typecheck disabled

CI typecheck for examples is disabled due to "Speakeasy generation bugs". Consider adding a tracking issue to re-enable when the upstream issue is resolved.

• Evidence: [CODE] .github/actions/validate-sdk/action.yaml - typecheck commented out

[nit] Serial tool execution

Tools are executed serially rather than in parallel (src/lib/tool-orchestrator.ts:94-122). For multiple independent tool calls, parallel execution could improve performance.

• Evidence: [CODE] Sequential for-loop for tool execution

Verdict: APPROVE

This is a well-implemented feature with comprehensive testing and good documentation. The code is production-ready with proper error handling and safeguards against infinite loops. The questions raised are about optimization opportunities rather than blocking issues.