Skip to content

Conversation

@nmaguiar
Copy link
Collaborator

@nmaguiar nmaguiar commented Jan 22, 2026

Motivation

  • Add opt-in streaming responses to the genai implementations so callers can receive progressive output (deltas) from providers that support SSE/streaming.
  • Reuse existing $rest(...).get2Stream/post2Stream plumbing and local stream helpers to parse SSE/NDJSON payloads and integrate with current conversation/tool-calling flow.

Description

  • Implemented an SSE/NDJSON helper _readSseStream and provider-specific _requestStream functions that call $rest(...).get2Stream/post2Stream with Accept: text/event-stream where appropriate.
  • Added rawPromptStream and promptStream methods to each provider implementation (OpenAI, Gemini, Ollama, Anthropic) to parse streamed payloads, aggregate content and events, surface tool-call deltas, and capture stats.
  • Exposed streaming APIs on the public interfaces by adding ow.ai.gpt.prototype.promptStream, ow.ai.gpt.prototype.rawPromptStream, and $gpt.promptStream / $gpt.rawPromptStream wrappers.
  • Kept non-streaming behavior unchanged and added odoc-compatible entries for the new streaming methods.

Testing

  • No automated tests were executed as part of this change (no CI/test run performed).
  • To validate manually with automation, create a streaming-capable provider client and call rawPromptStream/promptStream (for example via $gpt({...}).promptStream(...)), or run the repository test suite with cd tests && ojob autoTestAll.yaml as recommended by project guidelines.

@nmaguiar nmaguiar marked this pull request as ready for review January 23, 2026 04:03
@nmaguiar nmaguiar merged commit d0ca1ea into t8 Jan 23, 2026
2 checks passed
@nmaguiar nmaguiar deleted the codex/implement-stream-responses-in-genai-implementations branch January 23, 2026 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants