Convert JS SDK streaming APIs from callbacks to async iterables by Copilot · Pull Request #545 · microsoft/Foundry-Local

Copilot · 2026-03-23T22:22:46Z

⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.

…pattern Addresses #507 - ChatClient.completeStreamingChat() and AudioClient.transcribeStreaming() now return AsyncIterable<any> instead of accepting callback functions, enabling the standard `for await (const chunk of ...)` pattern. BREAKING CHANGE: completeStreamingChat() and transcribeStreaming() no longer accept callback parameters. Use `for await` iteration. Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com> Agent-Logs-Url: https://github.com/microsoft/Foundry-Local/sessions/f8227df8-7016-4fe1-9884-56bbdcf326b7

Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com> Agent-Logs-Url: https://github.com/microsoft/Foundry-Local/sessions/f8227df8-7016-4fe1-9884-56bbdcf326b7

vercel · 2026-03-23T22:22:51Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview, Comment	Mar 24, 2026 10:16pm

baijumeswani · 2026-03-23T23:48:25Z

@copilot do you need to make similar changes to rust sdk as well?

Copilot · 2026-03-23T23:49:35Z

@copilot do you need to make similar changes to rust sdk as well?

No — the Rust SDK already returns stream types (ChatCompletionStream and AudioTranscriptionStream via JsonStream<T>) from complete_streaming_chat and transcribe_streaming, so it already supports the idiomatic async iteration pattern. Only the JS SDK had the callback-based API.

Copilot

Pull request overview

This PR updates the JS SDK streaming APIs to follow the common for await (...) pattern by removing callback-based overloads and returning AsyncIterable streams for chat completions and audio transcription.

Changes:

Refactors ChatClient.completeStreamingChat() and AudioClient.transcribeStreaming() to return AsyncIterable<any> instead of accepting callbacks.
Updates streaming tests and examples to iterate with for await (...).
Regenerates/updates TypeDoc + READMEs to reflect the new streaming API surface.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
sdk/js/src/openai/chatClient.ts	Converts streaming chat completion from callback-based API to `AsyncIterable` adapter.
sdk/js/src/openai/audioClient.ts	Converts streaming transcription from callback-based API to `AsyncIterable` adapter.
sdk/js/test/openai/chatClient.test.ts	Migrates streaming chat tests to `for await` iteration and removes callback validation tests.
sdk/js/test/openai/audioClient.test.ts	Migrates streaming audio tests to `for await` iteration and removes callback validation tests.
sdk/js/examples/chat-completion.ts	Updates streaming example to `for await` usage.
sdk/js/examples/tool-calling.ts	Updates streaming tool-calling example to `for await` usage.
sdk/js/examples/audio-transcription.ts	Updates streaming transcription example to `for await` usage.
sdk/js/README.md	Updates SDK README examples to use async iteration for streaming.
README.md	Updates root README streaming transcription snippet to async iteration.
samples/js/native-chat-completions/app.js	Updates sample streaming chat usage to async iteration.
samples/js/chat-and-audio-foundry-local/src/app.js	Updates sample streaming summary generation to async iteration.
samples/js/audio-transcription-example/app.js	Updates sample streaming transcription usage to async iteration.
sdk/js/docs/classes/ChatClient.md	Updates generated API docs for new `AsyncIterable` streaming signature + example.
sdk/js/docs/classes/AudioClient.md	Updates generated API docs for new `AsyncIterable` streaming signature + example.
sdk/js/docs/classes/ChatClientSettings.md	Regenerated docs reflecting optional property markers.
sdk/js/docs/classes/AudioClientSettings.md	Regenerated docs reflecting optional property markers.
sdk/js/docs/classes/ResponsesClientSettings.md	Regenerated docs reflecting optional property markers.
sdk/js/docs/classes/Model.md	Updates generated throws description text.
sdk/js/docs/README.md	Regenerated docs reflecting optional property markers and signatures.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sdk/js/test/openai/audioClient.test.ts

sdk/js/src/openai/chatClient.ts

sdk/js/src/openai/audioClient.ts

sdk/js/test/openai/chatClient.test.ts

baijumeswani · 2026-03-24T00:26:09Z

@copilot please address this feedback.

…dation tests - Add return() method to async iterators in chatClient.ts and audioClient.ts to prevent unbounded buffering when consumers break out of for-await early - Add cancelled/error guards in native streaming callbacks to skip work after error or cancellation - Fix validation tests to assert synchronous throws directly instead of wrapping in unreachable for-await loops Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com> Agent-Logs-Url: https://github.com/microsoft/Foundry-Local/sessions/69aaf402-f4bd-496b-9d37-e139a3223cdb

Copilot · 2026-03-24T00:29:54Z

@copilot please address this feedback.

Addressed all 6 items from the review in commit 669ad5e:

return() hook added to both chatClient.ts and audioClient.ts async iterators — sets a cancelled flag, clears the buffer, and wakes any pending next() so early break from for await stops accumulating chunks.
Callback guards — the native streaming callback now checks if (cancelled || error) return; at the top, skipping JSON parse and push work once iteration is done or errored.
Test fixes — validation tests in chatClient.test.ts and audioClient.test.ts now assert the synchronous throw directly on the method call, removing the unreachable for await loop.

Copilot

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sdk/js/src/openai/chatClient.ts

sdk/js/src/openai/audioClient.ts

sdk/js/docs/classes/ChatClient.md

baijumeswani · 2026-03-24T05:21:55Z

@copilot please address this feedback.

…tion docs, tools example - Replace O(n) chunks.shift() with head-index + compaction for O(1) dequeue - Add nextInFlight guard to throw on concurrent next() calls - Add comment in return() explaining native stream cancellation limitation - Update completeStreamingChat JSDoc example to show both overloads - Regenerate TypeDoc API docs Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com> Agent-Logs-Url: https://github.com/microsoft/Foundry-Local/sessions/dfa24137-7f37-4aa6-adb9-a735b7ce4878

Copilot · 2026-03-24T05:26:53Z

@copilot please address this feedback.

All 7 items addressed in 1d4f8d7:

O(1) dequeue — replaced chunks.shift() with a head-index approach in both chatClient.ts and audioClient.ts; the array compacts when all buffered chunks are consumed (head === chunks.length).
Concurrent next() guard — added a nextInFlight flag that throws if next() is called before the previous call resolves.
Native cancellation docs — return() now documents that CoreInterop.executeCommandStreaming has no abort support, so the koffi callback may still fire but will no-op via the cancelled guard.
Docs example fix — the completeStreamingChat JSDoc now shows both the messages-only and messages+tools overloads; TypeDoc regenerated.

prathikr · 2026-03-24T21:20:26Z

sdk/js/src/openai/audioClient.ts

-                            // Don't throw from callback - store first error and stop processing
-                            error = new Error(`Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`, { cause: e });
-                            return;
+        const coreInterop = this.coreInterop;


is this necessary? can we just use this.coreInterop on L#130?

Yes, the local capture is necessary. The [Symbol.asyncIterator]() method on line 123 is a regular method on the returned object literal, so this inside it refers to that object — not the AudioClient instance. Added a comment explaining this in d39bad8.

prathikr · 2026-03-24T21:20:52Z

sdk/js/src/openai/audioClient.ts

-                            error = new Error(`Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`, { cause: e });
-                            return;
+        const coreInterop = this.coreInterop;
+        const modelId = this.modelId;


same question as for coreInterop

Same reason as coreInterop — this is unavailable inside the [Symbol.asyncIterator]() closure. Comment added in d39bad8.

prathikr · 2026-03-24T21:23:17Z

sdk/js/src/openai/audioClient.ts

+                    if (resolve) {
+                        const r = resolve;
+                        resolve = null;
+                        r();


what does this do? can we include a comment?

It resolves the promise that a pending next() call is awaiting, waking it up so it can see that the stream has ended (done = true). Added clarifying comments in d39bad8.

prathikr · 2026-03-24T21:25:17Z

sdk/js/src/openai/audioClient.ts

+                    }
+                });

+                return {


is this a nested return statement? kind of unintuitive. please add a comment why this is designed this way.

This is the standard AsyncIterable/AsyncIterator two-level factory pattern: [Symbol.asyncIterator]() returns the iterator object (with next() and return() methods) that for await consumes. Added a structural comment explaining this in d39bad8.

prathikr · 2026-03-24T21:25:48Z

sdk/js/src/openai/chatClient.ts

-        const tools = Array.isArray(toolsOrCallback) ? toolsOrCallback : undefined;
-        const callback = (Array.isArray(toolsOrCallback) ? maybeCallback : toolsOrCallback) as ((chunk: any) => void) | undefined;
-
+    public completeStreamingChat(messages: any[]): AsyncIterable<any>;


same general comments as in audioClient

Applied the same comments to chatClient.ts in d39bad8.

baijumeswani · 2026-03-24T22:12:26Z

@copilot please address the following feedback from @prathikr's pr comments.

@prathikr

…tern, and iterator structure Address @prathikr review feedback: - Explain why coreInterop/modelId are captured to local variables (this is not available inside [Symbol.asyncIterator]() on the returned object) - Explain the AsyncIterable/AsyncIterator factory pattern with structural comments - Add comments on .then() handler explaining the promise-resolve wake-up pattern - Apply same comments to both audioClient.ts and chatClient.ts Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com> Agent-Logs-Url: https://github.com/microsoft/Foundry-Local/sessions/b62befe1-e407-4668-82a1-a2a22267eed9

Copilot · 2026-03-24T22:16:04Z

@copilot please address the following feedback from @prathikr's pr comments.

Addressed all 5 comments from @prathikr in d39bad8:

Local variable captures (lines 113-114) — added comment explaining this is not available inside [Symbol.asyncIterator]() since it's a regular method on the returned object literal.
r() call (line 160) — added comment explaining it resolves the pending next() promise to wake it up when the stream ends.
Nested return (line 178) — added structural comment explaining the AsyncIterable/AsyncIterator two-level factory pattern.
chatClient.ts — same comments applied.

Copilot AI and others added 2 commits March 23, 2026 22:14

Add clarifying comments about thread safety in async iterator pattern

e9aa7f4

Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com> Agent-Logs-Url: https://github.com/microsoft/Foundry-Local/sessions/f8227df8-7016-4fe1-9884-56bbdcf326b7

Copilot AI assigned Copilot and baijumeswani Mar 23, 2026

Copilot created this pull request from a session on behalf of baijumeswani March 23, 2026 22:22 View session

Copilot started work on behalf of baijumeswani March 23, 2026 23:48 View session

Copilot AI requested a review from baijumeswani March 23, 2026 23:50

Copilot finished work on behalf of baijumeswani March 23, 2026 23:50

baijumeswani marked this pull request as ready for review March 24, 2026 00:11

Copilot AI review requested due to automatic review settings March 24, 2026 00:11

Copilot started reviewing on behalf of baijumeswani March 24, 2026 00:12 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

Copilot started work on behalf of baijumeswani March 24, 2026 00:26 View session

vercel bot deployed to Preview March 24, 2026 00:30 View deployment

Copilot finished work on behalf of baijumeswani March 24, 2026 00:30

baijumeswani requested a review from Copilot March 24, 2026 04:33

Copilot started reviewing on behalf of baijumeswani March 24, 2026 04:34 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

Copilot started work on behalf of baijumeswani March 24, 2026 05:22 View session

vercel bot deployed to Preview March 24, 2026 05:27 View deployment

Copilot finished work on behalf of baijumeswani March 24, 2026 05:27

baijumeswani approved these changes Mar 24, 2026

View reviewed changes

prathikr reviewed Mar 24, 2026

View reviewed changes

Copilot started work on behalf of baijumeswani March 24, 2026 22:12 View session

vercel bot deployed to Preview March 24, 2026 22:16 View deployment

Copilot AI requested a review from baijumeswani March 24, 2026 22:18

Copilot finished work on behalf of baijumeswani March 24, 2026 22:18

prathikr approved these changes Mar 24, 2026

View reviewed changes

Conversation

Copilot AI commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

baijumeswani commented Mar 23, 2026

Uh oh!

Copilot AI commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

baijumeswani commented Mar 24, 2026

Uh oh!

Copilot AI commented Mar 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

baijumeswani commented Mar 24, 2026

Uh oh!

Copilot AI commented Mar 24, 2026

Uh oh!

prathikr Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

prathikr Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

prathikr Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

prathikr Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

prathikr Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

baijumeswani commented Mar 24, 2026

Uh oh!

Copilot AI commented Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Copilot AI commented Mar 23, 2026 •

edited

Loading

vercel bot commented Mar 23, 2026 •

edited

Loading

prathikr Mar 24, 2026 •

edited

Loading