Merged
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds Layer 3 streaming primitives: an SSE byte→line reader ( Changes
Sequence Diagram(s)sequenceDiagram
participant ByteStream as Provider Byte Stream
participant SSEReader as sse_reader
participant HubChunk as HubChunkStream
participant Transform as ProviderCapabilities
participant State as ChatStreamState
ByteStream->>SSEReader: bytes (chunks)
SSEReader->>SSEReader: buffer & split at newlines
SSEReader->>HubChunk: complete SSE lines (String)
HubChunk->>HubChunk: drain buffered ChatCompletionChunk(s) first
HubChunk->>Transform: transform_stream_chunk(raw_line)
Transform->>HubChunk: Vec<ChatCompletionChunk>
HubChunk->>HubChunk: enqueue additional chunks (VecDeque)
HubChunk->>State: update usage (prompt_tokens, completion_tokens) & chunk_index
HubChunk->>Client: return first ChatCompletionChunk
Note over HubChunk,State: subsequent polls emit queued chunks before polling inner stream
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Pull request overview
Adds a new gateway::streams core module to standardize provider streaming: decoding raw SSE byte streams into lines and converting provider stream lines into hub ChatCompletionChunk items with centralized state/usage tracking.
Changes:
- Introduce
sse_readerto split areqwestbyte stream into SSE lines and flush trailing data at EOF. - Introduce
HubChunkStreamto buffer multi-chunk provider transforms, enforce correct polling order, and accumulate usage intoChatStreamState. - Add internal documentation describing the stream core building blocks and intended scope.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/gateway/streams/reader.rs | Adds SSE byte-stream → line reader plus unit test. |
| src/gateway/streams/hub.rs | Adds line-stream → ChatCompletionChunk adapter with buffering/state updates plus tests. |
| src/gateway/streams/mod.rs | Wires the new streams submodules and re-exports key types. |
| src/gateway/mod.rs | Exposes the new streams module from gateway. |
| docs/internals/llm-streams.md | Documents the new stream-core components and behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/gateway/streams/hub.rs`:
- Around line 17-38: Add Rustdoc comments to the public HubChunkStream type and
its constructor new: document that HubChunkStream buffers ChatCompletionChunk
items in buffer (VecDeque) to preserve ordering across the inner Stream<String>
input, describe the semantics of state mutations via the state field
(ChatStreamState) and any side effects consumers should expect, and note
thread/send guarantees (Send) and that def: Arc<dyn ProviderCapabilities> is
consulted for provider-specific behavior; place these /// comments immediately
above the pub struct HubChunkStream declaration and above impl
HubChunkStream::new so generated API docs clearly state buffering, ordering
guarantees, and state side effects.
In `@src/gateway/streams/reader.rs`:
- Around line 8-10: Add a /// doc comment to the public function sse_reader
describing its contract: explain that sse_reader accepts a byte stream and
yields UTF-8 decoded Strings split on newline boundaries, that it filters out
empty lines (skipping zero-length messages), and that it flushes any partial
buffered line as a final item on EOF; reference the function name sse_reader and
mention the stream input type and the returned Stream<Item = Result<String>> so
callers know the expected behavior and error handling.
- Around line 12-37: The scan closure currently flushes buffered partial data
after a transport Err because the Err arm returns Some(stream) and the earlier
synthetic newline can cause emission of truncated data; change the Err arm in
the scan over buffer to clear the buffer (buffer.clear()) and return
futures::future::ready(None) so the scan terminates and no further items are
emitted after the first error, and add a regression test that exercises the
sequence Ok(partial) -> Err(_) -> EOF to verify no partial line is emitted.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a704503e-44b0-426d-bd2b-167a98ec6439
📒 Files selected for processing (5)
docs/internals/llm-streams.mdsrc/gateway/mod.rssrc/gateway/streams/hub.rssrc/gateway/streams/mod.rssrc/gateway/streams/reader.rs
This was referenced Apr 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As the 8th part of the major refactoring of the provider, add the core components of the stream module.
Summary by CodeRabbit
New Features
Documentation
Tests