Bug: SSE Stream Crashes on Empty Events (retry directives, comment lines)

[c4milo]

Bug Report: SSE Stream Crashes on Empty Events (retry directives, comment lines)

Description

The SDK's ssestream package crashes with unexpected end of JSON input errors when processing Server-Sent Events (SSE) streams that contain empty events, which are commonly generated by:

• retry: directives (used for connection management)
• Comment lines (: comment - used for keep-alive)

This breaks compatibility with any SSE server following standard practices for connection reliability.

Environment

• SDK Version: v3.7.0
• Go Version: 1.22+
• Affected Package: packages/ssestream/ssestream.go

Steps to Reproduce

1. Create a streaming request through a proxy/gateway that adds SSE retry directives
2. The proxy returns a stream like:

retry: 3000

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"choices":[...]}

data: [DONE]

3. The SDK crashes with: Stream error: unexpected end of JSON input

Expected Behavior

The SDK should gracefully handle empty SSE events, which are valid per the W3C SSE specification. Empty events should be skipped, allowing the stream to continue processing data events.

Actual Behavior

Stream.Next() attempts to unmarshal empty event data as JSON, causing a crash:

Stream error: unexpected end of JSON input

This happens because:

1. The eventStreamDecoder creates events with empty Data when it encounters lines like retry: 3000 followed by an empty line
2. Per SSE spec, empty lines dispatch events regardless of whether data was present
3. Stream.Next() calls json.Unmarshal() on the empty byte slice
4. JSON unmarshaling fails on empty input

Root Cause

From the SSE specification:

"If the line is empty (a blank line) Dispatch the event, as defined below."

And for empty data handling:

"If the data buffer is an empty string, set the data buffer and the event type buffer to the empty string and return."

The SDK doesn't check for empty data before attempting JSON unmarshaling in packages/ssestream/ssestream.go.

Impact

This affects:

• ✅ AI Gateways & Proxies: Any middleware that adds SSE retry directives for connection management (standard practice)
• ✅ Load Balancers: Systems that inject keep-alive comments
• ✅ Production Systems: Applications following SSE best practices for reconnection handling
• ✅ Multi-Provider SDKs: Tools that proxy between different AI providers (OpenAI, Anthropic, etc.)

Real-World Example

When using the SDK through an AI gateway that routes to multiple providers:

Before Fix:

Stream error: unexpected end of JSON input
[Stream terminates prematurely]

After Fix:

Successfully receives and processes all 12 streaming chunks
Tool calls properly extracted from delta chunks
Stream completes gracefully

Proposed Solution

Check if event.Data is empty before attempting JSON unmarshaling:

// Skip events with empty data (e.g., from SSE retry: or comment lines)
if len(s.decoder.Event().Data) == 0 {
    continue
}

This:

• ✅ Maintains full compatibility with OpenAI API
• ✅ Adds resilience to spec-compliant SSE streams
• ✅ No breaking changes - only handles edge case
• ✅ Follows SSE specification correctly

Additional Context

• PR with Fix: fix(ssestream): skip events with empty data to prevent JSON unmarshal… #555 (includes comprehensive tests)
• Related Issues: This may be related to streaming issues mentioned in openrouter returns an error directly when using Stream mode #227
• SSE Spec Reference: https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events
• Common Practice: Major SSE libraries in other languages (JavaScript EventSource, Python sseclient, etc.) all handle empty events gracefully

Test Coverage

The fix includes three test cases:

1. TestStream_EmptyEvents: Verifies handling of retry directive with empty event
2. TestStream_OnlyRetryDirective: Tests stream with only retry (no data)
3. TestStream_MultipleEmptyEvents: Tests multiple empty events interspersed with data

All tests pass with the fix applied.

Workaround (Temporary)

Until this is fixed, users can:

1. Strip retry: directives in their proxy/gateway before forwarding to the SDK
2. Use a forked version with the fix applied
3. Avoid using SSE retry directives (not recommended for production)

Note on Code Generation

Since this code is generated by Stainless from OpenAPI specs, the fix may need to be applied in the generator configuration or maintained as a persistent patch per your CONTRIBUTING.md guidance:

"Modifications to code will be persisted between generations, but may result in merge conflicts between manual patches and changes from the generator."

---

Happy to provide additional context or testing if needed.