wrapper.ProcessStreamingResponseBody details

[victorserbu2709]

Hello.
I would like to know more about wrapper.ProcessStreamingResponseBody.
Specifically, in https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-proxy/main.go#L351 i see following example logs:

[2a90e7ed-377d-4faa-b5f5-51e41e8af07e] [onStreamingResponseBody] isLastChunk=false chunk: data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":"I"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" need"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" to"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" see"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" the"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" actual"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" list"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" of"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" files"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" and"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-a98edd08749c4948ade06dfa34578472","object":"chat.completion.chunk","created":1755701719,"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","choices":[{"index":0,"delta":{"content":" their"},"logprobs":null,"finish_reason":null}]}

It is quaranteed that onStreamingResponseBody is called with complete chunks? (I would like to know if it is possible that a data: line to contain a truncated json, because this would break for example https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-proxy/provider/claude.go#L222)

Also, it is possible to know what is the maximum time or body response size that is buffered until onStreamingResponseBody is called?

[CH3CHO]

It is quaranteed that onStreamingResponseBody is called with complete chunks?

No, it's not. In fact, the Claude provider implementation isn't correct. You can check out the logic in main.go here ( https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-proxy/main.go#L342 ). And it is recommended to use StreamingEventHandler instead of StreamingResponseBodyHandler now.