流响应接收的chunk与postman接收到的不一样

[ddxl123]

Package

dio

Version

5.8.0+1

Operating-System

Android

Adapter

Default Dio

Output of flutter doctor -v

Flutter assets will be downloaded from https://storage.flutter-io.cn. Make sure you trust this source!
[√] Flutter (Channel stable, 3.27.1, on Microsoft Windows [版本 10.0.26100.3194], locale zh-CN)
    • Flutter version 3.27.1 on channel stable at D:\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 17025dd882 (2 months ago), 2024-12-17 03:23:09 +0900
    • Engine revision cb4b5fff73
    • Dart version 3.6.0
    • DevTools version 2.40.2
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
    • Android SDK at D:\AndroidSdk
    • Platform android-35, build-tools 35.0.0
    • Java binary at: D:\AndroidStudio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 21.0.3+-12282718-b509.11)
    • All Android licenses accepted.

[X] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.12.3)
    • Visual Studio at C:\Program Files\Microsoft Visual Studio\2022\Community
    • Visual Studio Community 2022 version 17.12.35527.113
    • Windows 10 SDK version 10.0.22621.0

[√] Android Studio (version 2024.2)
    • Android Studio at D:\AndroidStudio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 21.0.3+-12282718-b509.11)

[√] IntelliJ IDEA Ultimate Edition (version 2023.3)
    • IntelliJ at C:\Program Files\JetBrains\IntelliJ IDEA 2023.3.4
    • Flutter plugin version 82.0.1
    • Dart plugin version 233.15325.11

[√] Connected device (3 available)
    • sdk gphone64 x86 64 (mobile) • emulator-5554 • android-x64    • Android 12 (API 31) (emulator)
    • Windows (desktop)            • windows       • windows-x64    • Microsoft Windows [版本 10.0.26100.3194]
    • Edge (web)                   • edge          • web-javascript • Microsoft Edge 133.0.3065.82

[√] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

Dart Version

3.6.0

Steps to Reproduce

我使用阿里云百炼的Deepseek V3 api，使用dio进行流式接收。
dio代码：
注意 Bearer xxx 改成自己的

void test() async {
  Dio dio = Dio();
  final response = await dio.request(
    "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
    options: Options(
      method: "POST",
      responseType: ResponseType.stream,
      headers: {
        "Content-Type": "application/json",
        "Authorization": "Bearer xxx", // 改成自己的
      },
    ),
    data: {
      "stream": true,
      "model": "deepseek-v3",
      "messages": [
        {"role": "user", "content": "您好"}
      ],
    },
  );
  final stream = (response.data as ResponseBody).stream;

  await for (List<int> chunk in stream) {
    String chunkString = utf8.decode(chunk);
    print("---chunk start---");
    print(chunkString);
    print("---end---");
  }
}

输出形式：

I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"","role":"assistant"},"index":0,"logprobs":null,"finish_reason":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): data: {"choices":[{"finish_reason":null,"delta":{"content":"您好"},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"！"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"很高兴"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"见到您。有什么"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"我可以帮助您的吗"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"？无论是关于某个"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"话题的疑问，"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"还是需要建议，"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"我都在这里为您"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"delta":{"content":"提供帮助。😊"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): ---end---
I/flutter (11467): ---chunk start---
I/flutter (11467): data: {"choices":[{"finish_reason":"stop","delta":{"content":""},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1740670403,"system_fingerprint":null,"model":"deepseek-v3","id":"chatcmpl-53618005-a643-936a-ade6-aa56232c2468"}
I/flutter (11467): 
I/flutter (11467): data: [DONE]
I/flutter (11467): 
I/flutter (11467): ---end---

Expected Result

以postman为准

Actual Result

两个问题：

1. 可以发现第一个chuck接收的数据是两个“data:”，而在postman中的这个chuck是正常分成两个chuck的，不知道到底是不是dio的bug。
   2.如何才能直接接收chuck内数据，而不带“data:”前缀？

附图：
dio

postman

[AlexV525]

请提供最小可运行复现代码，而不是项目里的部分代码。

[ddxl123]

请提供最小可运行复现代码，而不是项目里的部分代码。

已重新编辑

[ddxl123]

@AlexV525
我问了AI，看是不是这个原因：

====================================================

针对您的问题，以下是解决方案：

问题1：Chunk合并问题

这是由于 SSE 协议的数据帧分割方式决定的，Dio 的流式接收是按 TCP 包粒度返回的（不能保证和 SSE 的数据帧完全对应），需要手动处理数据分割。解决方案：

StringBuffer buffer = StringBuffer(); // 新增缓冲区

await for (List<int> chunk in stream) {
  String chunkString = utf8.decode(chunk);
  buffer.write(chunkString);
  
  // 按 SSE 协议分割数据帧（两个换行符）
  while (true) {
    int splitIndex = buffer.toString().indexOf('\n\n');
    if (splitIndex == -1) break;

    String frame = buffer.toString().substring(0, splitIndex);
    buffer = StringBuffer(buffer.toString().substring(splitIndex + 2));
    
    // 处理单个数据帧
    processFrame(frame);
  }
}

void processFrame(String frame) {
  if (frame.isEmpty) return;
  print("---frame start---");
  print(frame);
  print("---frame end---");
}

问题2：去除 data: 前缀

在分割出单个数据帧后，可以这样处理：

void processFrame(String frame) {
  if (frame.startsWith('data: ')) {
    String jsonStr = frame.substring(6); // 去掉 data: 前缀
    if (jsonStr.trim() == '[DONE]') return;
    
    try {
      Map<String, dynamic> json = jsonDecode(jsonStr);
      String content = json['choices'][0]['delta']['content'] ?? '';
      print('收到内容: $content');
    } catch (e) {
      print('JSON 解析失败: $e');
    }
  }
}

完整优化代码

void test() async {
  Dio dio = Dio();
  final response = await dio.request(
    "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
    options: Options(
      method: "POST",
      responseType: ResponseType.stream,
      headers: {
        "Content-Type": "application/json",
        "Authorization": "Bearer xxx",
      },
    ),
    data: {
      "stream": true,
      "model": "deepseek-v3",
      "messages": [
        {"role": "user", "content": "您好"}
      ],
    },
  );

  final stream = (response.data as ResponseBody).stream;
  StringBuffer buffer = StringBuffer();

  await for (List<int> chunk in stream) {
    String chunkString = utf8.decode(chunk);
    buffer.write(chunkString);

    while (true) {
      int splitIndex = buffer.toString().indexOf('\n\n');
      if (splitIndex == -1) break;

      String frame = buffer.toString().substring(0, splitIndex);
      buffer = StringBuffer(buffer.toString().substring(splitIndex + 2));

      if (frame.startsWith('data: ')) {
        String jsonStr = frame.substring(6);
        if (jsonStr.trim() == '[DONE]') continue;

        try {
          Map<String, dynamic> json = jsonDecode(jsonStr);
          String content = json['choices'][0]['delta']['content'] ?? '';
          if (content.isNotEmpty) {
            print('实时更新: $content');
          }
        } catch (e) {
          print('解析异常: $e | 原始数据: $jsonStr');
        }
      }
    }
  }
}

关键点说明

1. 数据帧分割：通过 indexOf('\n\n') 严格按 SSE 协议分割数据帧
2. 前缀处理：使用 substring(6) 跳过 "data: " 前缀
3. 容错处理：添加 try-catch 防止不完整 JSON 导致的崩溃
4. DONE 处理：过滤服务器最后的结束标识

这个方案可以保证：

• 正确处理多个数据帧合并到一个 TCP 包的情况
• 正确处理单个数据帧被分割到多个 TCP 包的情况
• 实时输出 AI 回复内容
• 自动过滤心跳包等无效数据

Postman 和代码表现差异的原因是：Postman 的 SSE 实现会自动处理数据帧分割，而 Dio 的流式请求需要手动实现这一逻辑。

====================================================

在Flutter中使用Dio库处理流式响应时，可能会遇到与Postman等工具不同的数据处理方式。这通常是由于不同的客户端库或工具对HTTP响应流的解析和分块方式存在差异。
在服务器发送事件（Server-Sent Events, SSE）的情况下，数据通常是通过HTTP连接以流的形式发送的。每个数据块（chunk）可能包含一个或多个事件，每个事件由特定的格式组成，例如：
data: Some data\n\n
data: Some more data\n\n
这里的\n\n表示一个事件的结束。
在Dio中遇到的“data:”问题
如果你在Dio中接收到的第一个数据块包含两个“data:”前缀，可能是因为以下原因：
事件合并：Dio可能在某个时刻接收到两个事件的数据，并将它们合并为一个数据块。这通常发生在服务器快速连续发送事件，而Dio在处理时未能及时将它们分开。
解析逻辑：Dio的流式响应处理逻辑可能与Postman的处理逻辑不同。Postman可能更严格地按照HTTP分块来解析数据，而Dio可能按照其他逻辑来处理。
缓冲区大小：Dio的内部缓冲区大小可能导致了数据块的合并。如果缓冲区较大，可能会在一次读取操作中获取多个事件的数据。
解决方案
要解决这个问题，你可以尝试以下方法：
自定义解析器：创建一个自定义的解析器来处理Dio接收到的流。这样你可以控制如何读取和解析每个数据块。
Response response = await Dio().get(
  'your-sse-url',
  options: Options(
    responseType: ResponseType.stream,
  ),
);
response.data.stream.listen((dataChunk) {
  // 自定义解析逻辑
  // 例如，按 \n\n 分割事件，并处理每个事件
  String dataString = String.fromCharCodes(dataChunk);
  List<String> events = dataString.split('\n\n');
  for (var event in events) {
    if (event.startsWith('data:')) {
      // 处理事件数据
      print(event.substring(5).trim());
    }
  }
});
调整缓冲区大小：如果可能，尝试调整Dio的内部缓冲区大小，看是否能改善问题。
检查服务器端：确保服务器端发送事件的方式是正确的，并且事件之间有适当的分隔。
更新Dio库：如果你使用的Dio版本较旧，尝试更新到最新版本，看看问题是否已被修复。
通过上述方法，你应该能够更准确地处理Dio接收到的流式响应数据，确保每个事件都能被正确解析和处理。

[AlexV525]

https://techhub.iodigital.com/articles/sse