请求增加执行翻译时的completion类型，以支持增量返回翻译结果

[veraposeidon]

背景信息
我正在开发一个基于 OpenAI API 的 Bob 翻译插件。OpenAI 的 GPT 模型翻译质量不错，但其完成翻译的速度较 DeepL 等服务慢得多。然而，OpenAI API 允许使用流式传输，将翻译结果通过事件流的形式部分返回，我可以提前获取部分翻译结果。

我试图实现增量展示翻译结果，但是 Bob 提供的 translate 接口的 completion 只有两种类型：result 和 error。这两种类型都是在一次调用中执行并结束。如果等待 OpenAI API 返回完整的翻译结果再展示，等待时间将随文本长度的增长而增加，这不利于用户体验。

功能建议
我建议将 translate 接口的 completion 类型增加一种类型 update，即调用 update 时展示翻译文本，调用 result 时结束整个 translate 调用。
这样，对于可以提前获取部分翻译结果的情况，可以多次调用 update 来实现部分翻译结果的提前展示，避免用户等待时间过长。

作者可以考虑这个建议吗？谢谢！

[ripperhe]

你好，这个是可以考虑的，不过感觉对目前的改动会较大，麻烦加我 QQ 453942056 一起评估一下可行性。

[cruxim]

你好，这个是可以考虑的，不过感觉对目前的改动会较大，麻烦加我 QQ 453942056 一起评估一下可行性。

好希望能够加入这个功能，每次等chat GPT给出完整回答太漫长了，还是很坐牢（但还是很感谢，bob真的太好用了

[Rory-Z]

你好，这个是可以考虑的，不过感觉对目前的改动会较大，麻烦加我 QQ 453942056 一起评估一下可行性。

好希望能够加入这个功能，每次等chat GPT给出完整回答太漫长了，还是很坐牢（但还是很感谢，bob真的太好用了

+1

[lastIndexOf]

@ripperhe 可以问下后续有支持的计划吗？

[ripperhe]

@ripperhe 可以问下后续有支持的计划吗？

后续会支持的

[liby]

hi，我看到 1.7.0 的更新文档讲已经支持流式输出，请问近期有考虑更新 API 文档以供插件开发使用嘛

[ripperhe]

@liby 你好，后续会把能力封装下，开放给插件

[ripperhe]

各位，Bob 1.8.0 支持了这个能力，请参考这篇文档：
https://bobtranslate.com/blog/2023-05-18-180-plugin.html

[liby]

各位，Bob 1.8.0 支持了这个能力，请参考这篇文档： bobtranslate.com/blog/2023-05-18-180-feature.html#%E5%AF%B9%E4%BA%8E%E6%8F%92%E4%BB%B6%E5%BC%80%E5%8F%91%E8%80%85

halo，我看文档中并没有提到如何 parse streamHandler 的 stream.text 参数，这样就没法取出里面的 choices 字段，可以请教一下如何处理吗？

[ripperhe]

@liby

这里以 OpenAI 为例，它的流式数据返回的是类似本文底部格式的文本，你收到到 stream.text 其实就是这样的文本，最终以 data: [DONE] 结尾。

你可以进行如下操作：

• 先将收到的文本进行换行分割
• 然后判断是否为 data: 开头，并且不是 data: [DONE] 的数据，去掉 data: 头部，将后面部分的内容进行 JSON 解析
• 获取到 choices 中的 content 进行拼接

需要注意，每次可能会收到 1 个或多个 data: 数据（有时候甚至是 0 个有效数据）。

不过，以上说的都是 OpenAI 这一个接口，每个接口不同，有的 data: 后面的文本不一定是 JSON 格式，有的甚至没有 data: ，主要看服务器怎么返回的。

可以参考以下文档：

• https://platform.openai.com/docs/api-reference/chat/create
• https://github.com/openai/openai-cookbook/blob/main/apps/file-q-and-a/nextjs/src/services/openai.ts
• https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"人"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"们"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"在"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"使用"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"这"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"些"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"设置"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"时"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"犯"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"下"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"最"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"大"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"错误"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"是"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"认"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"为"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"它"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"们"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"是"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"“"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"聪"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"明"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"”"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"或"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"“"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"创"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"造"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"力"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"”"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"控"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"件"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"。"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-73oijajfoijIOJOSIDFOIJOisdfsojrvowVdVekC","object":"chat.completion.chunk","created":1681091822,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]}

data: [DONE]

[liby]

@ripperhe 多谢提供思路

我已经实现了一版，但我想我肯定没有原作者对 API 了解的深，比如我对 cancelSignal 的使用就不确定是否有用对，还有其他一些边界情况、错误处理等。可否请您在有空的时候帮忙 Review 一下呢？🙏🙏🙏

[ripperhe]

@liby 🤝🏻 好的，我今天一整天都在户外，晚上回去看看哈。

[jtsang4]

@ripperhe 请问流式请求返回的行文本会被 Bob 截断吗，我在开发 https://github.com/jtsang4/bob-plugin-claude-translator 插件，也想支持流式输出。但是在遇到长文本时总是会出现某一行的文本不完整，导致 parse 出错的情况。

Claude API 的返回不像 OpenAI 一样是只返回增量，它是持续返回全量，因此每一行的内容会越来越多，我在想 Bob 是否对行文本进行了截断，导致插件获取不到过长的行。以下是一个 Claude API 调用的示例返回：

data: {"completion": " Hi there! My", "stop": null, "stop_reason": null, "truncated": false, "log_id": "b439081bcc3", "model": "claude-v1", "exception": null}

data: {"completion": " Hi there! My name", "stop": null, "stop_reason": null, "truncated": false, "log_id": "b439081bcc3", "model": "claude-v1", "exception": null}

data: {"completion": " Hi there! My name is.", "stop": null, "stop_reason": null, "truncated": false, "log_id": "b439081bcc3", "model": "claude-v1", "exception": null}

data: {"completion": " Hi there! My name is Claude.", "stop": "\n\nHuman:", "stop_reason": "stop_sequence", "truncated": false, "log_id": "b439081bcc3", "model": "claude-v1", "exception": null}

data: [DONE]

[ripperhe]

@jtsang4 你好，麻烦把 streamHandler 每次回调的 text 打印一下，看下 Bob 每次返回的数据是什么，我这边也可以分析下

[ripperhe]

@jtsang4 我猜你这边也是数据截断的问题，可以参考下这个 issue #542 的处理方案

[jtsang4]

@jtsang4 我猜你这边也是数据截断的问题，可以参考下这个 issue #542 的处理方案

感谢，看起来是相似的问题，我明天或者周末再处理一下

[jtsang4]

@jtsang4 我猜你这边也是数据截断的问题，可以参考下这个 issue #542 的处理方案

定位到问题了，我的情况还不一样，因为服务端没有截断输出，是我的烂代码的问题……通过打日志已经解决了问题，感谢过程中提供的帮助