Skip to content

[Bug] Call the vllm interface, and the last token output cannot receive is_end, resulting in a 10-second jam #3314

Open
@chenslcool

Description

@chenslcool

Contact Information

No response

MaxKB Version

10.0.2

Problem Description

我有一个api接口,和vllm格式的返回不完全一致,我成功接入了模型,但是调用应用的时候,输出最后一个token后总要卡十秒钟才显示对话结束。接口的格式是这样的,请问这个接口是哪个字段不对呢?导致maxkb不能确认已经停止

Image

Steps to Reproduce

如上所述

The expected correct result

No response

Related log output

Additional Information

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions