generate API response null in swaggerUI

[maxchiron]

precondision

# lmdeploy intern-chat-7b by RestfulAPI method
python3 -m lmdeploy.serve.openai.api_server ./workspace server_ip server_port --instance_num 32 --tp 1

# Enter swaggerUI
http://{server_ip}:{server_port}

issue

In generate part, response body returns null in key "text"

, while the /v1/chat/completions, /models, /embedding APIs are all fine.
btw, it is tested through ngrok public url cause the localhost is in indirectly unaccessable subnet.

[AllentDan]

FAQ 3
https://github.com/InternLM/lmdeploy/blob/main/docs/en/restful_api.md#faq

[maxchiron]

FAQ 3 https://github.com/InternLM/lmdeploy/blob/main/docs/en/restful_api.md#faq

Same issue.
But I tried the python implement of /generate API and run in cli. It returns contents instead of null.

[AllentDan]

You were running on an old session_id. Please set sequence_end = True first.

[AllentDan]

chat.py shows how a session should be started and ended.

[maxchiron]

You were running on an old session_id. Please set sequence_end = True first.

Thank you very much. It works fine. Is there any docs about sequence_start and sequence_end ? I want to know more. Thanks for the patience and speed~

[AllentDan]

I suppose the docstring on swagger UI explains something. We will provide more detailed documents or cookbooks later.

[maxchiron]

I suppose the docstring explains something. We will provide more detailed documents or cookbooks later.

I read the simple explain (Don't know is it the docstring u mentioned.)

sequence_start (bool): indicator for starting a sequence.
sequence_end (bool): indicator for ending a sequence

Is there any links or pages to explain this? Like what is the indicator and sequence mean here.
Thanks for your reply~❤

[AllentDan]

I will update it with a detailed description.