通过paddlenlp启动的大模型兼容openai接口

[jiaolongxue]

Feature request

希望通过paddlenlp启动的大模型服务可以兼容openai接口

# import openai
# openai.api_key = "EMPTY" # Not support yet
# openai.api_base = "http://localhost:8888/v1"

# model = "chatglm2-6b"

# # create a chat completion
# completion = openai.ChatCompletion.create(
#   model=model,
#   messages=[{"role": "user", "content": "Hello! What is your name?"}]
# )
# # print the completion
# print(completion.choices[0].message.content)

Motivation

希望将大模型运行在paddle的推理框架上，可以兼容openai接口

Your contribution

希望将大模型运行在paddle的推理框架上，可以兼容openai接口

[w5688414]

目前没有开发的计划，欢迎开发者的贡献。

[ZHUI]

最新的flask server已经兼容。https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm#71-flask--gradio-ui-%E6%9C%8D%E5%8A%A1%E5%8C%96%E9%83%A8%E7%BD%B2