调用api接口如何传参？

[minmie]

使用如下命令部署api服务后
CUDA_VISIBLE_DEVICES=0 llamafactory-cli api examples/inference/llama3_lora_sft.yaml

接口怎么传参？没有找到相关文档说明

[minmie]

在这里进行调用尝试，结果报错了，还是不知道该如何传参，请问有没有提供接口调用的demo

05/24/2024 17:23:39 - INFO - llamafactory.api.chat - ==== request ====
{
  "model": "string",
  "messages": [
    {
      "role": "user",
      "content": "string",
      "tool_calls": [
        {
          "id": "string",
          "type": "function",
          "function": {
            "name": "string",
            "arguments": "string"
          }
        }
      ]
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "string",
        "description": "string",
        "parameters": {}
      }
    }
  ],
  "do_sample": true,
  "temperature": 0.0,
  "top_p": 0.0,
  "n": 1,
  "max_tokens": 0,
  "stop": "string",
  "stream": false
}
INFO:     172.50.93.47:58993 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/middleware/cors.py", line 93, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/middleware/cors.py", line 148, in simple_response
    await self.app(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/chenjq/pythonWork/LLaMA-Factory/src/llamafactory/api/app.py", line 85, in create_chat_completion
    return await create_chat_completion_response(request, chat_model)
  File "/home/chenjq/pythonWork/LLaMA-Factory/src/llamafactory/api/chat.py", line 101, in create_chat_completion_response
    responses = await chat_model.achat(
  File "/home/chenjq/pythonWork/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 56, in achat
    return await self.engine.chat(messages, system, tools, image, **input_kwargs)
  File "/home/chenjq/pythonWork/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 272, in chat
    return await loop.run_in_executor(pool, self._chat, *input_args)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/chenjq/miniconda3/envs/llama-factory/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/chenjq/pythonWork/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 159, in _chat
    gen_kwargs, prompt_length = HuggingfaceEngine._process_args(
  File "/home/chenjq/pythonWork/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 96, in _process_args
    raise ValueError("Stop parameter is not supported in Huggingface engine yet.")
ValueError: Stop parameter is not supported in Huggingface engine yet.

[hiyouga]

https://zhuanlan.zhihu.com/p/695287607

[xiangru2020]

who knows how to use python-request or apenai to use this api?