Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

连接使用LMStudio启动的Server时,无法正常使用 #175

Closed
nosilence1994 opened this issue May 3, 2024 · 2 comments
Closed

连接使用LMStudio启动的Server时,无法正常使用 #175

nosilence1994 opened this issue May 3, 2024 · 2 comments

Comments

@nosilence1994
Copy link

Describe the bug
使用AutoDev连接LMStudio启动的本地Server时,无法正常使用。

To Reproduce
Steps to reproduce the behavior:

  1. 在LMStudio加载模型,开始本地Server
  2. 在Intellij IDEA里使用AutoDev插件连接本地的Server

Expected behavior
AudoDev插件连接LMStudio创建的Server时,能够正常工作。

Desktop (please complete the following information):

  • IDEA Version:#IC-241.14494.240
  • AutoDev Version:1.8.2-RC-241
  • LMStudio Version:0.2.22

Additional context
AutoDev报错log:

Cannot construct instance of com.theokanning.openai.OpenAiError$OpenAiErrorDetails (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value (''messages' array must only contain objects with a 'content' field that is not empty.') at [Source: (String)"{"error":"'messages' array must only contain objects with a 'content' field that is not empty."}"; line: 1, column: 10] (through reference chain: com.theokanning.openai.OpenAiError["error"])

LMStudio Server的log:

[2024-05-04 00:55:31.319] [INFO] [LM STUDIO SERVER] Processing queued request...
[2024-05-04 00:55:31.319] [INFO] Received POST request to /v1/chat/completions with body: {
  "model": "Qwen/CodeQwen1.5-7B-Chat-GGUF",
  "messages": [
    {
      "role": "system",
      "content": ""
    },
    {
      "role": "user",
      "content": "hi"
    }
  ],
  "temperature": 0,
  "stream": true
}
[2024-05-04 00:55:31.320] [ERROR] [Server Error] {"title":"'messages' array must only contain objects with a 'content' field that is not empty"}

经测试,是{"role": "system","content": ""}中,content内容为空引起的报错。
使用LMStudio自带的示例,不报错。

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ 
    "model": "Qwen/CodeQwen1.5-7B-Chat-GGUF",
    "messages": [ 
      { "role": "system", "content": "Always answer in rhymes." },
      { "role": "user", "content": "Introduce yourself." }
    ], 
    "temperature": 0.7, 
    "max_tokens": -1,
    "stream": true
}'

将某个content删除后,报错。

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{ 
    "model": "Qwen/CodeQwen1.5-7B-Chat-GGUF",
    "messages": [ 
      { "role": "system", "content": "" },
      { "role": "user", "content": "Introduce yourself." }
    ], 
    "temperature": 0.7, 
    "max_tokens": -1,
    "stream": true
}'

请问是否能够自定义role为system的content?例如CodeGPT这个插件默认的内容是

    {
      "role": "system",
      "content": "You are an AI programming assistant.\nFollow the user's requirements carefully & to the letter.\nYour responses should be informative and logical.\nYou should always adhere to technical information.\nIf the user asks for code or technical questions, you must provide code suggestions and adhere to technical information.\nIf the question is related to a developer, you must respond with content related to a developer.\nFirst think step-by-step - describe your plan for what to build in pseudocode, written out in great detail.\nThen output the code in a single code block.\nMinimize any other prose.\nKeep your answers short and impersonal.\nUse Markdown formatting in your answers.\nMake sure to include the programming language name at the start of the Markdown code blocks.\nAvoid wrapping the whole response in triple backticks.\nThe user works in an IDE built by JetBrains which has a concept for editors with open files, integrated unit test support, and output pane that shows the output of running the code as well as an integrated terminal.\nYou can only give one reply for each conversation turn."
    },

CodeGPT+LMStudio Server能够正常工作。

@phodal
Copy link
Member

phodal commented May 4, 2024

我再确定一下,问题是:不支持 role 是 system 的 message?

@nosilence1994
Copy link
Author

我再确定一下,问题是:不支持 role 是 system 的 message?

实测下来,是LMStudio开启的Local Interface Service不支持content的内容为空,无论是什么角色。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants