Can't connect to zhipu's api interface after updating to the latest version.

[flyfox666]

Hi guys.

The latest version of the UI revamp really wowed me, there are a lot of improvements, kudos.

After updating to the new version (v0.1.125), I found that the api interface of zhipu, which I have been using for a long time, doesn't work, but the other api interfaces are working, and I've also rolled back the test (v0.1.124), and I found that it was working under the previous version. The specific errors are as follows

Oops! Problem connecting to chatglm-GLM4-vip.

External: OpenAIException - Error code: 400 - {'error': {'code': '1214', 'message': 'messages[2]:content and tool_calls fields can't both be empty'}}

! image

The backend reports the following error:

2024-05-21 13:45:36 INFO: 14.216.93.58:0 - "POST /api/v1/auths/signup HTTP/1.1" 200 OK
2024-05-21 13:45:45 INFO: 127.0.0.1:36522 - "GET /health HTTP/1.1" 200 OK
2024-05-21 13:45:52 ERROR:apps.litellm.main:400 Client Error: Bad Request for url: http://localhost:4000/v1/chat/completions
2024-05-21 13:45:52 Traceback (most recent call last):
2024-05-21 13:45:52 File "/app/backend/apps/litellm/main.py", line 354, in proxy
2024-05-21 13:45:52 r.raise_for_status()
2024-05-21 13:45:52 File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
2024-05-21 13:45:52 raise HTTPError(http_error_msg, response=self)
2024-05-21 13:45:52 requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:4000/v1/chat/completions
2024-05-21 13:45:52 INFO: 172.17.0.1:53098 - "POST /litellm/api/v1/chat/completions HTTP/1.1" 500 Internal Server Error

I also tested litellm after installing it separately and found that litellm did not respond when the error was reported.

I also consulted zhipu's backend customer service, they replied me that there is a null value in the list of messages, and the content can't be null.

I've looked at the code but can't find where the problem lies.

Thank you very much!

[igeekbb]

gemini-Connection failed These endpoints work fine on other clients.

[flyfox666]

Hahaha. Didn't expect to see geek here.

[justinh-rahb]

As usual with any new release, we often do some fixes afterwards that push direct to main and build new Docker images. If you haven't already done so as of 3hrs ago please try re-pulling again.

[flyfox666]

HI
Just re-pull and the error still exist

[bahuzh]

+1

[flyfox666]

I tried to update the main.py under backend/apps/litellm with help of chatgpt, added the following code and now it works, is it ok to make this change or can I go ahead and mention a pull:

@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])

async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
body = await request.body()

log.info(f"Original request body: {body}")

try:
    body_json = json.loads(body)
    
    # Remove 'num_ctx' if it exists
    if "num_ctx" in body_json:
        log.info("Removing 'num_ctx' from request body.")
        del body_json["num_ctx"]
    
    # Check for messages[5]:content and tool_calls
    if "messages" in body_json and isinstance(body_json["messages"], list):
        for message in body_json["messages"]:
            if message.get("role") == "assistant" and "content" in message and not message["content"]:
                log.warning("Found 'assistant' role with empty 'content'. Setting a default non-empty content.")
                message["content"] = "Default content"  # or handle appropriately
            if "content" not in message and "tool_calls" not in message:
                log.error("messages[5]:content 和 tool_calls 字段不能同时为空")
                raise HTTPException(
                    status_code=status.HTTP_400_BAD_REQUEST,
                    detail="messages[5]:content 和 tool_calls 字段不能同时为空"
                )
    
    body = json.dumps(body_json)
    log.info(f"Modified request body: {body}")
except json.JSONDecodeError:
    log.warning("Request body is not JSON.")
    pass  # If body is not JSON, ignore it

[justinh-rahb]

This would need thorough testing to ensure it doesn't break compatibility for other APIs at the very least.

[Yanyutin753]

估计是空消息的锅

[flyfox666]

嗯，就是空消息

[hiddenblue]

found the same problem in other litellm model
#2494