Support Anthropic API /v1/messages Endpoint by LiuLi1998 · Pull Request #22627 · vllm-project/vllm

LiuLi1998 · 2025-08-11T08:20:21Z

Relate issue: #21313
This PR adds support for the Anthropic /v1/messages REST API endpoint to the vLLM FastAPI server.

Support /v1/messages API
Compatibale with all existed tool call parser in OpenAI API

gemini-code-assist

Code Review

This pull request introduces support for the Anthropic Messages API by adding a new API server, protocol definitions, and a serving layer for format conversion. The implementation is based on the existing OpenAI-compatible server. My review has identified several critical and high-severity issues, including a potential NoneType access error, incorrect Pydantic model usage that could lead to validation errors, a risk of generating duplicate tool call IDs, and another case of incorrect attribute access on a Pydantic model that would cause a runtime error. I have provided specific code suggestions to address these issues and ensure the stability and correctness of the new endpoint.

gemini-code-assist · 2025-08-11T08:21:59Z

When handler is None, messages(raw_request) is also None. Calling create_error_response on a None object will raise an AttributeError, causing an unhandled exception and a 500 server error. You should construct an ErrorResponse directly to ensure a proper error is returned. You will need to import ErrorResponse from vllm.entrypoints.openai.protocol and HTTPStatus from http.

Suggested change

return messages(raw_request).create_error_response(

message="The model does not support Chat Completions API")

return ErrorResponse(message="The model does not support Chat Completions API",

type="model_not_found",

code=HTTPStatus.NOT_FOUND.value)

gemini-code-assist · 2025-08-11T08:21:59Z

anthropic_request.tool_choice is a Pydantic model instance, not a dictionary. Accessing its attributes should be done with dot notation (e.g., .name). Using .get("name") will result in an AttributeError at runtime.

Suggested change

"name": anthropic_request.tool_choice.get("name")

"name": anthropic_request.tool_choice.name

gemini-code-assist · 2025-08-11T08:21:59Z

The id field is defined as a required field. The model_post_init method, which attempts to set a default value, is called after Pydantic's validation. If id is not provided during initialization, a ValidationError will be raised before model_post_init can execute. To correctly provide a default value for an optional field, you should use default_factory in the field definition and remove the model_post_init method.

Suggested change

id: str

id: str = Field(default_factory=lambda: f"msg_{int(time.time() * 1000)}")

gemini-code-assist · 2025-08-11T08:21:59Z

Using int(time.time()) to generate tool call IDs is not safe as it can produce duplicate IDs for tool calls created in the same second. This can lead to incorrect behavior when matching tool calls to their results. It's better to use a UUID-based approach for uniqueness. You can use random_tool_call_id from vllm.entrypoints.chat_utils for this, which needs to be imported.

Suggested change

"id": block.id or f"call_{int(time.time())}",

"id": block.id or random_tool_call_id(),

gemini-code-assist · 2025-08-11T08:22:00Z

A text AnthropicContentBlock is created even if generator.choices[0].message.content is None. This can lead to an invalid content block, as the Anthropic API requires the text field for text blocks. When serialized with exclude_none=True, this would result in an invalid content block. You should only create the text content block if there is content available.

Suggested change

content: List[AnthropicContentBlock] = [

AnthropicContentBlock(

type="text",

text=generator.choices[0].message.content

)

]

content: List[AnthropicContentBlock] = []

if generator.choices[0].message.content:

content.append(

AnthropicContentBlock(

type="text",

text=generator.choices[0].message.content))

github-actions · 2025-08-11T08:43:51Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

mgoin · 2025-08-11T19:09:07Z

Exciting! Make sure to add some unit tests before ready and I wonder if we can also include a smoke test that claude code/some other application can communicate with the API correctly

njhill · 2025-08-11T21:13:10Z

Thanks @LiuLi1998! Would you also be willing to help with ongoing support/maintenance of the API?

LiuLi1998 · 2025-08-12T02:41:43Z

Exciting! Make sure to add some unit tests before ready and I wonder if we can also include a smoke test that claude code/some other application can communicate with the API correctly

Thanks for the input! I agree — I’ll add some tests soon to make sure everything works as expected.

LiuLi1998 · 2025-08-12T02:43:32Z

Thanks @LiuLi1998! Would you also be willing to help with ongoing support/maintenance of the API?

Definitely! I’m glad to take part in the support/maintenance of the API

Signed-off-by: liuli <ll407707@alibaba-inc.com>

LiuLi1998 · 2025-08-13T09:00:30Z

Exciting! Make sure to add some unit tests before ready and I wonder if we can also include a smoke test that claude code/some other application can communicate with the API correctly

I've added initial tests.I'm not entirely sure if the current approach follows best practices or covers everything needed—would really appreciate your feedback on improvements or any other cases

Signed-off-by: liuli <ll407707@alibaba-inc.com>

mergify · 2025-08-18T05:20:45Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LiuLi1998.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

# Conflicts: # tests/utils.py

LiuLi1998 · 2025-08-20T06:37:48Z

@mgoin While adding tests, I triggered the CI and encountered the following error:
ModuleNotFoundError: No module named 'anthropic'.
Could someone advise how to add the required dependency to the project's requirements? Should I add anthropic to requirements.txt or requirements-test.txt (or another file)? Any guidance on the correct procedure would be appreciated!

Signed-off-by: liuli <ll407707@alibaba-inc.com>

mergify · 2025-08-23T03:02:17Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LiuLi1998.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: liuli <ll407707@alibaba-inc.com> Co-authored-by: liuli <ll407707@alibaba-inc.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> Co-authored-by: Michael Goin <mgoin64@gmail.com>

shoted · 2025-10-24T03:22:36Z

how to support both openai and anthropic api

tlipoca9 · 2025-10-24T07:36:33Z

how to support both openai and anthropic api

@LiuLi1998 +1, is it possible? I also want it

Signed-off-by: liuli <ll407707@alibaba-inc.com> Co-authored-by: liuli <ll407707@alibaba-inc.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> Co-authored-by: Michael Goin <mgoin64@gmail.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>

LiuLi1998 · 2025-10-27T03:09:30Z

how to support both openai and anthropic api

@LiuLi1998 +1, is it possible? I also want it

Currently, the OpenAI and Anthropic APIs are separate api servers and cannot be used at the same time.

shoted · 2025-10-28T07:03:26Z

how to support both openai and anthropic api如何同时支持 OpenAI 和 Anthropic API

@LiuLi1998 +1, is it possible? I also want it+1，有可能吗？我也想要

Currently, the OpenAI and Anthropic APIs are separate api servers and cannot be used at the same time.目前，OpenAI 和 Anthropic API 是独立的 api 服务器，不能同时使用。

Is there a plan to support them simultaneously?

youkaichao · 2025-10-29T14:53:38Z

+        See https://docs.anthropic.com/en/api/messages
+        for the API specification. This API mimics the Anthropic messages API.
+        """
+        logger.debug("Received messages request %s", request.model_dump_json())


would this be super slow? it calls request.model_dump_json() unconditionally.

this will only work when VLLM_LOGGING_LEVEL=DEBUG

youkaichao · 2025-10-29T14:57:31Z

+        return JSONResponse(content=generator.model_dump())
+
+    elif isinstance(generator, AnthropicMessagesResponse):
+        logger.debug(


similar problem, unconditional call of generator.model_dump(exclude_none=True)

youkaichao · 2025-10-29T15:09:15Z

+
+
+@router.post(
+    "/v1/messages",


it seems anthropic API only has a new /v1/messages endpoint, why not merge it with the openai server? like serving both v1/chat/completions and /v1/messages endpoints together.

I think they are two different protocols, and It's possible to merge them together for functional compatibility, but I think it could lead to semantic confusion.

I don't think Kaichao is saying to merge them into one endpoint, just hosting them side-by-side. So when you run vllm serve you get /v1/completions, /v1/chat/completions, /v1/messages, etc. I agree this would be optimal for user ease

I don't think Kaichao is saying to merge them into one endpoint, just hosting them side-by-side. So when you run vllm serve you get /v1/completions, /v1/chat/completions, /v1/messages, etc. I agree this would be optimal for user ease

I agree it's the most user-friendly solution.

bbartels · 2025-10-31T15:07:00Z

@youkaichao @mgoin @shoted Raised #27882 to add /v1/messages to openai api_server

shoted · 2025-11-04T02:15:46Z

@youkaichao @mgoin @shoted Raised #27882 to add /v1/messages to openai api_server

nice， bro

Signed-off-by: liuli <ll407707@alibaba-inc.com> Co-authored-by: liuli <ll407707@alibaba-inc.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> Co-authored-by: Michael Goin <mgoin64@gmail.com>

LiuLi1998 requested a review from aarnphm as a code owner August 11, 2025 08:20

mergify Bot added the frontend label Aug 11, 2025

gemini-code-assist Bot reviewed Aug 11, 2025

View reviewed changes

mgoin self-requested a review August 11, 2025 19:09

LiuLi1998 requested review from DarkLight1337, robertgshaw2-redhat and simon-mo as code owners August 13, 2025 08:36

Liliu1997 added 2 commits August 13, 2025 16:37

support anthropic endpoint

20f42bd

Signed-off-by: liuli <ll407707@alibaba-inc.com>

add antropic api test

82851a3

Signed-off-by: liuli <ll407707@alibaba-inc.com>

LiuLi1998 force-pushed the dev/antropic_v2 branch from 4465c0f to 82851a3 Compare August 13, 2025 08:38

add antropic unit test

34f17fc

Signed-off-by: liuli <ll407707@alibaba-inc.com>

add spdx-header

0697fe8

Signed-off-by: liuli <ll407707@alibaba-inc.com>

mgoin changed the title ~~Support Anthropic API Endponit~~ Support Anthropic API /v1/messages Endpoint Aug 13, 2025

Liliu1997 added 2 commits August 14, 2025 13:34

stream bug fix

9b3fc78

Signed-off-by: liuli <ll407707@alibaba-inc.com>

python lint fix

d5b9860

Signed-off-by: liuli <ll407707@alibaba-inc.com>

mergify Bot added the needs-rebase label Aug 18, 2025

Merge remote-tracking branch 'origin/main'

8f54993

# Conflicts: # tests/utils.py

mergify Bot removed the needs-rebase label Aug 20, 2025

some pylint fix

78b1601

Signed-off-by: liuli <ll407707@alibaba-inc.com>

mergify Bot added the ci/build label Aug 20, 2025

mergify Bot added the needs-rebase label Aug 23, 2025

youkaichao reviewed Oct 29, 2025

View reviewed changes

mgoin mentioned this pull request Nov 4, 2025

[Feature]: Support Anthropic API /v1/messages endpoint #21313

Closed

1 task

dsfaccini mentioned this pull request Jan 13, 2026

[Build]: Relax anthropic version pin from ==0.71.0 to >=0.71.0 #32288

Closed

msanft mentioned this pull request Feb 5, 2026

[Feature]: Report cached tokens in Anthropic /v1/messages API #33923

Closed

1 task

SilviuSavu mentioned this pull request Feb 7, 2026

[Anthropic API] Fix 6 protocol compliance bugs in Messages endpoint #34053

Open

3 tasks

ehfd mentioned this pull request Feb 19, 2026

/anthropic/v1/messages should be available for OpenAI schema when /v1/messages is available with same API provider envoyproxy/ai-gateway#1867

Closed

voipmonitor mentioned this pull request Feb 27, 2026

[Bugfix] Fix Anthropic API base64 image handling in Messages endpoint #35557

Merged

5 tasks

blamechris mentioned this pull request Jun 10, 2026

OpenAI-compatible provider (LM Studio, vLLM, llama.cpp server, OpenRouter) blamechris/chroxy#5420

Closed

4 tasks

SebTardif mentioned this pull request Aug 16, 2026

[Bugfix][Frontend] Abort realtime STT on disconnect; harden Anthropic tool JSON #52519

Open

3 tasks

-        return messages(raw_request).create_error_response(
-            message="The model does not support Chat Completions API")
+        return ErrorResponse(message="The model does not support Chat Completions API",
+                             type="model_not_found",
+                             code=HTTPStatus.NOT_FOUND.value)

	"name": anthropic_request.tool_choice.get("name")
	"name": anthropic_request.tool_choice.name

	id: str
	id: str = Field(default_factory=lambda: f"msg_{int(time.time() * 1000)}")

	"id": block.id or f"call_{int(time.time())}",
	"id": block.id or random_tool_call_id(),

-        content: List[AnthropicContentBlock] = [
-            AnthropicContentBlock(
-                type="text",
-                text=generator.choices[0].message.content
-            )
-        ]
+        content: List[AnthropicContentBlock] = []
+        if generator.choices[0].message.content:
+            content.append(
+                AnthropicContentBlock(
+                    type="text",
+                    text=generator.choices[0].message.content))



		@router.post(
		"/v1/messages",

Uh oh!

Uh oh!

Conversation

LiuLi1998 commented Aug 11, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 11, 2025

Uh oh!

mgoin commented Aug 11, 2025

Uh oh!

njhill commented Aug 11, 2025

Uh oh!

LiuLi1998 commented Aug 12, 2025

Uh oh!

LiuLi1998 commented Aug 12, 2025

Uh oh!

LiuLi1998 commented Aug 13, 2025

Uh oh!

mergify Bot commented Aug 18, 2025

Uh oh!

LiuLi1998 commented Aug 20, 2025

Uh oh!

mergify Bot commented Aug 23, 2025

Uh oh!

shoted commented Oct 24, 2025

Uh oh!

tlipoca9 commented Oct 24, 2025

Uh oh!

LiuLi1998 commented Oct 27, 2025

Uh oh!

shoted commented Oct 28, 2025

Uh oh!

youkaichao Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

LiuLi1998 Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

youkaichao Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

youkaichao Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

LiuLi1998 Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

mgoin Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

LiuLi1998 Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

bbartels commented Oct 31, 2025

Uh oh!

shoted commented Nov 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

LiuLi1998 commented Aug 11, 2025 •

edited by github-actions Bot

Loading