[Feature]: DAY 1 SUPPORT Add Claude -3

[ishaan-jaff]

The Feature

Motivation, pitch

New model

Twitter / LinkedIn details

No response

[ishaan-jaff]

We'll need to add the messages endpoint for claude3 - it fails on their completion endpoint

Error calling model: AnthropicException - {"type":"error","error":{"type":"invalid_request_error","message":"\"claude-3-opus-20240229\" is not supported on this API. Please use the Messages API instead."}}
 

[krrishdholakia]

No bedrock support yet.

[krrishdholakia]

I don't think the legacy text completion endpoint supports claude 3. We should move to using the anthropic 2.1 endpoint - #1209

[krrishdholakia]

You can also send images in claude 3 - will need to add pricing for this

[krrishdholakia]

support tool calling - https://github.com/anthropics/anthropic-cookbook/blob/main/function_calling/function_calling.ipynb

[ishaan-jaff]

Action Items:

• Migrate to /messages + add claude-3 @ishaan-jaff
• Add tool calling / images @krrishdholakia

[ishaan-jaff]

PR to add initial support : #2315

[ishaan-jaff]

Model Pricing info here for reference https://docs.anthropic.com/claude/docs/models-overview#meet-claude-3

[krrishdholakia]

I'll take care of the image + tool calling cost tracking in my PR

[krrishdholakia]

[ishaan-jaff]

Notes about the /messages endpoint

• Anthropic Deprecated support for claude-instant-1 on the new messages endpoint

1. Anthropic supports roles like "user" and "assistant",
2. The first message always needs to be of role "user"
3. Each message must alternate between "user" and "assistant"

3. Each message must alternate between "user" and "assistant"

For v0 PR not addressing no.3