Support Anthropic Claude 3

[enricoros]

Looking for PR to add support for Claude 3, released today by Anthropic.

[jacksongoode]

#444

[IdkwhatImD0ing]

This pr does not seem to work for me. Error message stating it requires the new messages api.

[jacksongoode]

Ah there was an API change as well?

[IdkwhatImD0ing]

It seems like it, other claude models work, just not Claude 3

[Penagwin]

https://docs.anthropic.com/claude/docs/models-overview#model-comparison

All Claude 3 models only support the Messages API and I think currently only the completions API is implemented for Claude here?

[enricoros]

Correct @Penagwin - they just introduced the new Messages API and the Claude 3 models are the ones to only support that new API. We'll need to update the backend code.

[IdkwhatImD0ing]

Is this the backend code for claude: https://github.com/enricoros/big-AGI/blob/main/src/modules/llms/server/anthropic/anthropic.router.ts

Might be interested in taking a crack at this

[slapglif]

Hey everyone! 👋

I've taken a crack at adding support for Anthropic Claude 3 models, as mentioned by @enricoros and @Penagwin. The changes primarily focus on updating the backend code in anthropic.router.ts to use the new Anthropic Messages API.

Here's a quick rundown of what I've done:

🔧 Updated the backend code to use the new Messages API, as required by Claude 3 models.
🆕 Implemented support for Claude 3 models, so users can now leverage these powerful new tools.
📝 Updated the documentation and comments to reflect the changes and provide clarity on usage.
🧪 Added tests to ensure the Claude 3 functionality works as expected and to maintain code quality.
I've submitted a pull request with all the details, and I'm excited to hear your thoughts and feedback! Let's work together to get this merged and provide our users with access to the latest and greatest from Anthropic.

Thanks for your time, and let me know if you have any questions or suggestions! 😄

#450

[enricoros]

Thanks @slapglif - looking into it. Seems like there's no Streaming support; looking into that.

[slapglif]

@enricoros
no problem, been using this software since the day i hit you up, least i can do - i replied to the pr:

@enricoros as to the fidelity of the fix, upon further inspection, i receive some cases with:

 ○ Compiling /api/trpc-node/[trpc] ...
 ✓ Compiled /api/trpc-node/[trpc] in 1037ms (700 modules)
Error in stream: [SyntaxError: Expected ',' or '}' after property value in JSON at position 39]

this is likley a parsing error, ill see to it shortly, as for the lack of streaming support - the current situation is that streaming is supported and streams to console, but i wanted to push a working version before attacking the streaming to next bug,

to address the formatting - sorry yeah python guy, can you define your standard for me so i can use an llm to reformat?

#450 (comment)