Add gpt-4o metadata

[ConnorDoyle]

#3612

Title

Relevant issues

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
💻 Development Environment
🚄 Infrastructure
✅ Test

Changes

Testing

Notes

Pre-Submission Checklist (optional but appreciated):

• I have included relevant documentation updates (stored in /docs/my-website)

OS Tests (optional but appreciated):

• Tested on Windows
• Tested on MacOS
• Tested on Linux

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 13, 2024 6:52pm

[lowjiansheng]

Can we add an alias for gpt-4o-2024-05-13 too?

[mmmaia]

Thanks for working on this! A couple observations:

• max_output_tokens for GPT-4o is 2048 (at least it's what I see in the playgronud)
• we should also add gpt-4o-2024-05-13

[HowieG]

How is mode handled with multimodal input?
"mode": ["chat", "audio_transcription", "image_generation"],

[ConnorDoyle]

Thanks for working on this! A couple observations:

• max_output_tokens for GPT-4o is 2048 (at least it's what I see in the playgronud)
• we should also add gpt-4o-2024-05-13

I couldn't find docs supporting either explicit API limit, and I was unable to convince the model to produce more than 2048 tokens via some manual API testing. Updated this to 2048 since the limit is at least 2048 via the API (but possibly higher.)

[mmmaia]

I couldn't find docs supporting either explicit API limit, and I was unable to convince the model to produce more than 2048 tokens via some manual API testing. Updated this to 2048 since the limit is at least 2048 via the API (but possibly higher.)

Now my playground is showing 4096. Unfortunately, there's no mention about it in the docs.

[krrishdholakia]

LGTM - merging in, we'll take care of updating the max token limit based on testing

[Undertone0809]

How is mode handled with multimodal input? "mode": ["chat", "audio_transcription", "image_generation"],

#3629 So the question is, is it because of this?

[krrishdholakia]

@Undertone0809 mode is primarily used for vertex ai for routing between their different sdk's.

We also started using it for health checks, as a way to know whether it's a /completion, /embedding or /chat/completion call

No impact on multimodal.