feat: add support for copilot models

[nhs000]

solve #2227

Wait for BerriAI/litellm#8577 to be merged

[CLAassistant]

All committers have signed the CLA.

[beetleb]

I think for the claude models, we should keep haiku as the weak models. Having sonnet-3.5 as the weak model for sonnet-3.7 is expensive and overkill.

[nhs000]

@paul-gauthier Since the changes in litellm side were merged at BerriAI/litellm#8577, could you have a look at this to see if I'm missing anythings?
I think the remaining is to bump litellm version once it is merged to the main branch and officially released.
Also I saw some warnings about missing the model information, I guess we need to fix it in litellm side as well?

[beetleb]

I think for the claude models, we should keep haiku as the weak models. Having sonnet-3.5 as the weak model for sonnet-3.7 is expensive and overkill.

@nhs000

Please change the weak models to haiku. sonnet-3.5 costs the same as sonnet-3.7 - there's no point in making 3.5 the weaker model.

[necto]

Please change the weak models to haiku. sonnet-3.5 costs the same as sonnet-3.7 - there's no point in making 3.5 the weaker model.

Sorry to intrude here, are the sonnet costs relevant when used via GH Copilot? Isn't it a flat fee regardless of the model used?

[beetleb]

Please change the weak models to haiku. sonnet-3.5 costs the same as sonnet-3.7 - there's no point in making 3.5 the weaker model.

Sorry to intrude here, are the sonnet costs relevant when used via GH Copilot? Isn't it a flat fee regardless of the model used?

Fair point. One could even argue that the weak model could be the same as the strong model in that case - why have them be different?

My concern is twofold.

The first is simple: Using such a heavy model is overkill for the weak model.

The second is that (some) people who have used proxies to reroute Aider to copilot have gotten warning emails from Microsoft. It's not clear what triggers it - perhaps too much /architect use. I think that lowering the overall burden on their servers would make it less likely to be banned from Copilot. As such, using a much "lighter" model for the weak model makes sense.

[TheBestPessimist]

why is the number of tokens so small? As far as i see both 3.7 and 3.5 accept 200k context: https://docs.anthropic.com/en/docs/about-claude/models/all-models.

do I misunderstand how max_tokens works?

[bnn1]

@TheBestPessimist github has limits on input/output tokens. i think input is limited to ~80k, and output to ~8k

[TheBestPessimist]

Interesting, thanks for mentioning. In that case, does max_tokens represent the output value? i'd have expected that to be the max input size, so it's missing at least a 0: 8192 -> 81920

[bnn1]

the max_token parameter is equal to the max_output_tokens parameter everywhere in the aider docs website so i'd assume it's not the total amount of tokens

[bnn1]

I think for the claude models, we should keep haiku as the weak models. Having sonnet-3.5 as the weak model for sonnet-3.7 is expensive and overkill.

copilot doesn't provide haiku model. the list of copilot models:

- gpt-3.5-turbo
- gpt-3.5-turbo-0613
- gpt-4o-mini
- gpt-4o-mini-2024-07-18
- gpt-4
- gpt-4-0613
- gpt-4o
- gpt-4o-2024-05-13
- gpt-4-o-preview
- gpt-4o-2024-08-06
- gpt-4o-2024-11-20
- text-embedding-ada-002
- text-embedding-3-small
- text-embedding-3-small-inference
- o1
- o1-2024-12-17
- o3-mini
- o3-mini-2025-01-31
- o3-mini-paygo
- claude-3.5-sonnet
- claude-3.7-sonnet
- claude-3.7-sonnet-thought
- gemini-2.0-flash-001

the weak model should be set to either 4o-mini or gemini-2.0-flash-001

[paul-gauthier]

It would be nice if you could add entries for the model-metadata.json file too.

[paul-gauthier]

I merged this on my local to resolve conflicts. Thanks!

Users will have a better experience if you are able to add the metadata json for these models too.