Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

breaking change: adjust how models are selected #50

Open
jamesturk opened this issue Jun 13, 2023 · 0 comments
Open

breaking change: adjust how models are selected #50

jamesturk opened this issue Jun 13, 2023 · 0 comments
Labels
planned enhancement New feature or request
Milestone

Comments

@jamesturk
Copy link
Owner

probably going to introduce a breaking change in 0.6 or 0.7 to how models are selected

Now that OpenAI is releasing different versions of models (3.5-turbo has 4 current versions between the token limit and the different iterations) I think model configuration/fallback needs to change a bit.

If 3.5 parsed the results and didn't do well, the next fallback should be 4, if the token limit was exceeded however it should go to 3.5-16k. There are a lot of possible conditions for this, and people that want 100% control can explicitly pass a single model, but the way the fallback chain is traversed can improve to reduce redundant requests.

Something like:

models=[GPT35T(allow_16k=True), GPT4(allow_32k=False)]

This would try gpt-3.5-turbo only once, either at 4k or 16k based on input.
Then it would try gpt-4.
Selecting particular revisions could work this way as well.

Probably makes the most sense to do this as part of #18

@jamesturk jamesturk changed the title adjust how models are selected breaking change: adjust how models are selected Jun 13, 2023
@jamesturk jamesturk added the planned enhancement New feature or request label Jun 13, 2023
@jamesturk jamesturk added this to the 0.6.0 milestone Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
planned enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant