Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new models #51

Merged
merged 1 commit into from
Jun 15, 2023
Merged

Add new models #51

merged 1 commit into from
Jun 15, 2023

Conversation

rodumani
Copy link

@rodumani rodumani commented Jun 14, 2023

Hi! I'm so glad that I can contribute to the awesome package.

I've added the following new models introduced on June 13, 2023.

  • gpt-3.5-turbo-0613
  • gpt-3.5-turbo-16k
  • gpt-3.5-turbo-16k-0613
  • gpt-4-0613
  • gpt-4-32k-0613

@zerfl
Copy link

zerfl commented Jun 14, 2023

You'll want to update the wasm bindings as well to make this work.

See zerfl@7ed984e#diff-3920f5b0c1e7a80683f05ae43cb02274ad7d418643021a88b4cf765008c79779

- gpt-3.5-turbo-0613
- gpt-3.5-turbo-16k
- gpt-3.5-turbo-16k-0613
- gpt-4-0613
- gpt-4-32k-0613
@rodumani rodumani changed the title Add new models to model_to_encoding.json Add new models Jun 14, 2023
@dqbd
Copy link
Owner

dqbd commented Jun 15, 2023

Looks fine to me! Thanks @rodumani @zerfl!

@dqbd dqbd merged commit d1f8f9f into dqbd:main Jun 15, 2023
@tractorcow
Copy link

The new models are missing from typescript definition. See

export type TiktokenModel =

in wasm/src/lib.rs

@kiejo kiejo mentioned this pull request Jun 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants