-
Notifications
You must be signed in to change notification settings - Fork 826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't install tiktoken in Python 3.12 #205
Comments
@pamelafox It works fine on Python3.12 if you have a rust compiler. If you're building it in Docker, make sure you add it to your path.
|
@JoshJarabek7 Many thanks for your input. However, if I use Python 3.11 it works like a charm. Why is it suddenly causing trouble? |
@junoriosity not sure. Could be a few things. More than likely it's because Python 3.12 removed setuptools, which helps build Python packages. You can still install it via pip if you need it if you want to try that before pip installing tiktoken. You can also try installing via binary in pip instead of downloading setuptools. Another reason could be that tiktoken is written in Rust (not sure if it was before or not) for some speed gains solely for Python 3.12. |
@JoshJarabek7 Essentially, my Dockerfile looks like this
and I get the error
Perhaps you have an idea how to overcome this. 😊 |
@JoshJarabek7 Thanks for the suggestion! We are using it an open source sample for developers on many systems (Mac/Linux/Windows) so we can't assume they'll have a Rust compiler. We eagerly await formal support via a built wheel. |
tiktoken 0.5.2 ships Python 3.12 wheels :-) |
We are trying to install tiktoken in Python 3.12, but get an error:
Can 3.12 wheels be released?
The text was updated successfully, but these errors were encountered: