Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why token counts & costs are 40x? #410

Closed
lectair opened this issue Jan 30, 2024 · 3 comments
Closed

Why token counts & costs are 40x? #410

lectair opened this issue Jan 30, 2024 · 3 comments

Comments

@lectair
Copy link

lectair commented Jan 30, 2024

I send a message to gpt-4-1106-preview which is 6 tokens (in the official OpenAI tokenizer) or 19 characters, and GPT gives me a response of 71 tokens or 273 characters, and below GPT's response it says "gpt-4-1106-preview using 2919 tokens ~= $0. 030610", which is an increase of a x40 of tokens that I can't explain even adding the tokens in my message with the response. Is it a counting bug or am I losing 40 times more money unnecessarily?

Is this happening to anyone else?
Is there any way I can see the raw HTTP requests to the OpenAI endpoint?

Thanks a lot and best regards.

PS: I am using the online version (https://niek.github.io/chatgpt-web)

@Niek
Copy link
Owner

Niek commented Jan 30, 2024

Are you sure this was the initial message and there was no context sent (i.e. earlier messages in the same thread)?

@lectair
Copy link
Author

lectair commented Jan 30, 2024

No, it wasn't the original message. It's because of the settings then, right?

Thanks for the fast response.

@Niek
Copy link
Owner

Niek commented Jan 30, 2024

You probably have a huge custom system prompt. I checked again and the tokens should be counted correctly. If it's more than what you typed/the context, it's in your system prompt.

@Niek Niek closed this as completed Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants