Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 804 #856

Merged
merged 3 commits into from Mar 28, 2024
Merged

Fix 804 #856

merged 3 commits into from Mar 28, 2024

Conversation

langchain4j
Copy link
Owner

@langchain4j langchain4j commented Mar 28, 2024

Context

See #804

Change

  • OpenAiStreamingChatModel: in case modelName is not one of the known OpenAI models, do not return TokenUsage in the Response. This is done for cases when OpenAiStreamingChatModel is used to connect to other OpenAI-API-compatible LLM providers like Ollama and Groq. In such cases it is better to not return TokenUsage then returning a wrong one.
  • For all OpenAI models, default Tokenizer will now use "gpt-3.5-turbo" model name instead of the one provided by the user in the modelName parameter. This is done to avoid crashing with "Model 'ft:gpt-3.5-turbo:my-org:custom_suffix:id' is unknown to jtokkit" for fine-tuned OpenAI models. It should be safe to use "gpt-3.5-turbo" by default with all current OpenAI models, as they all use the same cl100k_base encoding.

Checklist

Before submitting this PR, please check the following points:

  • I have added unit and integration tests for my change
  • All unit and integration tests in the module I have added/changed are green
  • All unit and integration tests in the core and main modules are green
  • I have added/updated the documentation
  • I have added an example in the examples repo (only for "big" features)
  • I have added my new module in the BOM (only when a new module is added)

@langchain4j langchain4j merged commit a2572c8 into main Mar 28, 2024
6 checks passed
@langchain4j langchain4j deleted the fix_804 branch March 28, 2024 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant