Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Ollama keep_alive param to control how long models stay loaded #319

Merged
merged 1 commit into from
Feb 2, 2024

Conversation

davidmigloz
Copy link
Owner

@davidmigloz davidmigloz commented Feb 2, 2024

https://github.com/ollama/ollama/releases/tag/v0.1.23

keepAlive: How long (in minutes) to keep the model loaded in memory.

  • If set to a positive duration (e.g. 20), the model will stay loaded for the provided duration.
  • If set to a negative duration (e.g. -1), the model will stay loaded indefinitely.
  • If set to 0, the model will be unloaded immediately once finished.
  • If not set, the model will stay loaded for 5 minutes by default

@davidmigloz davidmigloz self-assigned this Feb 2, 2024
@davidmigloz davidmigloz added t:enhancement New feature or request p:ollama_dart ollama_dart package. p:langchain_ollama langchain_ollama package. labels Feb 2, 2024
@davidmigloz davidmigloz added this to the v0.5.0 milestone Feb 2, 2024
@davidmigloz davidmigloz merged commit 3b86e22 into main Feb 2, 2024
1 check passed
@davidmigloz davidmigloz deleted the keep-alive branch February 2, 2024 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p:langchain_ollama langchain_ollama package. p:ollama_dart ollama_dart package. t:enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

1 participant