Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate Ollama #91

Closed
10 tasks done
bauersimon opened this issue May 6, 2024 · 3 comments · Fixed by #95 or #96
Closed
10 tasks done

Integrate Ollama #91

bauersimon opened this issue May 6, 2024 · 3 comments · Fixed by #95 or #96
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@bauersimon
Copy link
Member

bauersimon commented May 6, 2024

We've gotten many requests for Ollama plus a PR. This would also allow us to run models locally or add new ones that are not yet released.

However, we need to do a few preparations / settle a few questions first before we can do that:

  • having multiple providers means we need a mechanic to detect them (cause i.e. people should be able to use ollama without having an openrouter API key)
    • add an Available to a provider that checks if it is available/installed/configered with an API key
  • i.e. Ollama is a provider that needs to be installed locally. should we handle the installation if one wants to use it?
    • yes cause we need it for testing in the CI
    • same behavior as symflower, i.e. check if it is installed, and install if not
  • i.e. Ollama is a provider that is running as a background service with an API endpoint... should we make sure it runs?
    • yes
  • there are some other parameters to an Ollama provider. I.e. the local IP and/or port it is running on. how can a user configure these?
    • for now assume defaults
  • the PR Ollama provider #27 crafts the API requests manually, but Ollama has experimental support for the OpenAI API that we already use with openrouter: https://github.com/ollama/ollama/blob/main/docs/openai.md so use that instead
@bauersimon bauersimon added the enhancement New feature or request label May 6, 2024
@bauersimon bauersimon added this to the v0.5.0 milestone May 6, 2024
@bauersimon bauersimon self-assigned this May 6, 2024
@bauersimon bauersimon changed the title Integrate more providers Integrate Ollama May 6, 2024
bauersimon added a commit that referenced this issue May 6, 2024
bauersimon added a commit that referenced this issue May 6, 2024
bauersimon added a commit that referenced this issue May 6, 2024
Since Ollama is optional, it is not part of the default tool
installation.

Part of #91
bauersimon added a commit that referenced this issue May 6, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it.

Part of #91
bauersimon added a commit that referenced this issue May 6, 2024
bauersimon added a commit that referenced this issue May 6, 2024
Since Ollama is optional, it is not part of the default tool
installation.

Part of #91
bauersimon added a commit that referenced this issue May 6, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it.

Part of #91
bauersimon added a commit that referenced this issue May 7, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it in the stock evaluation.

Part of #91
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
Since Ollama is optional, it is not part of the default tool
installation.

Part of #91
bauersimon added a commit that referenced this issue May 7, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it in the stock evaluation.

Part of #91
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
Since Ollama is optional, it is not part of the default tool
installation.

Part of #91
bauersimon added a commit that referenced this issue May 7, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it in the stock evaluation.

Part of #91
bauersimon added a commit that referenced this issue May 7, 2024
bauersimon added a commit that referenced this issue May 7, 2024
Since Ollama is optional, it is not part of the default tool
installation.

Part of #91
bauersimon added a commit that referenced this issue May 7, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it in the stock evaluation.

Part of #91
zimmski pushed a commit that referenced this issue May 11, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it in the stock evaluation.

Part of #91
zimmski pushed a commit that referenced this issue May 11, 2024
zimmski pushed a commit that referenced this issue May 11, 2024
zimmski pushed a commit that referenced this issue May 11, 2024
Since Ollama is optional, it is not part of the default tool
installation.

Part of #91
zimmski pushed a commit that referenced this issue May 11, 2024
Ollama has 300MB so we don't want people installing it even if they
don't use it in the stock evaluation.

Part of #91
zimmski pushed a commit that referenced this issue May 11, 2024
zimmski pushed a commit that referenced this issue May 11, 2024
zimmski pushed a commit that referenced this issue May 11, 2024
Since Ollama is optional, it is not part of the default tool installation.

Part of #91
zimmski pushed a commit that referenced this issue May 11, 2024
zimmski pushed a commit that referenced this issue May 11, 2024
zimmski pushed a commit that referenced this issue May 11, 2024
Since Ollama is optional, it is not part of the default tool installation.

Part of #91
zimmski pushed a commit that referenced this issue May 11, 2024
@zimmski
Copy link
Member

zimmski commented May 11, 2024

Moved rest of tasks to #100

@zimmski zimmski linked a pull request May 11, 2024 that will close this issue
@zimmski zimmski mentioned this issue May 16, 2024
45 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
2 participants