Skip to content

Support for Local Models with Ollama

Choose a tag to compare

View all tags
@jamesrochabrun jamesrochabrun released this 30 Jun 04:59
· 17 commits to main since this release
2.0
08648d7

This release adds support for SwiftOpenAI v3.3 https://github.com/jamesrochabrun/SwiftOpenAI/releases/tag/v3.3

Ollama now has built-in compatibility with the OpenAI Chat Completions API, making it possible to use more tooling and applications with Ollama locally.

Ollama

To interact with local models using Ollama OpenAI compatibility endpoints, use the following configuration setup.

1 - Download Ollama if yo don't have it installed already.
2 - Download the model you need, e.g for llama3 type in terminal:

ollama pull llama3

Once you have the model installed locally you are ready to use PolyAI!

let ollamaConfiguration: LLMConfiguration = .ollama(url: "http://localhost:11434")

More information can be found here.

Assets 2
Loading