Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Ollama provider #96

Merged
merged 2 commits into from
May 15, 2024
Merged

Support Ollama provider #96

merged 2 commits into from
May 15, 2024

Conversation

bauersimon
Copy link
Member

Early merger is #95.

@bauersimon bauersimon requested a review from zimmski May 7, 2024 10:22
@bauersimon bauersimon self-assigned this May 7, 2024
@bauersimon bauersimon force-pushed the ollama branch 2 times, most recently from b3da82a to 51ad917 Compare May 7, 2024 16:35
@bauersimon
Copy link
Member Author

TODO: tests pass when targeted manually but running make test fails nearly all the tests with the reason that the local Ollama server is unreachable. I guess this is because we always use the default port and the tests run in parallel, but running them with the -p 1 flag does not work either.

@bauersimon
Copy link
Member Author

bauersimon commented May 8, 2024

# === TERMINAL A ===
# Checkout and install Ollama as part of tools.
git fresh-checkout ollama
make install-all

# === TERMINAL B
# Ollama server instance which manages all models.
~/.eval-dev-quality/bin/ollama serve &

# === TERMINAL A ===
# Pull some models (check https://ollama.com/library for more but watch out, many are several GBs large).
~/.eval-dev-quality/bin/ollama pull qwen:0.5b
~/.eval-dev-quality/bin/ollama pull tinyllama:latest

# === TERMINAL B ===
# Kill the server, or don't cause our eval sould either start a new one or reuse the existing one.

# === TERMINAL A ===
# Run the eval with only the models we pulled.
eval-dev-quality evaluate --model ollama/qwen:0.5b --model ollama/tinyllama:latest

@Munsio @ahumenberger

@Munsio
Copy link
Contributor

Munsio commented May 8, 2024

I've run the bash script and it gave me the following output:
https://gist.github.com/Munsio/8dcae3ff9d58c0ccf25401287ea42619

@bauersimon bauersimon force-pushed the ollama branch 3 times, most recently from 89e9eff to 1311539 Compare May 8, 2024 10:32
@bauersimon bauersimon marked this pull request as ready for review May 8, 2024 10:36
util/exec.go Outdated Show resolved Hide resolved
util/exec.go Outdated Show resolved Hide resolved
zimmski pushed a commit that referenced this pull request May 11, 2024
zimmski pushed a commit that referenced this pull request May 11, 2024
@zimmski zimmski force-pushed the ollama branch 2 times, most recently from f603a82 to e256bda Compare May 11, 2024 19:43
@zimmski zimmski added the enhancement New feature or request label May 11, 2024
@zimmski zimmski added this to the v0.5.0 milestone May 11, 2024
@zimmski zimmski linked an issue May 11, 2024 that may be closed by this pull request
10 tasks
provider/provider.go Outdated Show resolved Hide resolved
tools/ollama.go Outdated Show resolved Hide resolved
@zimmski zimmski force-pushed the ollama branch 13 times, most recently from f44693f to cc46ef3 Compare May 15, 2024 12:20
util/exec.go Outdated Show resolved Hide resolved
cmd/eval-dev-quality/cmd/evaluate_test.go Outdated Show resolved Hide resolved
cmd/eval-dev-quality/cmd/evaluate_test.go Show resolved Hide resolved
provider/ollama/ollama_test.go Outdated Show resolved Hide resolved
tools/ollama.go Show resolved Hide resolved
tools/ollama.go Outdated Show resolved Hide resolved
tools/ollama.go Outdated Show resolved Hide resolved
tools/ollama.go Outdated Show resolved Hide resolved
tools/ollama.go Outdated Show resolved Hide resolved
util/exec.go Outdated Show resolved Hide resolved
@zimmski zimmski merged commit 6f0e539 into main May 15, 2024
3 checks passed
@zimmski zimmski deleted the ollama branch May 15, 2024 12:54
@zimmski zimmski mentioned this pull request May 16, 2024
45 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate Ollama
3 participants