is it possible with open source models?

[quetzalsly]

is it possible with open source models?
So no api keys are needed?

[saidkaban]

Hi @quetzalsly, thanks for the suggestion!

Yes, you can now use open-source models with no API keys needed via Ollama support that was just added

How to use it

1. Install Ollama and pull a vision model:

   ollama pull llama3.2-vision

2. Install evalmedia with Ollama support:

   pip install evalmedia[ollama]

3. Use it — no API key required:

   import evalmedia
   evalmedia.set_judge("ollama")

   Or via CLI:

   evalmedia check image.png --judge ollama

You can also point to any Ollama-compatible endpoint or change the model:

evalmedia.set_judge("ollama", model="llava")

Let me know if you have any other questions!