Skip to content

AI: Generate captions using the Ollama API #5123

Closed

Description

@lastzero

As an Ollama user, I would like to generate captions directly through the Ollama service API, so that I don't need to set up a dedicated PhotoPrism Vision API service if I don't need it otherwise.

Below is an example of the required vision.yml model configuration from our Developer Guide :

Models:
- Type: labels
  Default: true
- Type: nsfw
  Default: true
- Type: face
  Default: true
- Type: caption
  Name: gemma3
  Version: latest
  Prompt: Create a caption with exactly one sentence in the active voice that describes
    the main visual content. Begin with the main subject and clear action. Avoid text
    formatting, meta-language, and filler words.
  Resolution: 720
  Options:
    Temperature: 0.1
  Service:
    # Ollama API endpoint (adjust as needed):
    Uri: http://ollama:11434/api/generate
    FileScheme: base64
    RequestFormat: ollama
    ResponseFormat: ollama
Thresholds:
  Confidence: 10

Note that the Options parameters are optional and will have no effect when running Ollama through the dedicated Vision Service.

Documentation:

Related Discussions:

Activity

  1. self-assigned this
    on Jul 21, 2025
  2. added
    aiArtificial Intelligence, Machine Learning (ML)
    configGlobal Configuration, CLI Flags, and Environment Variables
    on Jul 21, 2025
  3. added theissue type on Jul 21, 2025
  4. moved this to Preview 🐳 in Roadmap 🚀✨on Jul 21, 2025
  5. lastzero commented on Jul 21, 2025

    @lastzero
    MemberAuthor

    The request and response handling should be refactored eventually for better maintainability. Other than that, these changes seem to work, so now both the PhotoPrism Vision API and the Ollama API request and response formats are supported. An updated preview build will be available for testing soon!

  6. 16 remaining items

  7. lastzero commented on Sep 2, 2025

    @lastzero
    MemberAuthor

    Our Developer Guide now includes a comparison of Ollama vision models:

    The setup instructions have also been refined to make configuration easier.

  8. moved this from Preview 🐳 to Release 🌈 in Roadmap 🚀✨on Dec 1, 2025
  9. added
    releasedAvailable in a stable release
    and removed
    testedChanges have been tested successfully
    on Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

aiArtificial Intelligence, Machine Learning (ML)configGlobal Configuration, CLI Flags, and Environment VariablesreleasedAvailable in a stable release

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions