Vision Service: Add Ollama and Data URL support

[sgflt]

Refactored code in a way that will support easy addition of another caption providers.

Related Issues:

• Vision Service: Use an inference server or framework instead of custom microservices #1
• Vision Service: Add support for data URLs so that no HTTPS file server is required #4

[lastzero]

@sgflt Thank you very much! 🥳

See comment in the related issue for feedback:

• Vision Service: Use an inference server or framework instead of custom microservices #1 (comment)

[lastzero]

@sgflt Awesome, that looks like a huge improvement! 🥇

How about adding some documentation about the new features and changes? 🤔

1. It looks like OLLAMA_ENABLED needs to be set to "true" in the compose.yaml configuration and prompts can optionally be customized with OLLAMA_NSFW_PROMPT, OLLAMA_CAPTION_PROMPT and/or OLLAMA_LABELS_PROMPT?
2. The model name to pass to the service probably has to match one of these when Ollama is enabled?
3. Are the previously implemented models still available or do any of the defaults change when Ollama is enabled?
4. Potential users might also be interested in the hardware requirements (depending on the endpoint and/or model) and which models are recommended, especially for Ollama?
5. For example, I'm using an Nvidia RTX 4060 with 8 GB of RAM in my Linux workstation, which generally allows only small LLMs to be loaded. Would that still be enough? There's a good chance that not all models will fit in memory at the same time, but if the same model can be reused for all endpoints, that's not a big deal.
6. Some users have also asked if it's possible to run one of these models without a GPU, just on the CPU? If so, can this be recommended or would it be too slow anyway?

To reduce the effort, it would be sufficient to add this information to the existing README.md.

It also seems like a good idea to include the (most important) variables in the compose.yaml configuration example, so that users see them even if they don't read the docs. Note that the service/compose.yaml file is for development only and not strictly needed when you run the service locally with Python.

Don't worry too much about typos, as we can edit the docs later to improve readability. If you have any questions, please feel free to write an email or get in touch via Element/Matrix! 💬

[sgflt]

1. added information into README
2. yes, added info into README
3. yes, enabling of Ollama is additive, local configuration will work without any change
4. added some notes
5. dtto 4
6. dtto 4

[lastzero]

@sgflt I don't see these changes. Maybe you haven't pushed them yet?

[lastzero]

Thank you very much! I'll go ahead and merge this if there's nothing you want to add?

[sgflt]

Okay. I have some minor cleanups as had found that python since 3.9 have better typing hints, but I will do other pull request under different issue.