feat: VLM-MLX Support for Apple Silicon #16102
spaxxilein
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Check Existing Issues
Problem Description
I am trying to use docling with MLX-VLM support with Open-webui. I can use it just fine by CLI with
docling https://arxiv.org/pdf/2206.01062 --pipeline vlm --vlm-model smoldocling --output-format markdown
but i have no way to add the options --pipeline and --vlm-model inside Open-Webui. That way i cannot use Hardware Acceleration on Apple Silicon
Desired Solution you'd like
I want you to add the possiblity to add the VLM-Model and Pipeline that should be used when configuring the backend of Open-Webui.
https://github.com/docling-project/docling-serve/blob/main/docs/usage.md
States that i can forward the pipeline to the API while doing the API call.
Alternatives Considered
No response
Additional Context
No response
Beta Was this translation helpful? Give feedback.
All reactions