Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enh: Add Valves support to MLX Pipeline #137

Merged
merged 1 commit into from
Jun 30, 2024

Conversation

justinh-rahb
Copy link
Collaborator

@justinh-rahb justinh-rahb commented Jun 30, 2024

Update MLX Pipeline (v1.2)

This PR updates the MLX Pipeline to version 1.2, adding support for the following Valves:

  • MLX_MODEL: Allows specifying the MLX model to be used in the pipeline.
    • If not found locally, it will be downloaded (similar to Ollama's pull feature).
    • Reloads the MLX pipeline when changed.
  • MLX_STOP: Define stop words that will trigger the inference to end.
  • HUGGINGFACE_TOKEN: Enables authentication with the Hugging Face Hub for downloading private models.

These new Valves enhance the pipeline's flexibility and functionality, allowing for:

  • Dynamic model selection and execution.

@justinh-rahb justinh-rahb marked this pull request as draft June 30, 2024 05:14
@justinh-rahb justinh-rahb marked this pull request as ready for review June 30, 2024 05:14
@justinh-rahb justinh-rahb changed the title feat: Add Valves support to MLX Pipeline enh: Add Valves support to MLX Pipeline Jun 30, 2024
@tjbck
Copy link
Collaborator

tjbck commented Jun 30, 2024

Let's goooo 🚀

@tjbck tjbck merged commit 412ef9d into open-webui:main Jun 30, 2024
@justinh-rahb justinh-rahb deleted the mlx-pipeline-valves branch July 4, 2024 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants