Skip to content

docs: add MLflow AI Gateway as LLM provider#5228

Open
PattaraS wants to merge 2 commits intocrewAIInc:mainfrom
PattaraS:add-mlflow-gateway-integration
Open

docs: add MLflow AI Gateway as LLM provider#5228
PattaraS wants to merge 2 commits intocrewAIInc:mainfrom
PattaraS:add-mlflow-gateway-integration

Conversation

@PattaraS
Copy link
Copy Markdown

@PattaraS PattaraS commented Apr 2, 2026

Summary

Adds an MLflow AI Gateway Accordion entry to docs/en/concepts/llms.mdx, showing how to use it as an OpenAI-compatible LLM backend in CrewAI.

English only — translations for ar, ko, and pt-BR are welcome as a follow-up.

Note: This PR should be labeled llm-generated per the contributing guide — we cannot add labels from a fork.

What is MLflow AI Gateway?

MLflow AI Gateway is a database-backed LLM proxy built into the MLflow tracking server (MLflow ≥ 3.0). It provides a unified OpenAI-compatible API across dozens of providers with encrypted secrets management, fallback/retry, traffic splitting, and per-endpoint token budgets.

Changes

  • Single Accordion section added to docs/en/concepts/llms.mdx with:
    • Brief description of MLflow AI Gateway
    • Setup instructions (start MLflow server, create endpoint in UI)
    • Code example using LLM(model="openai/<endpoint-name>", base_url="...")
    • Note about LiteLLM dependency

AI Disclosure

This pull request was AI-assisted by Claude. All content was reviewed and validated by a human contributor.

@PattaraS PattaraS marked this pull request as draft April 2, 2026 09:17
@PattaraS PattaraS marked this pull request as ready for review April 10, 2026 04:32
@PattaraS
Copy link
Copy Markdown
Author

@greysonlalonde Would you mind taking a look? Thanks!

Adds an Accordion entry for MLflow AI Gateway in the LLMs concept page,
showing how to use it as an OpenAI-compatible LLM backend with base_url
pointing to the gateway server.
- Add mlflow[genai] install step
- Add screenshot of the Create Endpoint UI
- Add link to MLflow endpoint configuration docs
- Number the setup steps for clarity
@PattaraS PattaraS force-pushed the add-mlflow-gateway-integration branch from ffde95e to bb2cde6 Compare April 21, 2026 04:44
@PattaraS
Copy link
Copy Markdown
Author

Hi @lorenzejay, friendly ping on this PR. We're from the MLflow team and working on adding MLflow AI Gateway integration guides across major agent frameworks. MLflow AI Gateway (MLflow >= 3.0) is an open-source LLM proxy with built-in secrets management, fallback/retry, traffic splitting, and usage tracing. This PR adds a small Accordion section to the LLMs page showing how to use it with CrewAI via base_url (English only, happy to add translations as follow-up). Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant