Skip to content

Commit a760f84

Browse files
hellovaihellovai
authored
[docs] improve documentation for llm clients and highlight openai-generic compatability (#1655)
<!-- ELLIPSIS_HIDDEN --> > [!IMPORTANT] > Improves LLM client documentation with OpenAI-generic compatibility details and updates navigation and redirects. > > - **Documentation Enhancements**: > - Added title and introductory section to `client-llm.mdx`. > - Updated `overview.mdx` to link to `/ref/baml/client-llm`. > - Added provider-specific notes and links in `client-constructor.mdx`. > - **Navigation and Redirects**: > - Updated `docs.yml` to include new provider pages and slugs for OpenAI-generic compatibility. > - Added redirects for OpenAI-generic providers in `docs.yml`. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=BoundaryML%2Fbaml&utm_source=github&utm_medium=referral)<sup> for 76ded17. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN -->
1 parent 30543cc commit a760f84

4 files changed

Lines changed: 80 additions & 22 deletions

File tree

‎fern/03-reference/baml/client-llm.mdx‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
---
2+
title: LLM Clients (client<llm>)
3+
---
14

25
Clients are used to configure how LLMs are called, like so:
36

@@ -10,7 +13,7 @@ function MakeHaiku(topic: string) -> string {
1013
}
1114
```
1215

13-
This is `<provider>/<model>` shorthand for:
16+
`<provider>/<model>` shorthand for the Named Client version of `MyClient`:
1417

1518
```rust BAML
1619
client<llm> MyClient {
@@ -38,7 +41,7 @@ the latter form.
3841

3942
<Tip>
4043
If you want to specify which client to use at runtime, in your Python/TS/Ruby code,
41-
you can use the [client registry](/guide/baml-advanced/client-registry) to do so.
44+
you can use the [client registry](/ref/baml_client/client-registry) to do so.
4245

4346
This can come in handy if you're trying to, say, send 10% of your requests to a
4447
different model.

‎fern/03-reference/overview.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ For more in-depth explanations, we recommend reading the [Guides](/guide) first.
2525
VSCode BAML Extension settings
2626
</Card>
2727

28-
<Card title="LLM Clients" icon="fa-solid fa-brain" href="/ref/llm-client-providers">
28+
<Card title="LLM Clients" icon="fa-solid fa-brain" href="/ref/baml/client-llm">
2929
LLM clients and how to configure them.
3030
</Card>
3131

‎fern/docs.yml‎

Lines changed: 47 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -480,6 +480,8 @@ navigation:
480480
path: 03-reference/baml/attributes/dynamic.mdx
481481
- section: LLM Client Providers
482482
contents:
483+
- page: "Overview"
484+
path: 03-reference/baml/client-llm.mdx
483485
- page: "AWS Bedrock"
484486
path: 03-reference/baml/clients/providers/aws-bedrock.mdx
485487
- page: "Anthropic"
@@ -494,25 +496,38 @@ navigation:
494496
path: 03-reference/baml/clients/providers/azure.mdx
495497
- page: "openai-generic"
496498
path: 03-reference/baml/clients/providers/openai-generic.mdx
497-
- page: "openai-generic: Azure AI Foundary"
499+
- page: "Azure AI Foundary (openai-generic)"
500+
slug: azure-ai-foundary
498501
path: 03-reference/baml/clients/providers/azure-ai-foundary.mdx
499-
- page: "openai-generic: Groq"
502+
- page: "Groq (openai-generic)"
503+
slug: groq
500504
path: 03-reference/baml/clients/providers/groq.mdx
501-
- page: "openai-generic: Hugging Face"
505+
- page: "Hugging Face (openai-generic)"
506+
slug: huggingface
502507
path: 03-reference/baml/clients/providers/huggingface.mdx
503-
- page: "openai-generic: Keywords AI"
508+
- page: "Keywords AI (openai-generic)"
509+
slug: keywordsai
504510
path: 03-reference/baml/clients/providers/keywordsai.mdx
505-
- page: "openai-generic: LM Studio"
511+
- page: "Litellm (openai-generic)"
512+
slug: litellm
513+
path: 03-reference/baml/clients/providers/litellm.mdx
514+
- page: "LM Studio (openai-generic)"
515+
slug: lmstudio
506516
path: 03-reference/baml/clients/providers/lmstudio.mdx
507-
- page: "openai-generic: Ollama"
517+
- page: "Ollama (openai-generic)"
518+
slug: ollama
508519
path: 03-reference/baml/clients/providers/ollama.mdx
509-
- page: "openai-generic: OpenRouter"
520+
- page: "OpenRouter (openai-generic)"
521+
slug: openrouter
510522
path: 03-reference/baml/clients/providers/openrouter.mdx
511-
- page: "openai-generic: TogetherAI"
523+
- page: "TogetherAI (openai-generic)"
524+
slug: together
512525
path: 03-reference/baml/clients/providers/together.mdx
513-
- page: "openai-generic: Unify AI"
526+
- page: "Unify AI (openai-generic)"
527+
slug: unify
514528
path: 03-reference/baml/clients/providers/unify.mdx
515-
- page: "openai-generic: vLLM"
529+
- page: "vLLM (openai-generic)"
530+
slug: vllm
516531
path: 03-reference/baml/clients/providers/vllm.mdx
517532
- section: LLM Client Strategies
518533
contents:
@@ -843,3 +858,25 @@ redirects:
843858
destination: "/ref/boundary-extraction-api/extract"
844859
- source: "/document-extraction-api/overview/docs/api/extraction-examples"
845860
destination: "/ref/boundary-extraction-api/extract/examples"
861+
- source: "/ref/llm-client-providers/openai-generic-azure-ai-foundary"
862+
destination: "/ref/llm-client-providers/azure-ai-foundary"
863+
- source: "/ref/llm-client-providers/openai-generic-groq"
864+
destination: "/ref/llm-client-providers/groq"
865+
- source: "/ref/llm-client-providers/openai-generic-hugging-face"
866+
destination: "/ref/llm-client-providers/huggingface"
867+
- source: "/ref/llm-client-providers/openai-generic-keywords-ai"
868+
destination: "/ref/llm-client-providers/keywordsai"
869+
- source: "/ref/llm-client-providers/openai-generic-litellm"
870+
destination: "/ref/llm-client-providers/litellm"
871+
- source: "/ref/llm-client-providers/openai-generic-lm-studio"
872+
destination: "/ref/llm-client-providers/lmstudio"
873+
- source: "/ref/llm-client-providers/openai-generic-ollama"
874+
destination: "/ref/llm-client-providers/ollama"
875+
- source: "/ref/llm-client-providers/openai-generic-openrouter"
876+
destination: "/ref/llm-client-providers/openrouter"
877+
- source: "/ref/llm-client-providers/openai-generic-togetherai"
878+
destination: "/ref/llm-client-providers/together"
879+
- source: "/ref/llm-client-providers/openai-generic-unify-ai"
880+
destination: "/ref/llm-client-providers/unify"
881+
- source: "/ref/llm-client-providers/openai-generic-vllm"
882+
destination: "/ref/llm-client-providers/vllm"

‎fern/snippets/client-constructor.mdx‎

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,33 @@ This configures which provider to use. The provider is responsible for handling
33

44
The configuration modifies the URL request BAML runtime makes.
55

6-
| Provider Name | Docs | Notes |
7-
| ---------------- | ------------------------------------------------------------------- | ---------------------------------------------------------- |
8-
| `anthropic` | [Anthropic](/ref/llm-client-providers/anthropic) | |
9-
| `aws-bedrock` | [AWS Bedrock](/ref/llm-client-providers/aws-bedrock) | |
10-
| `azure-openai` | [Azure OpenAI](/ref/llm-client-providers/open-ai-from-azure) | |
11-
| `google-ai` | [Google AI](/ref/llm-client-providers/google-ai-gemini) | |
12-
| `openai` | [OpenAI](/ref/llm-client-providers/open-ai) | |
13-
| `openai-generic` | [OpenAI (generic)](/ref/llm-client-providers/openai-generic) | Any model provider that supports an OpenAI-compatible API |
14-
| `vertex-ai` | [Vertex AI](/ref/llm-client-providers/google-vertex) | |
6+
| Provider Name | Docs | Notes |
7+
| ---------------- | ------------------------------------------------------------ | ---------------------------------------------------------- |
8+
| `anthropic` | [Anthropic](/ref/llm-client-providers/anthropic) | Supports [/v1/messages](https://docs.anthropic.com/en/api/messages) endpoint |
9+
| `aws-bedrock` | [AWS Bedrock](/ref/llm-client-providers/aws-bedrock) | Supports [Converse](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html) and [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html) endpoint |
10+
| `google-ai` | [Google AI](/ref/llm-client-providers/google-ai-gemini) | Supports Google AI's [generateContent](https://ai.google.dev/api/generate-content) and [streamGenerateContent](https://ai.google.dev/api/generate-content#method:-models.streamgeneratecontent) endpoints |
11+
| `vertex-ai` | [Vertex AI](/ref/llm-client-providers/google-vertex) | Supports Vertex's [generateContent](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.publishers.models/generateContent) and [streamGenerateContent](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.publishers.models/streamGenerateContent) endpoints |
12+
| `openai` | [OpenAI](/ref/llm-client-providers/open-ai) | Supports [/chat/completions](https://platform.openai.com/docs/api-reference/chat) endpoint |
13+
| `azure-openai` | [Azure OpenAI](/ref/llm-client-providers/open-ai-from-azure) | Supports Azure's [/chat/completions](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#chat-completions) endpoint |
14+
| `openai-generic` | [OpenAI (generic)](/ref/llm-client-providers/openai-generic) | Any other provider that supports OpenAI's `/chat/completions` endpoint |
15+
16+
17+
A non-exhaustive list of providers you can use with `openai-generic`:
18+
19+
| Inference Provider | Docs |
20+
| -------------- | -------------- |
21+
| Azure AI Foundary | [Azure AI Foundary](/ref/llm-client-providers/azure-ai-foundary) |
22+
| Groq | [Groq](/ref/llm-client-providers/groq) |
23+
| Hugging Face | [Hugging Face](/ref/llm-client-providers/huggingface) |
24+
| Keywords AI | [Keywords AI](/ref/llm-client-providers/keywordsai) |
25+
| Litellm | [Litellm](/ref/llm-client-providers/litellm) |
26+
| LM Studio | [LM Studio](/ref/llm-client-providers/lmstudio) |
27+
| Ollama | [Ollama](/ref/llm-client-providers/ollama) |
28+
| OpenRouter | [OpenRouter](/ref/llm-client-providers/openrouter) |
29+
| TogetherAI | [TogetherAI](/ref/llm-client-providers/together) |
30+
| Unify AI | [Unify AI](/ref/llm-client-providers/unify) |
31+
| vLLM | [vLLM](/ref/llm-client-providers/vllm) |
32+
1533

1634
We also have some special providers that allow composing clients together:
1735
| Provider Name | Docs | Notes |

0 commit comments

Comments
 (0)