From 266da0b4ec32bdac1c50a87c63f5cbdd93df3680 Mon Sep 17 00:00:00 2001 From: Allie Sadler <102604716+aevesdocker@users.noreply.github.com> Date: Tue, 1 Jul 2025 11:46:14 +0100 Subject: [PATCH 1/4] Merge pull request #22966 from aevesdocker/ENGDOCS-2798 models in Compose apps --- content/manuals/ai/compose/_index.md | 9 + .../manuals/ai/compose/models-and-compose.md | 184 ++++++++++++++++++ 2 files changed, 193 insertions(+) create mode 100644 content/manuals/ai/compose/_index.md create mode 100644 content/manuals/ai/compose/models-and-compose.md diff --git a/content/manuals/ai/compose/_index.md b/content/manuals/ai/compose/_index.md new file mode 100644 index 000000000000..a861d426a60e --- /dev/null +++ b/content/manuals/ai/compose/_index.md @@ -0,0 +1,9 @@ +--- +build: + render: never +title: AI and Docker Compose +weight: 40 +params: + sidebar: + group: AI +--- \ No newline at end of file diff --git a/content/manuals/ai/compose/models-and-compose.md b/content/manuals/ai/compose/models-and-compose.md new file mode 100644 index 000000000000..bdfb0de5eb65 --- /dev/null +++ b/content/manuals/ai/compose/models-and-compose.md @@ -0,0 +1,184 @@ +--- +title: Define AI Models in Docker Compose applications +linkTitle: Use AI models in Compose +description: Learn how to define and use AI models in Docker Compose applications using the models top-level element +keywords: compose, docker compose, models, ai, machine learning, cloud providers, specification +weight: 10 +params: + sidebar: + badge: + color: green + text: New +--- + +{{< summary-bar feature_name="Compose models" >}} + +Compose lets you define AI models as core components of your application, so you can declare model dependencies alongside services and run the application on any platform that supports the Compose Specification. + +## Prerequisites + +- Docker Compose v2.38 or later +- A platform that supports Compose models such as Docker Model Runner or compatible cloud providers + +## What are Compose models? + +Compose `models` are a standardized way to define AI model dependencies in your application. By using the []`models` top-level element](/reference/compose-file/models.md) in your Compose file, you can: + +- Declare which AI models your application needs +- Specify model configurations and requirements +- Make your application portable across different platforms +- Let the platform handle model provisioning and lifecycle management + +## Basic model definition + +To define models in your Compose application, use the `models` top-level element: + +```yaml +services: + chat-app: + image: my-chat-app + models: + - llm + +models: + llm: + image: ai/smollm2 +``` + +This example defines: +- A service called `chat-app` that uses a model named `llm` +- A model definition for `llm` that references the `ai/smollm2` model image + +## Model configuration options + +Models support various configuration options: + +```yaml +models: + llm: + image: ai/smollm2 + context_size: 1024 + runtime_flags: + - "--a-flag" + - "--another-flag=42" +``` + +Common configuration options include: +- `model` (required): The OCI artifact identifier for the model. This is what Compose pulls and runs via the model runner. +- `context_size`: Defines the maximum token context size for the model. +- `runtime_flags`: A list of raw command-line flags passed to the inference engine when the model is started. +- Platform-specific options may also be available via extensions attributes `x-*` + +## Service model binding + +Services can reference models in two ways: short syntax and long syntax. + +### Short syntax + +The short syntax is the simplest way to bind a model to a service: + +```yaml +services: + app: + image: my-app + models: + - llm + - embedding-model + +models: + llm: + image: ai/smollm2 + embedding-model: + image: ai/all-minilm +``` + +With short syntax, the platform automatically generates environment variables based on the model name: +- `LLM_URL` - URL to access the llm model +- `LLM_MODEL` - Model identifier for the llm model +- `EMBEDDING_MODEL_URL` - URL to access the embedding-model +- `EMBEDDING_MODEL_MODEL` - Model identifier for the embedding-model + +### Long syntax + +The long syntax allows you to customize environment variable names: + +```yaml +services: + app: + image: my-app + models: + llm: + endpoint_var: AI_MODEL_URL + model_var: AI_MODEL_NAME + embedding-model: + endpoint_var: EMBEDDING_URL + model_var: EMBEDDING_NAME + +models: + llm: + image: ai/smollm2 + embedding-model: + image: ai/all-minilm +``` + +With this configuration, your service receives: +- `AI_MODEL_URL` and `AI_MODEL_NAME` for the LLM model +- `EMBEDDING_URL` and `EMBEDDING_NAME` for the embedding model + +## Platform portability + +One of the key benefits of using Compose models is portability across different platforms that support the Compose specification. + +### Docker Model Runner + +When Docker Model Runner is enabled: + +```yaml +services: + chat-app: + image: my-chat-app + models: + - llm + +models: + llm: + image: ai/smollm2 +``` + +Docker Model Runner will: +- Pull and run the specified model locally +- Provide endpoint URLs for accessing the model +- Inject environment variables into the service + +### Cloud providers + +The same Compose file can run on cloud providers that support Compose models: + +```yaml +services: + chat-app: + image: my-chat-app + models: + - llm + +models: + llm: + image: ai/smollm2 + # Cloud-specific configurations + labels: + - "cloud.instance-type=gpu-small" + - "cloud.region=us-west-2" +``` + +Cloud providers might: +- Use managed AI services instead of running models locally +- Apply cloud-specific optimizations and scaling +- Provide additional monitoring and logging capabilities +- Handle model versioning and updates automatically + +## Reference + +- [`models` top-level element](/reference/compose-file/models.md) +- [`models` attribute](/reference/compose-file/services.md#models) +- [Docker Model Runner documentation](/manuals/ai/model-runner.md) +- [Compose Model Runner documentation](/manuals/compose/how-tos/model-runner.md)] \ No newline at end of file From 03ae6f16f899fd1398624fcd0e33b452860981b1 Mon Sep 17 00:00:00 2001 From: Arthur Date: Tue, 1 Jul 2025 13:00:56 +0200 Subject: [PATCH 2/4] chore: fix search result color (#22962) Some parts of the search results window are hard to read. Let's fix this. Before: see live website After: ![Screenshot 2025-07-01 at 12 45 40](https://github.com/user-attachments/assets/34e8e10c-d2ab-4c8b-9d83-07299915464e) ![Screenshot 2025-07-01 at 12 45 49](https://github.com/user-attachments/assets/a851dcdb-5423-40bc-9a8b-19cd949c408c) --- assets/css/global.css | 6 ++++++ assets/css/utilities.css | 4 ++++ layouts/_default/search.html | 12 ++++++------ layouts/partials/search-bar.html | 2 +- 4 files changed, 17 insertions(+), 7 deletions(-) diff --git a/assets/css/global.css b/assets/css/global.css index 90887417a2ab..82f0fc386479 100644 --- a/assets/css/global.css +++ b/assets/css/global.css @@ -95,3 +95,9 @@ input[type="search"]::-ms-clear { .navbar-group:first-of-type { margin-top: 0.2rem !important; } + +#search-page-results { + mark:where(.dark, .dark *) { + color: var(--color-blue-400); + } +} diff --git a/assets/css/utilities.css b/assets/css/utilities.css index d3cbc10a734f..ee01aaadc7ff 100644 --- a/assets/css/utilities.css +++ b/assets/css/utilities.css @@ -253,3 +253,7 @@ @utility chip { @apply border-divider-light dark:border-divider-dark inline-flex items-center gap-1 rounded-full border bg-gray-100 px-2 text-sm text-gray-800 select-none dark:bg-gray-700 dark:text-gray-200; } + +@utility pagination-link { + @apply flex items-center justify-center rounded-sm p-2; +} diff --git a/layouts/_default/search.html b/layouts/_default/search.html index f83708f9c24b..841cd4276baa 100644 --- a/layouts/_default/search.html +++ b/layouts/_default/search.html @@ -133,14 +133,14 @@

{{ .Title }}

} // Generate the search results HTML - let resultsHTML = `
${resultsLength} results
`; + let resultsHTML = `
${resultsLength} results
`; // Map results to HTML resultsHTML += results .map((item) => { return `
- ${item.meta.breadcrumbs} + ${item.meta.breadcrumbs} ${item.meta.title}

…${item.excerpt}…

@@ -153,12 +153,12 @@

{{ .Title }}

resultsHTML += `