Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions website/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,7 @@ This is a warning
- **Style**:
- **Imperative**: Use command style for steps (e.g., "Create a file...").
- **Concise**: Avoid marketing fluff. Link to prerequisites instead of repeating them.

## 4. Content Guidelines

- **Inference Deployment**: When documenting deployment of inference services (e.g., vLLM, SGLang), instructions MUST use the `InferenceService` resource with a preset.
4 changes: 2 additions & 2 deletions website/docs/features/preset.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebar_position: 1
sidebar_label: Presets
title: Presets
sidebar_label: Presets
sidebar_position: 1
---

The MoAI Inference Framework provides a set of pre-configured `InferenceServiceTemplate`s, known as presets. These presets encapsulate standard configurations for various models and hardware setups, simplifying the deployment of inference services.
Expand Down
2 changes: 1 addition & 1 deletion website/docs/getting-started/prerequisites.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ device node GUID

### RDMA device plugin installation

This section describes how to install the **rdma-shared-device-plugin**. See [k8s-rdma-shared-dev-plugin / README](https://github.com/Mellanox/k8s-rdma-shared-dev-plugin/blob/master/README.mdx) for more details.
This section describes how to install the **rdma-shared-device-plugin**. See [k8s-rdma-shared-dev-plugin / README](https://github.com/Mellanox/k8s-rdma-shared-dev-plugin/blob/master/README.md) for more details.

First, create a `rdma-shared-device-plugin.yaml` file as follows. **You need to replace `<device>` with your RDMA NIC's network interface name**. If multiple NICs are installed on the server, you must list all interface names (e.g., `"devices": ["ib0", "ib1"]`).

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Container image caching with Harbor
sidebar_position: 4
sidebar_label: Container image caching (Harbor)
sidebar_position: 4
---

import Tabs from '@theme/Tabs';
Expand Down Expand Up @@ -354,7 +354,8 @@ spec:
background: true
emitWarning: false
rules:
- match:
- name: ecr-rewrite-registry-container
match:
any:
- resources:
kinds:
Expand All @@ -365,11 +366,11 @@ spec:
patchStrategicMerge:
spec:
containers: # set registry port as your setup (80 or 443)
- image: '{{ regex_replace_all_literal("255250787067.dkr.ecr.ap-northeast-2.amazonaws.com", "{{ element.image }}, "harbor.harbor.svc.cluster.local:<yourPort>/mif") }}'
- image: "{{ regex_replace_all_literal('255250787067.dkr.ecr.ap-northeast-2.amazonaws.com', '{{ element.image }}', 'harbor.harbor.svc.cluster.local:<yourPort>/mif') }}"
name: '{{ element.name }}'
name: ecr-rewrite-registry-container
skipBackgroundRequests: true
- match:
- name: ecr-rewrite-registry-init-container
match:
any:
- resources:
kinds:
Expand All @@ -380,12 +381,11 @@ spec:
patchStrategicMerge:
spec:
initContainers: # set registry port as your setup (80 or 443)
- image: '{{ regex_replace_all_literal("255250787067.dkr.ecr.ap-northeast-2.amazonaws.com", "{{ element.image }}, "harbor.harbor.svc.cluster.local:<yourPort>/mif") }}'
- image: "{{ regex_replace_all_literal('255250787067.dkr.ecr.ap-northeast-2.amazonaws.com', '{{ element.image }}', 'harbor.harbor.svc.cluster.local:<yourPort>/mif') }}"
name: '{{ element.name }}'
name: ecr-rewrite-registry-init-container
preconditions:
all:
- key: '{{ request.object.spec.initContainers[] || `[]` | length(@) }}
- key: '{{ request.object.spec.initContainers[] || `[]` | length(@) }}'
operator: GreaterThanOrEquals
value: 1
skipBackgroundRequests: true
Expand Down
16 changes: 9 additions & 7 deletions website/docs/reference/heimdall/api-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -73,16 +73,18 @@ kubectl explain --api-version inference.networking.k8s.io/v1 inferencepools.spec

### EndpointPickerConfig

| Field | Type | Description |
| :------------------- | :------------------------------------------ | :-------------------------------------------------------------------------------------- |
| `data` | [`DataLayerConfig`](#datalayerconfig) | Data configures the DataLayer. It is required if the new DataLayer is enabled. |
| `featureGates` | []string | FeatureGates is a set of flags that enable various experimental features with the EPP. |
| `plugins` | [`[]PluginSpec`](#pluginspec) | Plugins is the list of plugins that will be instantiated. |
| `saturationDetector` | [`SaturationDetector`](#saturationdetector) | SaturationDetector when present specifies the configuration of the Saturation detector. |
| `schedulingProfiles` | [`[]SchedulingProfile`](#schedulingprofile) | SchedulingProfiles is the list of named SchedulingProfiles that will be created. |
| Field | Type | Description |
| :------------------- | :------------------------------------------ | :------------------------------------------------------------------------------------------------------- |
| `data` | [`DataLayerConfig`](#datalayerconfig) | Data configures the DataLayer. It is required if the new DataLayer is enabled. |
| `featureGates` | `[]string` | FeatureGates is a set of flags that enable various experimental features with the EPP. |
| `plugins` | [`[]PluginSpec`](#pluginspec) | Plugins is the list of plugins that will be instantiated. See [Plugins](./plugins.mdx) for more details. |
| `saturationDetector` | [`SaturationDetector`](#saturationdetector) | SaturationDetector when present specifies the configuration of the Saturation detector. |
| `schedulingProfiles` | [`[]SchedulingProfile`](#schedulingprofile) | SchedulingProfiles is the list of named SchedulingProfiles that will be created. |

#### PluginSpec

For more details on available plugins, see [Plugins](./plugins.mdx).

| Field | Type | Description |
| :----------- | :------- | :------------------------------------------------------------------------------------------------------------------- |
| `name` | `string` | Name provides a name for plugin entries to reference. If omitted, the value of the Plugin's Type field will be used. |
Expand Down
6 changes: 3 additions & 3 deletions website/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.