Skip to content

MAF-19131: feat(preset): add new preset for DeepSeek-R1 max throughput#46

Merged
hhk7734 merged 8 commits intomainfrom
MAF-19131-update-preset
Feb 2, 2026
Merged

MAF-19131: feat(preset): add new preset for DeepSeek-R1 max throughput#46
hhk7734 merged 8 commits intomainfrom
MAF-19131-update-preset

Conversation

@bongwoobak
Copy link
Copy Markdown
Contributor

@bongwoobak bongwoobak commented Feb 2, 2026

https://docs.moreh.io/benchmarking/deepseek_r1_671b_on_amd_mi300x_gpus_maximum_throughput/ example is

apiVersion: odin.moreh.io/v1alpha1
kind: InferenceService
metadata:
  name: deepseek-r1-prefill
  namespace: mif
spec:
  replicas: 2
  inferencePoolRefs:
    - name: heimdall
  templateRefs:
    - name: vllm-prefill-dp
    - name: vllm-deepseek-r1-prefill-mi300x-dp8ep-maxthroughput
  workerTemplate:
    spec:
      containers:
        - name: main
          env:
            - name: HF_TOKEN
              value: "<huggingfaceToken>"
          resources:
            limits:
              mellanox/hca: "1"
            requests:
              mellanox/hca: "1"
---
apiVersion: odin.moreh.io/v1alpha1
kind: InferenceService
metadata:
  name: deepseek-r1-decode
  namespace: mif
spec:
  replicas: 3
  inferencePoolRefs:
    - name: heimdall
  templateRefs:
    - name: vllm-decode-dp
    - name: vllm-deepseek-r1-decode-mi300x-dp8ep-maxthroughput
  workerTemplate:
    spec:
      containers:
        - name: main
          env:
            - name: HF_TOKEN
              value: "<huggingfaceToken>"
          resources:
            limits:
              mellanox/hca: "1"
            requests:
              mellanox/hca: "1"

@bongwoobak bongwoobak requested a review from a team as a code owner February 2, 2026 10:41
@bongwoobak bongwoobak requested a review from hhk7734 February 2, 2026 10:41
@bongwoobak bongwoobak changed the title MAF-19131:feat(deploy): add new preset for DeepSeek-R1 max throughput MAF-19131: feat(deploy): add new preset for DeepSeek-R1 max throughput Feb 2, 2026
@hhk7734 hhk7734 changed the title MAF-19131: feat(deploy): add new preset for DeepSeek-R1 max throughput MAF-19131: feat(preset): add new preset for DeepSeek-R1 max throughput Feb 2, 2026
Copy link
Copy Markdown
Member

@hhk7734 hhk7734 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quickstart 로 이동시켜주세요

Copy link
Copy Markdown
Member

@hhk7734 hhk7734 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

          resources:
            requests:
              amd.com/gpu: "8"
            limits:
              amd.com/gpu: "8"
      nodeSelector:
        moai.moreh.io/accelerator.vendor: amd
        moai.moreh.io/accelerator.model: mi300x
      tolerations:
        - key: amd.com/gpu
          operator: Exists
          effect: NoSchedule

추가해주세요

@hhk7734 hhk7734 merged commit 76deb9e into main Feb 2, 2026
3 checks passed
@hhk7734 hhk7734 deleted the MAF-19131-update-preset branch February 2, 2026 16:06
@bongwoobak bongwoobak restored the MAF-19131-update-preset branch February 3, 2026 07:11
@bongwoobak bongwoobak deleted the MAF-19131-update-preset branch February 3, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants