Skip to content

[NVIDIA] chore: B300 single node DeepSeek v4 SGLang#1132

Merged
cquil11 merged 26 commits intomainfrom
chore/dsv4-sgl-b300
Apr 25, 2026
Merged

[NVIDIA] chore: B300 single node DeepSeek v4 SGLang#1132
cquil11 merged 26 commits intomainfrom
chore/dsv4-sgl-b300

Conversation

@cquil11
Copy link
Copy Markdown
Collaborator

@cquil11 cquil11 commented Apr 24, 2026

Summary

  • Add dsv4-fp4-b300-sglang to .github/configs/nvidia-master.yaml (TP=4/EP=4, conc 4–128 for 1k1k and 8k1k)
  • Add benchmarks/single_node/dsv4_fp4_b300.sh mirroring the SGLang DeepSeek-V4 cookbook B200 Flash Low-Latency recipe (cookbook ships no B300-specific recipe yet)
  • Speculative decoding (EAGLE) and prefix caching are disabled per request
  • Append perf-changelog entry to trigger the sweep

Image: lmsysorg/sglang:deepseek-v4-blackwell · Model: deepseek-ai/DeepSeek-V4-Flash

Test plan

  • python3 utils/matrix_logic/generate_sweep_configs.py full-sweep --config-files .github/configs/nvidia-master.yaml --runner-type b300 --model-prefix dsv4 → 12 matrix entries, validation passes
  • python3 -m pytest utils/matrix_logic/ -q → 149 passed
  • Sweep run on B300 completes end-to-end

@cquil11 @claude
feat: add DeepSeek-V4-Flash FP4 B300 SGLang benchmark
26e540d 
Adds dsv4-fp4-b300-sglang config, single-node benchmark script, and
perf-changelog entry for the DeepSeek-V4 recipe from the SGLang
cookbook. The cookbook ships a B200 (not B300) recipe, so this
reuses the B200 Flash Low-Latency recipe on B300 until a
B300-specific recipe lands. Speculative decoding (EAGLE) and prefix
caching are disabled per request.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 cquil11 requested a review from a team April 24, 2026 06:14
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 24, 2026

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

@cquil11 @claude
fix: switch dsv4-fp4-b300-sglang to Pro + Max-Throughput recipe
efdc8ba 
Match parallelism (TP=8/EP=8/dp-attn=true) and concurrency ranges
(4-1024 for 1k1k, 4-512 for 8k1k) to dsv4-fp4-b200-vllm. Use the
DeepSeek-V4-Pro variant with the cookbook Max-Throughput recipe
(DP=8 + DeepEP, no MTP), which aligns with the requested no-spec
parallelism. Prefix caching remains disabled.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@functionstackx functionstackx changed the title feat: add DeepSeek-V4-Flash FP4 B300 SGLang benchmark feat: add DeepSeek-V4 FP4 B300 SGLang benchmark Apr 24, 2026
cquil11 and others added 5 commits April 24, 2026 01:22
@cquil11 @claude
chore: sync launch_b200-dgxc-slurm.sh cache mount from claude/add-dsv…
cc35a12 
…4-fp4-b200-vllm

Port the HF cache mount rework from the DSV4 B200 VLLM branch so
both PRs stay consistent: use the shared /scratch/fsw/gharunners/hf-hub-cache
path, drop the local MODEL override, and mount onto \$HF_HUB_CACHE
inside the container.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
fix: restore trailing whitespace stripped from glm5.1 changelog entry
404a097 
The dsv4-fp4-b300-sglang entry was appended correctly, but the earlier
edit also stripped trailing spaces on an existing line, producing a
spurious deletion. Revert so the diff is additive-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
chore: add flock-guarded squash import to B300 runner
97a488e 
Mirror the lockfile logic already in launch_b200-dgxc-slurm.sh and
launch_h200-dgxc-slurm.sh: serialize concurrent enroot imports of
the same squash file via flock, skip the import when the squash is
already valid, and override ENROOT_CACHE_PATH to avoid permission
issues with the system-wide cache on worker nodes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
fix: drop ENROOT_CACHE_PATH override from B300 runner
106deea 
The override ("avoid permission issues with system-wide cache on
worker nodes") is a dgxc-slurm-specific workaround; launch_b300-nv.sh
is on the NV slurm cluster, not dgxc-slurm. Copying it in caused
the benchmark srun's pyxis shadow hook to fail with
'mkdir: cannot create directory pyxis_$JOBID.1/data: File exists'.
Keep the flock + skip-if-valid logic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
chore: point B300 runner at shared gharunners/{squash,hf-hub-cache}
4bb1f1a 
Move the squash cache from /data/squash to /data/home/sa-shared/gharunners/squash,
and the HF cache mount from /scratch/models to /data/home/sa-shared/gharunners/hf-hub-cache.
Also mount the host HF cache onto \$HF_HUB_CACHE inside the container so
tools reading the default HF path pick it up (matches the B200 dgxc-slurm
runner). Drop the /scratch/models Qwen3.5 path override since that path
is no longer used.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 cquil11 mentioned this pull request Apr 24, 2026
Closed
3 tasks
cquil11 and others added 11 commits April 24, 2026 02:24
@cquil11 @claude
fix: move enroot import out of srun to avoid pyxis namespace collision
744c5a0 
Running two srun steps in the same allocation (flock+import, then the
benchmark --container-image srun) reproducibly fails on this cluster
with:
  error: pyxis: mkdir: cannot create directory
    '/scratch/data/user-$UID/pyxis_$JOBID.1/data': File exists
  error: pyxis:     [ERROR] /etc/enroot/hooks.d/10-shadow.sh exited with return code 1

Per NVIDIA/pyxis#138, two srun steps sharing an allocation can leave
enroot/pyxis state between steps. Collapsing to a single srun (the
benchmark) is the cleanest workaround. Move the flock-guarded
enroot import to the host side, before salloc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
fix: wipe stale pyxis scratch dirs for this JOB_ID before benchmark srun
d003c59 
Even with a single srun step, pyxis fails with
  error: pyxis: mkdir: cannot create directory
      '/scratch/data/user-$UID/pyxis_$JOBID.0/data': File exists
on fresh SLURM JOB_IDs. The /scratch path is left behind by previous
jobs whose IDs SLURM later reuses (and the cluster's pyxis epilog
doesn't clean it up). Wipe pyxis_$JOBID.* from the host after salloc;
no-op if /scratch is node-local, effective if it's shared NFS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
Revert: drop all B300 runner changes, mirror #1128's approach
f00629f 
PR #1128 (dsv4-fp4fp8-b300-vllm) runs on the same cluster with ZERO
changes to launch_b300-nv.sh. The pyxis 10-shadow.sh failures we were
chasing aren't caused by the runner -- reset it to origin/main and
keep the sglang config/bench additions only.

Reverts (from this branch):
- 4bb1f1a point B300 runner at shared gharunners/{squash,hf-hub-cache}
- 106deea drop ENROOT_CACHE_PATH override
- 97a488e add flock-guarded squash import
- 744c5a0 move enroot import out of srun
- d003c59 wipe stale pyxis scratch before benchmark srun

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
runner: add head-node flock-guarded squash import on B300
570b0eb 
Move enroot import out of srun to the head node and serialize parallel
GH jobs with flock on the shared squash file. Skips the import when a
valid squash already exists. The benchmark srun is now the only step
in the allocation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
fix: mount at /ix and clear baked-in CUDA_VISIBLE_DEVICES
864419d 
Port the B200 branch's fix for the lmsysorg/sglang:deepseek-v4-blackwell
image on B300:
- The image installs sglang editable under /workspace/sglang; the default
  $GITHUB_WORKSPACE:/workspace/ bind-mount masks the install and breaks
  'import sglang'. For this image, mount at /ix instead.
- The image's ENV bakes CUDA_VISIBLE_DEVICES=4,5,6,7, masking half the
  GPUs Slurm allocates. Unset it in the bench script so TP=8 sees all 8.
- Write artefacts under $PWD instead of hard-coded /workspace.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11
Merge branch 'main' into chore/dsv4-sgl-b300
5d93913
@cquil11 @claude
runner: use /data/models pre-staged path for dsv4 on B300
9453676 
Pre-staged models on the B300 cluster live under /data/models
(Qwen3.5-397B-A17B-FP8, dsv4-pro, etc.). Switch HF_HUB_CACHE_MOUNT
from /scratch/models to /data/models, and export MODEL to
/data/models/dsv4-pro when MODEL_PREFIX=dsv4 so the benchmark reads
from the mounted dir directly. The bench script skips `hf download`
when MODEL looks like an absolute path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
fix: switch B300 dsv4 sglang to bw-ultra-compiled image
5db43b8 
The stock lmsysorg/sglang:deepseek-v4-blackwell image ships kernels
compiled for B200 (SM_100) and crashes on B300 with
  RuntimeError: RMSNorm failed with error code no kernel image is
  available for execution on the device
during CUDA graph capture. Switch to cquil/sglang-deepseek-v4-bw-ultra:v1,
which is recompiled with B300 SM support.

Broaden the /ix mount conditional to match both image tags: the fork
keeps the same /workspace/sglang editable install that would otherwise
be masked by $GITHUB_WORKSPACE:/workspace/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
fix: switch B300 dsv4 sglang image to yhyang201/sglang-b300:v3
c060c58 
Use the B300-recompiled image from yhyang201; extend the /ix mount
conditional to match the new tag in addition to the previous
deepseek-v4-blackwell / deepseek-v4-bw-ultra patterns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11
update b300
08edf26
@cquil11 @claude
feat(dsv4-fp4-b300-sglang): pick recipe by CONC; split search-space
a699ca0 
Mirror chore/dsv4-sgl-b200 commits 103a202 + 43be495 for B300:

Bench script now selects one of three cookbook recipes by CONC instead
of a single static flag set:
  CONC <= 32   -> low-latency    (TP only, chunked-prefill 4096,
                                  disable-flashinfer-autotune)
  33..128      -> balanced       (+ DP-attention, max-running-reqs=128,
                                  cuda-graph-max-bs=64, deepep-config)
  CONC > 128   -> max-throughput (+ DP-attention, max-running-reqs=256,
                                  cuda-graph-max-bs=64, deepep-config)
No speculative decoding in any recipe; --disable-radix-cache kept for
the no-prefix-caching baseline.

Split the dsv4-fp4-b300-sglang search-space rows per recipe boundary so
result filenames (ep=, dpa=) accurately reflect which recipe ran.
ep=8 on balanced/max-throughput reflects sglang's implicit
ep_size=tp_size override when --moe-a2a-backend deepep is set.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
update b300
d35696c 
Switch B300 dsv4 sglang image to lmsysorg/sglang:deepseek-v4-b300
and extend the /ix mount conditional to match the new tag.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 cquil11 changed the title feat: add DeepSeek-V4 FP4 B300 SGLang benchmark [NVIDIA] chore: B300 single node DeepSeek v4 SGLang Apr 24, 2026
@cquil11 cquil11 mentioned this pull request Apr 24, 2026
Merged
3 tasks
@cquil11 cquil11 changed the title [NVIDIA] chore: B300 single node DeepSeek v4 SGLang [NVIDIA][needs fixing] chore: B300 single node DeepSeek v4 SGLang Apr 24, 2026
@cquil11 @claude
feat(dsv4-fp4-b300-sglang): hardcode low-latency recipe at every CONC
bc43672 
The DeepEP FP8 weight-postprocess path is broken for
deepseek-ai/DeepSeek-V4-Pro on B300 with
lmsysorg/sglang:deepseek-v4-b300 -- every sglang launch with
--moe-a2a-backend deepep fails during model load with
  RuntimeError: Recipe must be a list/tuple of 3 integers.
raised from sglang.srt.layers.quantization.fp8
.process_weights_after_loading_block_quant (fp8.py:957). The balanced
and max-throughput recipes both go through that path; the low-latency
recipe (TP-only, flashinfer_mxfp4 MoE) does not and loads cleanly.

Collapse the yaml search-space back to a single row spanning the full
CONC range (4..1024 for 1k1k, 4..512 for 8k1k) and hardcode the bench
script to the low-latency flags at every CONC. TODO(Cam) noted in both
files to restore the recipe-per-CONC dispatch once the DeepEP FP8 load
path is fixed upstream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Klaud-Cold pushed a commit that referenced this pull request Apr 24, 2026
@github-actions @cquil11 @claude
fix: align perf-changelog and config comments with low-latency fallback
410df74 
The perf-changelog entry and outer NOTE comment in nvidia-master.yaml
described the max-throughput recipe from #1132, not the low-latency
fallback this PR actually adds. Rewrite both to match the actual config:
TP=8/EP=1, no DP-attn, no DeepEP, image deepseek-v4-b300, pr-link #1143.

Co-authored-by: Cameron Quilici <cquil11@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
cquil11 added a commit that referenced this pull request Apr 24, 2026
@cquil11 @claude @github-actions
[NVIDIA] chore: B300 single node DeepSeek v4 SGLang LOW LATENCY ONLY (#…
a0ddd19 
…1143)

* feat: add DeepSeek-V4-Flash FP4 B300 SGLang benchmark

Adds dsv4-fp4-b300-sglang config, single-node benchmark script, and
perf-changelog entry for the DeepSeek-V4 recipe from the SGLang
cookbook. The cookbook ships a B200 (not B300) recipe, so this
reuses the B200 Flash Low-Latency recipe on B300 until a
B300-specific recipe lands. Speculative decoding (EAGLE) and prefix
caching are disabled per request.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: switch dsv4-fp4-b300-sglang to Pro + Max-Throughput recipe

Match parallelism (TP=8/EP=8/dp-attn=true) and concurrency ranges
(4-1024 for 1k1k, 4-512 for 8k1k) to dsv4-fp4-b200-vllm. Use the
DeepSeek-V4-Pro variant with the cookbook Max-Throughput recipe
(DP=8 + DeepEP, no MTP), which aligns with the requested no-spec
parallelism. Prefix caching remains disabled.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: sync launch_b200-dgxc-slurm.sh cache mount from claude/add-dsv4-fp4-b200-vllm

Port the HF cache mount rework from the DSV4 B200 VLLM branch so
both PRs stay consistent: use the shared /scratch/fsw/gharunners/hf-hub-cache
path, drop the local MODEL override, and mount onto \$HF_HUB_CACHE
inside the container.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: restore trailing whitespace stripped from glm5.1 changelog entry

The dsv4-fp4-b300-sglang entry was appended correctly, but the earlier
edit also stripped trailing spaces on an existing line, producing a
spurious deletion. Revert so the diff is additive-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: add flock-guarded squash import to B300 runner

Mirror the lockfile logic already in launch_b200-dgxc-slurm.sh and
launch_h200-dgxc-slurm.sh: serialize concurrent enroot imports of
the same squash file via flock, skip the import when the squash is
already valid, and override ENROOT_CACHE_PATH to avoid permission
issues with the system-wide cache on worker nodes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: drop ENROOT_CACHE_PATH override from B300 runner

The override ("avoid permission issues with system-wide cache on
worker nodes") is a dgxc-slurm-specific workaround; launch_b300-nv.sh
is on the NV slurm cluster, not dgxc-slurm. Copying it in caused
the benchmark srun's pyxis shadow hook to fail with
'mkdir: cannot create directory pyxis_$JOBID.1/data: File exists'.
Keep the flock + skip-if-valid logic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: point B300 runner at shared gharunners/{squash,hf-hub-cache}

Move the squash cache from /data/squash to /data/home/sa-shared/gharunners/squash,
and the HF cache mount from /scratch/models to /data/home/sa-shared/gharunners/hf-hub-cache.
Also mount the host HF cache onto \$HF_HUB_CACHE inside the container so
tools reading the default HF path pick it up (matches the B200 dgxc-slurm
runner). Drop the /scratch/models Qwen3.5 path override since that path
is no longer used.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: move enroot import out of srun to avoid pyxis namespace collision

Running two srun steps in the same allocation (flock+import, then the
benchmark --container-image srun) reproducibly fails on this cluster
with:
  error: pyxis: mkdir: cannot create directory
    '/scratch/data/user-$UID/pyxis_$JOBID.1/data': File exists
  error: pyxis:     [ERROR] /etc/enroot/hooks.d/10-shadow.sh exited with return code 1

Per NVIDIA/pyxis#138, two srun steps sharing an allocation can leave
enroot/pyxis state between steps. Collapsing to a single srun (the
benchmark) is the cleanest workaround. Move the flock-guarded
enroot import to the host side, before salloc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: wipe stale pyxis scratch dirs for this JOB_ID before benchmark srun

Even with a single srun step, pyxis fails with
  error: pyxis: mkdir: cannot create directory
      '/scratch/data/user-$UID/pyxis_$JOBID.0/data': File exists
on fresh SLURM JOB_IDs. The /scratch path is left behind by previous
jobs whose IDs SLURM later reuses (and the cluster's pyxis epilog
doesn't clean it up). Wipe pyxis_$JOBID.* from the host after salloc;
no-op if /scratch is node-local, effective if it's shared NFS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Revert: drop all B300 runner changes, mirror #1128's approach

PR #1128 (dsv4-fp4fp8-b300-vllm) runs on the same cluster with ZERO
changes to launch_b300-nv.sh. The pyxis 10-shadow.sh failures we were
chasing aren't caused by the runner -- reset it to origin/main and
keep the sglang config/bench additions only.

Reverts (from this branch):
- 4bb1f1a point B300 runner at shared gharunners/{squash,hf-hub-cache}
- 106deea drop ENROOT_CACHE_PATH override
- 97a488e add flock-guarded squash import
- 744c5a0 move enroot import out of srun
- d003c59 wipe stale pyxis scratch before benchmark srun

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* runner: add head-node flock-guarded squash import on B300

Move enroot import out of srun to the head node and serialize parallel
GH jobs with flock on the shared squash file. Skips the import when a
valid squash already exists. The benchmark srun is now the only step
in the allocation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: mount at /ix and clear baked-in CUDA_VISIBLE_DEVICES

Port the B200 branch's fix for the lmsysorg/sglang:deepseek-v4-blackwell
image on B300:
- The image installs sglang editable under /workspace/sglang; the default
  $GITHUB_WORKSPACE:/workspace/ bind-mount masks the install and breaks
  'import sglang'. For this image, mount at /ix instead.
- The image's ENV bakes CUDA_VISIBLE_DEVICES=4,5,6,7, masking half the
  GPUs Slurm allocates. Unset it in the bench script so TP=8 sees all 8.
- Write artefacts under $PWD instead of hard-coded /workspace.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* runner: use /data/models pre-staged path for dsv4 on B300

Pre-staged models on the B300 cluster live under /data/models
(Qwen3.5-397B-A17B-FP8, dsv4-pro, etc.). Switch HF_HUB_CACHE_MOUNT
from /scratch/models to /data/models, and export MODEL to
/data/models/dsv4-pro when MODEL_PREFIX=dsv4 so the benchmark reads
from the mounted dir directly. The bench script skips `hf download`
when MODEL looks like an absolute path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: switch B300 dsv4 sglang to bw-ultra-compiled image

The stock lmsysorg/sglang:deepseek-v4-blackwell image ships kernels
compiled for B200 (SM_100) and crashes on B300 with
  RuntimeError: RMSNorm failed with error code no kernel image is
  available for execution on the device
during CUDA graph capture. Switch to cquil/sglang-deepseek-v4-bw-ultra:v1,
which is recompiled with B300 SM support.

Broaden the /ix mount conditional to match both image tags: the fork
keeps the same /workspace/sglang editable install that would otherwise
be masked by $GITHUB_WORKSPACE:/workspace/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: switch B300 dsv4 sglang image to yhyang201/sglang-b300:v3

Use the B300-recompiled image from yhyang201; extend the /ix mount
conditional to match the new tag in addition to the previous
deepseek-v4-blackwell / deepseek-v4-bw-ultra patterns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* update b300

* feat(dsv4-fp4-b300-sglang): pick recipe by CONC; split search-space

Mirror chore/dsv4-sgl-b200 commits 103a202 + 43be495 for B300:

Bench script now selects one of three cookbook recipes by CONC instead
of a single static flag set:
  CONC <= 32   -> low-latency    (TP only, chunked-prefill 4096,
                                  disable-flashinfer-autotune)
  33..128      -> balanced       (+ DP-attention, max-running-reqs=128,
                                  cuda-graph-max-bs=64, deepep-config)
  CONC > 128   -> max-throughput (+ DP-attention, max-running-reqs=256,
                                  cuda-graph-max-bs=64, deepep-config)
No speculative decoding in any recipe; --disable-radix-cache kept for
the no-prefix-caching baseline.

Split the dsv4-fp4-b300-sglang search-space rows per recipe boundary so
result filenames (ep=, dpa=) accurately reflect which recipe ran.
ep=8 on balanced/max-throughput reflects sglang's implicit
ep_size=tp_size override when --moe-a2a-backend deepep is set.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* update b300

Switch B300 dsv4 sglang image to lmsysorg/sglang:deepseek-v4-b300
and extend the /ix mount conditional to match the new tag.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dsv4-fp4-b300-sglang): low-latency recipe at every CONC (fallback)

The DeepEP FP8 weight-postprocess path is broken for
deepseek-ai/DeepSeek-V4-Pro on B300 with
lmsysorg/sglang:deepseek-v4-b300 -- every sglang launch with
--moe-a2a-backend deepep fails during model load with
  RuntimeError: Recipe must be a list/tuple of 3 integers.
raised from sglang.srt.layers.quantization.fp8
.process_weights_after_loading_block_quant (fp8.py:957).

Hardcode the bench script to the low-latency recipe flags at every
CONC (drop the CONC-based dispatch) and collapse the yaml search-space
back to a single row spanning CONC 4..1024 / 4..512 so the full sweep
still runs, just without DP-attn or DeepEP.

TODO(Cam) noted in both files to revert to the recipe-per-CONC dispatch
on chore/dsv4-sgl-b300 once sglang can load the checkpoint under
--moe-a2a-backend deepep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: align perf-changelog and config comments with low-latency fallback

The perf-changelog entry and outer NOTE comment in nvidia-master.yaml
described the max-throughput recipe from #1132, not the low-latency
fallback this PR actually adds. Rewrite both to match the actual config:
TP=8/EP=1, no DP-attn, no DeepEP, image deepseek-v4-b300, pr-link #1143.

Co-authored-by: Cameron Quilici <cquil11@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Cameron Quilici <cquil11@users.noreply.github.com>
@cquil11 cquil11 changed the title [NVIDIA][needs fixing] chore: B300 single node DeepSeek v4 SGLang [NVIDIA] chore: B300 single node DeepSeek v4 SGLang Apr 25, 2026
@cquil11 cquil11 mentioned this pull request Apr 25, 2026
Closed
3 tasks
cquil11 and others added 2 commits April 25, 2026 01:40
@cquil11 @claude
Move dsv4 b300 sglang bench script to framework-tagged path
623baa1 
Per the runner naming convention introduced in #1146
(BENCH_SCRIPT="${BENCH_BASE}_${FRAMEWORK}${SPEC_SUFFIX}.sh"), the b300
runner now prefers benchmarks/single_node/dsv4_fp4_b300_sglang.sh over
the legacy dsv4_fp4_b300.sh. The merge from main left this branch with
both scripts: the legacy file carrying the recipe-per-CONC dispatch
this PR added, and the framework-tagged file with the low-latency-only
fallback content from main. CI was therefore picking the wrong script.

Move the recipe-per-CONC dispatch onto dsv4_fp4_b300_sglang.sh and
delete the legacy filename so the runner picks up the intended logic.
Update the yaml comment to point at the new path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 @claude
chore(perf-changelog): tighten dsv4-fp4-b300-sglang entry
54b2ced 
Now that DeepEP FP8 loads cleanly, this PR is purely about restoring
the recipe-per-CONC split on top of the low-latency-only fallback
from #1143. Trim the changelog to that delta.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cquil11 cquil11 merged commit c52a8e9 into main Apr 25, 2026
2 checks passed
@cquil11 cquil11 deleted the chore/dsv4-sgl-b300 branch April 25, 2026 06:44
@claude claude Bot mentioned this pull request Apr 26, 2026
Merged
@cquil11 cquil11 mentioned this pull request Apr 26, 2026
Merged
2 tasks
cquil11 added a commit that referenced this pull request Apr 26, 2026
@cquil11 @claude
dsv4-fp4-b300-sglang: revert to #1143 low-latency-only baseline (#1184)
0f4e42f 
* dsv4-fp4-b300-sglang: revert to #1143 low-latency-only baseline

Reverts the matrix expansion (#1132), script edits (#1158, #1173, #1174),
and changelog retriggers (#1178) on top of the original #1143 entry.
Restores the script and config block to their #1143 state and clears
all prior dsv4-fp4-b300-sglang changelog entries to start fresh.

The dsv4-fp4-b300-sglang-mtp config (#1166) is untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf-changelog: add pr-link for #1184

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf-changelog: keep only the original #1143 entry, drop new entry

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kedarpotdar-nv kedarpotdar-nv Awaiting requested review from kedarpotdar-nv kedarpotdar-nv is a code owner

@jgangani jgangani Awaiting requested review from jgangani jgangani is a code owner

Assignees

No one assigned

Labels

full-sweep-enabled

Projects

InferenceMAX Board
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@cquil11