Skip to content

[NV] H200 GLM5 fp8 update sglang container#1033

Merged
functionstackx merged 11 commits into
mainfrom
nv/glm5-fp8-h200-sglang-v2
May 19, 2026
Merged

[NV] H200 GLM5 fp8 update sglang container#1033
functionstackx merged 11 commits into
mainfrom
nv/glm5-fp8-h200-sglang-v2

Conversation

@hshrivastava-droid
Copy link
Copy Markdown
Collaborator

@hshrivastava-droid hshrivastava-droid commented Apr 15, 2026

Summary

Update the GLM-5 FP8 H200 SGLang benchmark configuration and launch script:

  • SGLang image: Change from lmsysorg/sglang:v0.5.12-cu130 to lmsysorg/sglang:v0.5.11-cu129
  • Runner: Switch from h200 to h200-dgxc
  • Launch option: Add --enable-flashinfer-allreduce-fusion to the server launch command for improved allreduce performance

Changed Files

File Change
.github/configs/nvidia-master.yaml Updated image tag and runner type for glm5-fp8-h200-sglang
benchmarks/single_node/glm5_fp8_h200.sh Added --enable-flashinfer-allreduce-fusion flag
perf-changelog.yaml Added changelog entry for these config changes

Validation

@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

3 similar comments
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

@hshrivastava-droid hshrivastava-droid changed the title [WIP][NV] update sglang container [WIP][NV] GLM5 fp8 update sglang container Apr 16, 2026
@Ankur-singh Ankur-singh changed the title [WIP][NV] GLM5 fp8 update sglang container [Do Not Merge][NV] GLM5 fp8 update sglang container Apr 17, 2026
@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

Updated SGLang image version and added server launch option.
@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

3 similar comments
@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

@hshrivastava-droid hshrivastava-droid changed the title [Do Not Merge][NV] GLM5 fp8 update sglang container [NV] GLM5 fp8 update sglang container May 19, 2026
@hshrivastava-droid
Copy link
Copy Markdown
Collaborator Author

@claude add PR description

@Klaud-Cold
Copy link
Copy Markdown
Collaborator

Klaud-Cold commented May 19, 2026

Claude finished @hshrivastava-droid's task in 37s —— View job


Adding PR Description

  • Gather context and analyze changes
  • Write and update PR description

Added a PR description covering the three key changes in this PR:

  1. SGLang image update (v0.5.12-cu130v0.5.11-cu129)
  2. Runner switch (h200h200-dgxc)
  3. Added --enable-flashinfer-allreduce-fusion launch flag

Also included links to the validation run and a summary table of changed files.

@kedarpotdar-nv kedarpotdar-nv changed the title [NV] GLM5 fp8 update sglang container [NV] H200 GLM5 fp8 update sglang container May 19, 2026
@hshrivastava-droid hshrivastava-droid changed the title [NV] H200 GLM5 fp8 update sglang container [WIP][NV] H200 GLM5 fp8 update sglang container May 19, 2026
@github-actions
Copy link
Copy Markdown
Contributor

@hshrivastava-droid
Copy link
Copy Markdown
Collaborator Author

sglang cookbook- sgl-project/sgl-cookbook#276

@hshrivastava-droid
Copy link
Copy Markdown
Collaborator Author

@functionstackx could you please help reviewing this

@Ankur-singh Ankur-singh changed the title [WIP][NV] H200 GLM5 fp8 update sglang container [NV] H200 GLM5 fp8 update sglang container May 19, 2026
Copy link
Copy Markdown
Collaborator

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u doc update to correct repo https://github.com/sgl-project/sgl-cookbook
is depreipcated

https://github.com/sgl-project/sglang/tree/main/docs_new
is the new repo

@hshrivastava-droid
Copy link
Copy Markdown
Collaborator Author

updated sglang receipe- sgl-project/sglang#25814
@functionstackx

@functionstackx
Copy link
Copy Markdown
Collaborator

/reuse-sweep-run

@functionstackx functionstackx merged commit 475218b into main May 19, 2026
3 of 5 checks passed
@functionstackx functionstackx deleted the nv/glm5-fp8-h200-sglang-v2 branch May 19, 2026 23:05
@github-actions
Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

4 participants