[WIP] [AMD/ROCM] atom minimaxm2.5 fp4 on mi355x

[seungrokj]

hi,

WIP.
internally tested. shipping soon.

cc. @ChangLiu0709 @andyluo7 @chunfangamd @ajith-sirra-amd

Regards,
Seungrok

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://cookbook.sglang.io/
• https://docs.vllm.ai/projects/recipes/en/latest/

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sgl-cookbook

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://cookbook.sglang.io/
• https://docs.vllm.ai/projects/recipes/en/latest/

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sgl-cookbook

[functionstackx]

plz submit vllm minimax fp4 first. as an reminder. atom can be submit as additional framework after u submit vllm first for inferencev3

[benenzhu]

plz submit vllm minimax fp4 first. as an reminder. atom can be submit as additional framework after u submit vllm first for inferencev3

@functionstackx Hi for vllm minimax fp4, #827 can be merged using the nightly vllm image of vllm/vllm-openai-rocm:nightly-c48b2b83bd160cd684c8a2357f229e90de99298d, I have checked the accuracy and benchmark all passed.

[seungrokj]

plz submit vllm minimax fp4 first. as an reminder. atom can be submit as additional framework after u submit vllm first for inferencev3

@functionstackx Hi for vllm minimax fp4, #827 can be merged using the nightly vllm image of vllm/vllm-openai-rocm:nightly-c48b2b83bd160cd684c8a2357f229e90de99298d, I have checked the accuracy and benchmark all passed.

hi @benenzhu once
v0.19.1 img (https://hub.docker.com/r/vllm/vllm-openai-rocm/tag) is available, can you create vllm+minimaxm2.5+fp4 PR ?
#827 (comment)

[benenzhu]

plz submit vllm minimax fp4 first. as an reminder. atom can be submit as additional framework after u submit vllm first for inferencev3

@functionstackx Hi for vllm minimax fp4, #827 can be merged using the nightly vllm image of vllm/vllm-openai-rocm:nightly-c48b2b83bd160cd684c8a2357f229e90de99298d, I have checked the accuracy and benchmark all passed.

hi @benenzhu once v0.19.1 img (https://hub.docker.com/r/vllm/vllm-openai-rocm/tag) is available, can you create vllm+minimaxm2.5+fp4 PR ? #827 (comment)

@seungrokj Yeah I will update it to the release img.

[functionstackx]

plz submit vllm minimax fp4 first. as an reminder. atom can be submit as additional framework after u submit vllm first for inferencev3

@functionstackx Hi for vllm minimax fp4, #827 can be merged using the nightly vllm image of vllm/vllm-openai-rocm:nightly-c48b2b83bd160cd684c8a2357f229e90de99298d, I have checked the accuracy and benchmark all passed.

Do u have link to full sweep pr validation pass + link to vllm recipe PR (if it needs any updating)