Skip to content

support one more mxfp4 variant - #2132

Merged
wenhuach21 merged 3 commits into
mainfrom
mxfp_v
Aug 7, 2026
Merged

support one more mxfp4 variant#2132
wenhuach21 merged 3 commits into
mainfrom
mxfp_v

Conversation

@wenhuach21

@wenhuach21 wenhuach21 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

for model_name in  "/models/Qwen3-8B"   "/models/Llama-3.1-8B-Instruct"
do
  CUDA_VISIBLE_DEVICES=$device python3 -m auto_round \
  --model_name  $model_name \
  --scheme mxfp4_rceil \
  --data_type mx_fp4_rceil_v2  \
  --act_data_type mx_fp4_rceil_v2 \
  --group_size 32 \
  --format fake \
  --iters 200 \
  --enable_torch_compile \
  --eval_bs $eval_bs \
  --tasks 'lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,mmlu,gsm8k' \
  --output_dir "/data2/wenhuach/mxfp4_rceil_tuning" \
  2>&1 | tee -a test_rceil_tuning.txt
done

Type of Change

Bug fix

Related Issues

Fixes or relates to #

Checklist Before Submitting

  • My code has been tested locally.
  • Documentation has been updated as needed.
  • New or updated tests are included where applicable.
  • The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.

@wenhuach21 wenhuach21 changed the title support one more mxfp4 varient support one more mxfp4 variant Aug 6, 2026
@AutoRoundBot

Copy link
Copy Markdown
Collaborator

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@wenhuach21
wenhuach21 requested a review from xin3he August 7, 2026 01:52

@xin3he xin3he left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenhuach21
wenhuach21 merged commit f706da6 into main Aug 7, 2026
45 checks passed
@wenhuach21
wenhuach21 deleted the mxfp_v branch August 7, 2026 03:18
jijiaz pushed a commit to jijiaz/auto-round that referenced this pull request Aug 8, 2026
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants