[Feature]: Quantize/save/evaluate the ByteDance-Seed/BAGEL-7B-MoT in w4a16 format #1608
Description
Feature Description
Quantize/save/evaluate the ByteDance-Seed/BAGEL-7B-MoT in w4a16 format
Motivation and Use Case
Model: https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT
Target dtypes: w4a16
Save the quantized model for vllm-omni.
Alternatives Considered
No response
Definition of Done
No response
Additional Context
No response
Metadata
Metadata
Assignees
Labels
No labels