Skip to content

Supported Models

Jump to bottom
Qubitium edited this page Nov 3, 2026 · 3 revisions

Supported Models

Verified against revision 55a3daf of ModelCloud/GPTQModel (GPT-QModel 7.8.0).

GPT-QModel resolves a checkpoint's config.json model_type string to a BaseQModel definition through the MODEL_MAP registry in gptqmodel/models/auto.py. The registry holds 195 model_type keys mapped to 171 definition classes. This page lists every supported model: the creator, official base weights on Hugging Face, a short summary, and the definition class declaration (the "model creator" code snippet) as it appears in gptqmodel/models/definitions/.

Tags used below: MoE = mixture-of-experts architecture; dynamic experts = expert count read from each checkpoint's config (dynamic_expert_index); multimodal = vision/audio/omni checkpoint; trust_remote_code = the definition sets require_trust_remote_code = True (custom checkpoint code permitted).

Index

model_type key(s) Definition class Creator Base weights (Hugging Face)
afmoe AfMoeQModel — —
agnes AgnesQModel Agnes AI Agnes-AI/Agnes-2.5-Flash-Base, Agnes-AI/Agnes-3.0-Flash
apertus1p5 Apertus1p5QModel Swiss AI Initiative (EPFL/ETH) swiss-ai/Apertus-8B-Instruct-2509, swiss-ai/Apertus-70B-Instruct-2509
apertus1p5_text Apertus1p5TextQModel Swiss AI Initiative (EPFL/ETH) swiss-ai/Apertus-8B-Instruct-2509, swiss-ai/Apertus-70B-Instruct-2509
apertus ApertusQModel Swiss AI Initiative (EPFL/ETH) swiss-ai/Apertus-8B-Instruct-2509, swiss-ai/Apertus-70B-Instruct-2509
axk2 AXK2QModel SK Telecom (A.X) skt/A.X-K2
baichuan BaiChuanQModel Baichuan baichuan-inc/Baichuan2-13B-Chat
bailing_moe, bailing_hybrid BailingMoeQModel inclusionAI (Ant Group) inclusionAI/Ling-2.6-flash
bloom BloomQModel BigScience bigscience/bloom
brumby BrumbyQModel Manifest AI manifestai/Brumby-14B-Base, manifestai/Brumby-4B-Base
chatglm ChatGLMQModel Zhipu AI / THUDM THUDM/chatglm3-6b, THUDM/glm-4-9b
codegen CodeGenQModel Salesforce Salesforce/codegen-2B-multi
cohere2_moe Cohere2MoeQModel Cohere CohereLabs/c4ai-command-a-03-2025, CohereLabs/command-a-vision-07-2025
cohere_compass CohereCompassQModel Cohere —
dbrx_converted DbrxConvertedQModel Databricks alpindale/dbrx-instruct
dbrx DbrxQModel Databricks alpindale/dbrx-instruct
deci, nemotron-nas DeciLMQModel Deci / NVIDIA Deci/DeciLM-7B, Deci/DeciLM-6b-instruct, nvidia/Llama-3_1-Nemotron-Ultra-253B-v1
deepseek_ocr2 DeepSeekOCR2QModel DeepSeek deepseek-ai/DeepSeek-OCR-2
deepseek_v2 DeepSeekV2QModel DeepSeek deepseek-ai/DeepSeek-V2, deepseek-ai/DeepSeek-V2-Lite
deepseek_v32 DeepSeekV32QModel DeepSeek deepseek-ai/DeepSeek-V3.2-Exp
kimi_k2, deepseek_v3 DeepSeekV3QModel DeepSeek deepseek-ai/DeepSeek-V3-0324, deepseek-ai/DeepSeek-V3.1
deepseek_v4 DeepSeekV4QModel DeepSeek deepseek-ai/DeepSeek-V4-Flash, deepseek-ai/DeepSeek-V4-Pro
deepseek_vl DeepSeekVLQModel DeepSeek deepseek-ai/deepseek-vl-1.3b-base
deepseek_vl_v2 DeepSeekVLV2QModel DeepSeek deepseek-ai/deepseek-vl2
diffusion_gemma DiffusionGemmaQModel Google DeepMind —
dots1 Dots1QModel dots studio (Xiaohongshu) dots-studio/dots.llm1.base, dots-studio/dots.llm1.inst
dream DreamQModel Dream team (HKU / Huawei Noah) Dream-org/Dream-v0-Instruct-7B
ernie4_5_moe Ernie4_5_MoeQModel Baidu baidu/ERNIE-4.5-21B-A3B-PT
ernie4_5_moe_vl, ernie4_5_vl_moe Ernie4_5_VLMoeQModel Baidu baidu/ERNIE-4.5-VL-28B-A3B-PT
ernie4_5 Ernie4_5QModel Baidu baidu/ERNIE-4.5-0.3B-Base-PT
exaone4_5 Exaone4_5QModel LG AI Research LGAI-EXAONE/EXAONE-4.5-33B
exaone4, exaone4_5_text Exaone4QModel LG AI Research LGAI-EXAONE/EXAONE-4.0-32B
exaone ExaOneQModel LG AI Research LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct
falcon_h1 FalconH1QModel Technology Innovation Institute (TII) tiiuae/falcon-h1-7b-base
falcon_mamba FalconMambaQModel Technology Innovation Institute (TII) tiiuae/falcon-mamba-7b, tiiuae/Falcon3-7B-Instruct, tiiuae/falcon-40b
flint FlintQModel RecursiveSI RecursiveSI/flint
gemma2 Gemma2QModel Google google/gemma-2-27b-it
gemma3 Gemma3ForConditionalGenerationGPTQ Google google/gemma-3-27b-it
gemma3n Gemma3nForConditionalGenerationGPTQ Google google/gemma-3n-E4B-it
gemma3n_text Gemma3nTextQModel Google google/gemma-3n-E4B-it
gemma3_text Gemma3QModel Google google/gemma-3-27b-it
gemma4 Gemma4ForConditionalGenerationGPTQ Google google/gemma-4-e2b-it
gemma4_text Gemma4TextQModel Google google/gemma-4-e2b-it
gemma4_unified Gemma4UnifiedForConditionalGenerationGPTQ Google google/gemma-4-e4b-it
gemma4_unified_text Gemma4UnifiedTextQModel Google google/gemma-4-e4b-it
glm4_moe GLM4MoEGPTQ Z.ai (Zhipu) zai-org/GLM-4.5-Air, zai-org/GLM-4.7-Flash
glm4_moe_lite Glm4MoeLiteQModel Z.ai (Zhipu) zai-org/GLM-4.5-Air
glm4v Glm4vGPTQ Z.ai (Zhipu) zai-org/GLM-4.6V-Flash
glm4v_moe Glm4vMoeQModel Z.ai (Zhipu) zai-org/GLM-4.5V
glm4v_moe_text Glm4vMoeTextQModel Z.ai (Zhipu) zai-org/GLM-4.5V
glm5_next Glm5NextQModel Z.ai (Zhipu) zai-org/GLM-5.3-Flash
glmasr GlmASRGPTQ Z.ai (Zhipu) zai-org/GLM-ASR-Nano-2512
glm_moe_dsa GlmMoeDsaQModel Z.ai (Zhipu) zai-org/GLM-4.7-Flash
glm_ocr GlmOCRGPTQ Z.ai (Zhipu) zai-org/GLM-OCR
glm, glm4 GlmQModel Zhipu AI THUDM/glm-4-9b
gpt2 GPT2QModel OpenAI openai-community/gpt2
gpt_bigcode GptBigCodeQModel BigCode bigcode/starcoder
gptj GptJQModel EleutherAI EleutherAI/gpt-j-6b
gpt_neo GptNeoQModel EleutherAI EleutherAI/gpt-neo-1.3B
gpt_neox GPTNeoXQModel EleutherAI EleutherAI/gpt-neox-20b, EleutherAI/pythia-12b
gpt_oss GPTOSSGPTQ OpenAI openai/gpt-oss-20b
granitemoehybrid GraniteMoeHybridQModel IBM ibm-granite/granite-4.0-h-small, ibm-granite/granite-4.0-tiny-preview
granitemoe GraniteMoeQModel IBM ibm-granite/granite-3.0-1b-a400m-instruct
grinmoe GrinMoeQModel Microsoft microsoft/Phi-3.5-MoE-instruct
hrm_text HrmTextQModel Sapient Intelligence sapientinc/HRM-Text-1B
hunyuan_v1_dense HunYuanDenseV1QModel Tencent tencent/HunYuan-7B-Instruct
hunyuan_v1_moe HunYuanMoEV1QModel Tencent tencent/Hunyuan-A13B-Instruct
hunyuan_vl HunYuanVLQModel Tencent —
hymba HymbaQModel NVIDIA nvidia/Hymba-1.5B-Base
hy_v3 HYV3QModel Tencent —
inkling_mm_model InklingMMQModel Thinking Machines Lab thinkingmachines/Inkling
instella InstellaQModel AMD amd/Instella-3B-Instruct
internlm2 InternLM2QModel Shanghai AI Laboratory internlm/internlm2-chat-7b
internlm InternLMQModel Shanghai AI Laboratory internlm/internlm-7b
interns1 InternS1QModel Shanghai AI Laboratory internlm/Intern-S1, internlm/Intern-S1-mini
intern_s2_preview InternS2PreviewQModel Shanghai AI Laboratory internlm/Intern-S2-Preview
internvl_chat InternVLChatQModel OpenGVLab OpenGVLab/InternVL-Chat-V1-5
internvl InternVLQModel OpenGVLab OpenGVLab/InternVL3-14B, OpenGVLab/InternVL3_5-14B
jamba JambaQModel AI21 Labs ai21labs/AI21-Jamba-Large-1.5, AI21Labs/Jamba-v0.1
k2_horizon K2HorizonQModel IFM IFM/K2-Horizon-7B, IFM/K2-Horizon-MoVA-36B-A4B
kimi_k25 KimiK25QModel Moonshot AI moonshotai/Kimi-K2.5
klear KlearQModel Kuaishou Kwai-Klear Kwai-Klear/Klear-46B-A2.5B-Base
laguna LagunaQModel poolside poolside/Laguna-XS-2.1, poolside/Laguna-XS.2-FP8
lfm2_moe LFM2MoeQModel Liquid AI LiquidAI/LFM2-8B-A1B
lfm2 LFM2QModel Liquid AI LiquidAI/LFM2-1.2B
lfm2_vl LFM2VLQModel Liquid AI LiquidAI/LFM2.5-VL-1.6B
lfm3_omni LFM3OmniQModel Liquid AI LiquidAI/LFM3-Omni-2B
lfm3 LFM3QModel Liquid AI LiquidAI/LFM3-1.7B
lfm3_vl LFM3VLQModel Liquid AI LiquidAI/LFM3-VL-2B
llada2_moe LLaDA2MoeQModel inclusionAI (Ant Group) inclusionAI/LLaDA2.2-mini, inclusionAI/LLaDA2.2-flash
llama4 Llama4QModel Meta meta-llama/Llama-4-Scout-17B-16E-Instruct
llama4_text Llama4TextQModel Meta meta-llama/Llama-4-Scout-17B-16E-Instruct
llama, smollm3, cohere, cohere2, mistral, yi, stablelm_epoch, stablelm, longllama, gemma, granite, olmo2, seed_oss LlamaQModel Meta meta-llama/Meta-Llama-3-8B
llava LlavaQModel LLaVA team llava-hf/llava-1.5-7b-hf
llava_qwen2 LlavaQwen2QModel LLaVA / lmms-lab lmms-lab/LLaVA-Video-7B-Qwen2
locateanything LocateAnythingQModel LocateAnything —
longcat_flash LongCatFlashQModel Meituan LongCat meituan-longcat/LongCat-Flash-Chat
mage_vl MageVLQModel Mage —
mimo MimoQModel Xiaomi MiMo XiaomiMiMo/MiMo-7B-Base, XiaomiMiMo/MiMo-VL-7B-RL
mimo_v2 MimoV2QModel Xiaomi MiMo XiaomiMiMo/MiMo-V2.6-Flash-RL
minicpm3 MiniCpm3QModel OpenBMB openbmb/MiniCPM3-4B
minicpm MiniCPMGPTQ OpenBMB openbmb/MiniCPM-2B-sft-bf16
minicpmo MiniCPMOQModel OpenBMB OpenBMB/MiniCPM-o-2_6
minicpmv4_6 MiniCPMV4_6QModel OpenBMB openbmb/MiniCPM-V-4_5
minicpmv MiniCPMVQModel OpenBMB openbmb/MiniCPM-Llama3-V-2_5, openbmb/MiniCPM-V-2_6
minimax, minimax_m2 MiniMaxM2GPTQ MiniMax MiniMaxAI/MiniMax-M2
minimax_m3_vl MiniMaxM3VLGPTQ MiniMax MiniMaxAI/MiniMax-M3.1-Flash, MiniMaxAI/MiniMax-M3
ministral3 Ministral3GPTQ Mistral AI mistralai/Ministral-3-8B-Instruct-2512
mistral3 Mistral3GPTQ Mistral AI mistralai/Mistral-Small-3.2-24B-Instruct-2506, mistralai/Mistral-Small-3.1-24B-Instruct-2503
mixtral MixtralQModel Mistral AI mistralai/Mixtral-8x7B-v0.1
mllama MLlamaQModel Meta meta-llama/Llama-3.2-11B-Vision-Instruct
mllama_text_model MLlamaTextQModel Meta meta-llama/Llama-3.2-11B-Vision-Instruct
mobilellm MobileLLMQModel Meta (FAIR) facebook/MobileLLM-125M
moss MossQModel Fudan / OpenLMLab fnlp/MOSS-MOON-003-Base, fnlp/moss-moon-003-sft
mpt MptQModel MosaicML TehVenom/mpt-7b-Apache-2.0
muse_glimmer MuseGlimmerQModel Meta Models meta-models/Muse-Glimmer-30B
nanbeige NanbeigeQModel Nanbeige Nanbeige/Nanbeige4.2-3B
nemotron_h_puzzle NemotronHPuzzleQModel NVIDIA —
nemotron_h NemotronHQModel NVIDIA nvidia/Nemotron-H-8B-Reasoning-128K, nvidia/Nemotron-H-8B-Base-8K
nemotron_labs_diffusion NemotronLabsDiffusionQModel NVIDIA nvidia/Nemotron-Labs-Diffusion-8B-Base
nemotronh_nano_omni_reasoning_v3 NemotronOmniQModel NVIDIA nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16
olmo3 Olmo3QModel Ai2 allenai/Olmo-3-1025-7B
olmoe OlmoeQModel Ai2 allenai/OLMoE-1B-7B-0924
openvla OpenVLAQModel OpenVLA openvla/openvla-7b
opt OptQModel Meta (FAIR) facebook/opt-1.3b
ouro OuroQModel ByteDance Seed ByteDance/Ouro-1.4B, ByteDance/Ouro-2.6B
ovis2_5 Ovis2_5QModel AIDC-AI (Alibaba) AIDC-AI/Ovis2.5-9B, AIDC-AI/Ovis2.5-2B
ovis2_6_moe Ovis2_6_MoeQModel AIDC-AI / ATH-MaaS ATH-MaaS/Ovis2.6-30B-A3B, ATH-MaaS/Ovis2.6-80B-A3B
ovis2_6_next Ovis2_6_NextQModel AIDC-AI / ATH-MaaS ATH-MaaS/Ovis2.6-30B-A3B, ATH-MaaS/Ovis2.6-80B-A3B
ovis2 Ovis2QModel AIDC-AI (Alibaba) AIDC-AI/Ovis2-8B
ovis OvisQModel AIDC-AI (Alibaba) AIDC-AI/Ovis1.6-Gemma2-9B
gpt_pangu PanguAlphaQModel Huawei Noah's Ark Lab ModelCloud/pangu_alpha_2_6B
phi3 Phi3QModel Microsoft microsoft/Phi-3-mini-4k-instruct
phi4mm Phi4MMGPTQ Microsoft microsoft/phi-4-multimodal-instruct
phimoe PhiMoEGPTQForCausalLM Microsoft microsoft/Phi-3.5-MoE-instruct
phi PhiQModel Microsoft microsoft/phi-2
qwen2_5_omni Qwen2_5_OmniGPTQ Alibaba Qwen Qwen/Qwen2.5-Omni-7B
qwen2_5_vl, qwen2_5_vl_text Qwen2_5_VLQModel Alibaba Qwen Qwen/Qwen2.5-VL-7B-Instruct
qwen2_audio Qwen2AudioGPTQ Alibaba Qwen Qwen/Qwen2-Audio-7B-Instruct
qwen2_moe Qwen2MoeQModel Alibaba Qwen Qwen/Qwen2-57B-A14B
qwen2 Qwen2QModel Alibaba Qwen Qwen/Qwen2-72B
qwen2_vl, qwen2_vl_text Qwen2VLQModel Alibaba Qwen Qwen/Qwen2-VL-7B-Instruct
qwen3_5_moe Qwen3_5_MoeQModel Alibaba Qwen Qwen/Qwen3.6-35B-A3B
qwen3_5_moe_text Qwen3_5_MoeTextQModel Alibaba Qwen Qwen/Qwen3.6-35B-A3B
qwen3_5 Qwen3_5QModel Alibaba Qwen Qwen/Qwen3.5-397B-A17B
qwen3_5_text Qwen3_5TextQModel Alibaba Qwen Qwen/Qwen3.5-397B-A17B
qwen3_vl_moe Qwen3_VL_MoeQModel Alibaba Qwen Qwen/Qwen3-VL-30B-A3B-Instruct
qwen3_vl Qwen3_VLQModel Alibaba Qwen Qwen/Qwen3-VL-8B-Instruct
qwen3_moe Qwen3MoeQModel Alibaba Qwen Qwen/Qwen3-30B-A3B
qwen3_next Qwen3NextGPTQ Alibaba Qwen Qwen/Qwen3-Next-80B-A3B-Instruct
qwen3_omni_moe Qwen3OmniMoeGPTQ Alibaba Qwen Qwen/Qwen3-Omni-30B-A3B-Instruct
qwen3, marin Qwen3QModel Alibaba Qwen Qwen/Qwen3-32B
qwen4_exp Qwen4ExpQModel Alibaba Qwen Qwen/Qwen3.8-Flash-Next
qwen4_exp_text Qwen4ExpTextQModel Alibaba Qwen Qwen/Qwen3.8-Flash-Next
qwen_drive QwenDriveQModel Alibaba Qwen —
qwen QwenQModel Alibaba Qwen Qwen/Qwen1.5-7B-Chat
raon1_mini Raon1MiniQModel KRAFTON KRAFTON/Raon-1-mini-preview
refinedWebModel, refinedWeb, falcon RwgQModel RWKV RWKV/v6-Finch-14B-HF
smolvlm SmolVLMQModel HuggingFaceTB HuggingFaceTB/SmolVLM-Instruct, HuggingFaceTB/SmolVLM-256M-Instruct
solar_open2 SolarOpen2QModel Upstage upstage/Solar-Open2-250B
solar_open SolarOpenQModel Upstage upstage/Solar-Open-100B
spark2_5 Spark2_5QModel XHToken XHToken/Spark-X2.5-4B
starcoder2 Starcoder2QModel BigCode bigcode/starcoder2-15b
telechat TeleChat2QModel TeleAI (China Telecom) Tele-AI/TeleChat2-7B-32K, Tele-AI/TeleChat2-115B
unlimited-ocr UnlimitedOCRQModel Baidu baidu/Unlimited-OCR
voxtral VoxtralGPTQ Mistral AI mistralai/Voxtral-Small-24B-2507, mistralai/Voxtral-Mini-3B-2507
xing4_0 Xing4_0QModel XingChen-AGI XingChen-AGI/Xing4.0-29B-A4B
xverse XverseQModel XVERSE xverse/XVERSE-13B-Chat
zamba2 Zamba2QModel Zyphra Zyphra/Zamba2-7B
zamba ZambaQModel Zyphra Zyphra/Zamba-7B-v1
zdtaichu5_0 ZDTaichu5QModel TaichuAI TaichuAI/ZDTaichu5.0-9B

AfMoeQModel

  • model_type key(s): afmoe
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Traits: MoE · dynamic experts (num_experts) · trust_remote_code
  • Definition: gptqmodel/models/definitions/afmoe.py
  • Summary: Sparse mixture-of-experts (afmoe) architecture; no public Hugging Face base-weights checkpoint located.
class AfMoeQModel(BaseQModel):

AgnesQModel

class AgnesQModel(BaseQModel):
    """Quantization layout for the multimodal Agnes 3.0 Flash decoder."""

Apertus1p5QModel

class Apertus1p5QModel(BaseQModel):
    """Apertus 1.5 multimodal wrapper with an Apertus language backbone."""

Apertus1p5TextQModel

class Apertus1p5TextQModel(ApertusQModel):
    """Text-only Apertus 1.5 checkpoints use the original Apertus decoder layout."""

ApertusQModel

class ApertusQModel(BaseQModel):

AXK2QModel

  • model_type key(s): axk2
  • Creator: SK Telecom (A.X)
  • Base weights (Hugging Face): skt/A.X-K2
  • Traits: MoE · dynamic experts (n_routed_experts)
  • Definition: gptqmodel/models/definitions/axk2.py
  • Summary: A.X-K2 sparse MoE/MLA model family.
class AXK2QModel(BaseQModel):
    """A.X-K2 (SKT) sparse MoE/MLA model support."""

BaiChuanQModel

class BaiChuanQModel(BaseQModel):

BailingMoeQModel

class BailingMoeQModel(BaseQModel):

BloomQModel

class BloomQModel(BaseQModel):

BrumbyQModel

class BrumbyQModel(BaseQModel):

ChatGLMQModel

class ChatGLMQModel(BaseQModel):

CodeGenQModel

class CodeGenQModel(BaseQModel):

Cohere2MoeQModel

class Cohere2MoeQModel(BaseQModel):

CohereCompassQModel

  • model_type key(s): cohere_compass
  • Creator: Cohere
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Definition: gptqmodel/models/definitions/cohere_compass.py
  • Summary: Cohere Compass VLM adapter (North Micro Vision); partner checkpoint, no public HF base weights located.
class CohereCompassQModel(BaseQModel):
    """Cohere Compass VLM adapter used by North Micro Vision."""

DbrxConvertedQModel

class DbrxConvertedQModel(BaseQModel):

DbrxQModel

class DbrxQModel(BaseQModel):

DeciLMQModel

class DeciLMQModel(LlamaQModel):

DeepSeekOCR2QModel

class DeepSeekOCR2QModel(BaseQModel):

DeepSeekV2QModel

class DeepSeekV2QModel(BaseQModel):

DeepSeekV32QModel

class DeepSeekV32QModel(BaseQModel):
    """DeepSeek-V3.2 MLA/DSA model with dense and routed/shared-MoE layers."""

DeepSeekV3QModel

class DeepSeekV3QModel(BaseQModel):

DeepSeekV4QModel

class DeepSeekV4QModel(DeepSeekV3QModel):

DeepSeekVLQModel

class DeepSeekVLQModel(BaseQModel):

DeepSeekVLV2QModel

class DeepSeekVLV2QModel(BaseQModel):

DiffusionGemmaQModel

  • model_type key(s): diffusion_gemma
  • Creator: Google DeepMind
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Traits: dynamic experts (num_experts)
  • Definition: gptqmodel/models/definitions/diffusion_gemma.py
  • Summary: Quantization definition for native Transformers DiffusionGemma checkpoints.
class DiffusionGemmaQModel(BaseQModel):
    """Quantization definition for native Transformers DiffusionGemma checkpoints."""

Dots1QModel

class Dots1QModel(BaseQModel):

DreamQModel

class DreamQModel(LlamaQModel):

Ernie4_5_MoeQModel

class Ernie4_5_MoeQModel(BaseQModel):

Ernie4_5_VLMoeQModel

class Ernie4_5_VLMoeQModel(BaseQModel):

Ernie4_5QModel

class Ernie4_5QModel(LlamaQModel):

Exaone4_5QModel

class Exaone4_5QModel(BaseQwen3VLGPTQ):
    """Quantization layout for the EXAONE 4.5 vision-language model."""

Exaone4QModel

class Exaone4QModel(BaseQModel):

ExaOneQModel

class ExaOneQModel(BaseQModel):

FalconH1QModel

class FalconH1QModel(BaseQModel):

FalconMambaQModel

class FalconMambaQModel(BaseQModel):

FlintQModel

class FlintQModel(BaseQModel):
    """RecursiveSI Flint hybrid-attention MoE model."""

Gemma2QModel

class Gemma2QModel(LlamaQModel):

Gemma3ForConditionalGenerationGPTQ

class Gemma3ForConditionalGenerationGPTQ(BaseQModel):

Gemma3nForConditionalGenerationGPTQ

class Gemma3nForConditionalGenerationGPTQ(BaseQModel):

Gemma3nTextQModel

class Gemma3nTextQModel(LlamaQModel):

Gemma3QModel

class Gemma3QModel(LlamaQModel):

Gemma4ForConditionalGenerationGPTQ

class Gemma4ForConditionalGenerationGPTQ(BaseQModel):
    """Quantization definition for composite Gemma 4 checkpoints."""

Gemma4TextQModel

class Gemma4TextQModel(LlamaQModel):
    """Quantization definition for text-only Gemma 4 checkpoints."""

Gemma4UnifiedForConditionalGenerationGPTQ

class Gemma4UnifiedForConditionalGenerationGPTQ(BaseQModel):
    """Quantization definition for Gemma 4 unified (multimodal) checkpoints."""

Gemma4UnifiedTextQModel

class Gemma4UnifiedTextQModel(BaseQModel):
    """Quantization definition for text-only Gemma 4 unified checkpoints (``gemma4_unified_text``)."""

GLM4MoEGPTQ

class GLM4MoEGPTQ(BaseQModel):

Glm4MoeLiteQModel

class Glm4MoeLiteQModel(BaseQModel):

Glm4vGPTQ

class Glm4vGPTQ(BaseQModel):

Glm4vMoeQModel

class Glm4vMoeQModel(Glm4vGPTQ):

Glm4vMoeTextQModel

class Glm4vMoeTextQModel(Glm4vMoeQModel):

Glm5NextQModel

class Glm5NextQModel(BaseQModel):
    """GLM-5.3-Flash hybrid KDA/DSA multimodal MoE model."""

GlmASRGPTQ

class GlmASRGPTQ(BaseQModel):

GlmMoeDsaQModel

class GlmMoeDsaQModel(BaseQModel):

GlmOCRGPTQ

class GlmOCRGPTQ(BaseQModel):

GlmQModel

class GlmQModel(BaseQModel):

GPT2QModel

class GPT2QModel(BaseQModel):

GptBigCodeQModel

class GptBigCodeQModel(BaseQModel):

GptJQModel

class GptJQModel(BaseQModel):

GptNeoQModel

class GptNeoQModel(BaseQModel):

GPTNeoXQModel

class GPTNeoXQModel(BaseQModel):

GPTOSSGPTQ

class GPTOSSGPTQ(BaseQModel):

GraniteMoeHybridQModel

class GraniteMoeHybridQModel(BaseQModel):

GraniteMoeQModel

class GraniteMoeQModel(BaseQModel):
    """Quantization definition for the Transformers Granite MoE decoder."""

GrinMoeQModel

class GrinMoeQModel(BaseQModel):

HrmTextQModel

class HrmTextQModel(BaseQModel):

HunYuanDenseV1QModel

class HunYuanDenseV1QModel(LlamaQModel):
    """Hunyuan Dense V1 follows a Llama-style decoder layout with per-head Q/K     RMSNorm modules inside attention. Those norms are metadata/base modules for     quantization and should not be replaced by quantized linear kernels."""

HunYuanMoEV1QModel

class HunYuanMoEV1QModel(BaseQModel):

HunYuanVLQModel

  • model_type key(s): hunyuan_vl
  • Creator: Tencent
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Definition: gptqmodel/models/definitions/hunyuan_vl.py
  • Summary: Quantization definition for native Transformers Hunyuan-VL checkpoints.
class HunYuanVLQModel(BaseQModel):
    """Quantization definition for native Transformers Hunyuan-VL checkpoints."""

HymbaQModel

class HymbaQModel(BaseQModel):

HYV3QModel

  • model_type key(s): hy_v3
  • Creator: Tencent
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Traits: MoE · dynamic experts (num_experts)
  • Definition: gptqmodel/models/definitions/hy_v3.py
  • Summary: HYV3 hybrid layout: dense first MLP layer followed by sparse MoE layers; no public HF base-weights checkpoint located.
class HYV3QModel(BaseQModel):

InklingMMQModel

class InklingMMQModel(BaseQModel):

InstellaQModel

class InstellaQModel(LlamaQModel):

InternLM2QModel

class InternLM2QModel(BaseQModel):

InternLMQModel

class InternLMQModel(LlamaQModel):

InternS1QModel

class InternS1QModel(BaseQModel):

InternS2PreviewQModel

class InternS2PreviewQModel(BaseQModel):

InternVLChatQModel

class InternVLChatQModel(BaseQModel):

InternVLQModel

class InternVLQModel(BaseQModel):
    """Quantization definition for the Transformers-native InternVL model."""

JambaQModel

class JambaQModel(BaseQModel):

K2HorizonQModel

class K2HorizonQModel(BaseQModel):
    """GPT-QModel definition for K2 Horizon dense and MoVA checkpoints."""

KimiK25QModel

class KimiK25QModel(BaseQModel):

KlearQModel

class KlearQModel(BaseQModel):

LagunaQModel

class LagunaQModel(BaseQModel):

LFM2MoeQModel

class LFM2MoeQModel(BaseQModel):

LFM2QModel

class LFM2QModel(BaseQModel):

LFM2VLQModel

class LFM2VLQModel(BaseQModel):

LFM3OmniQModel

  • model_type key(s): lfm3_omni
  • Creator: Liquid AI
  • Base weights (Hugging Face): LiquidAI/LFM3-Omni-2B
  • Traits: trust_remote_code
  • Definition: gptqmodel/models/definitions/lfm3_omni.py
  • Summary: LFM3-Omni multimodal: LFM3 text decoder plus audio encoder, depth decoder, vision tower and projectors; only the text decoder's linear projections are quantized.
class LFM3OmniQModel(LFM3QModel):

LFM3QModel

  • model_type key(s): lfm3
  • Creator: Liquid AI
  • Base weights (Hugging Face): LiquidAI/LFM3-1.7B
  • Traits: trust_remote_code
  • Definition: gptqmodel/models/definitions/lfm3.py
  • Summary: LFM3 tri-hybrid decoder alternating depthwise short-conv, full-attention and sliding-attention layers.
class LFM3QModel(BaseQModel):

LFM3VLQModel

class LFM3VLQModel(LFM3QModel):

LLaDA2MoeQModel

class LLaDA2MoeQModel(BaseQModel):

Llama4QModel

class Llama4QModel(BaseQModel):

Llama4TextQModel

class Llama4TextQModel(Llama4QModel):

LlamaQModel

  • model_type key(s): llama, smollm3, cohere, cohere2, mistral, yi, stablelm_epoch, stablelm, longllama, gemma, granite, olmo2, seed_oss
  • Creator: Meta
  • Base weights (Hugging Face): meta-llama/Meta-Llama-3-8B
  • Definition: gptqmodel/models/definitions/llama.py
  • Summary: Llama-family dense decoders; also covers Llama-layout derivatives (Llama 2/3.x, many community models).
class LlamaQModel(BaseQModel):

LlavaQModel

class LlavaQModel(BaseQModel):
    """Hugging Face Transformers LLaVA (e.g. LLaVA-1.5) model definition."""

LlavaQwen2QModel

class LlavaQwen2QModel(BaseQModel):

LocateAnythingQModel

  • model_type key(s): locateanything
  • Creator: LocateAnything
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Traits: trust_remote_code
  • Definition: gptqmodel/models/definitions/locateanything.py
  • Summary: LocateAnything open-vocabulary localization VLM; no public HF base-weights checkpoint verified.
class LocateAnythingQModel(BaseQModel):
    """Quantize LocateAnything's Qwen decoder while keeping its vision path dense."""

LongCatFlashQModel

class LongCatFlashQModel(BaseQModel):

MageVLQModel

  • model_type key(s): mage_vl
  • Creator: Mage
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Traits: trust_remote_code
  • Definition: gptqmodel/models/definitions/mage_vl.py
  • Summary: Quantize Mage-VL's Qwen3 decoder while keeping the custom vision tower dense.
class MageVLQModel(BaseQwen3VLGPTQ):
    """Quantize Mage-VL's Qwen3 decoder while keeping the custom vision tower dense."""

MimoQModel

class MimoQModel(LlamaQModel):

MimoV2QModel

class MimoV2QModel(BaseQModel):

MiniCpm3QModel

class MiniCpm3QModel(BaseQModel):

MiniCPMGPTQ

class MiniCPMGPTQ(BaseQModel):

MiniCPMOQModel

class MiniCPMOQModel(BaseQModel):

MiniCPMV4_6QModel

class MiniCPMV4_6QModel(BaseQModel):

MiniCPMVQModel

class MiniCPMVQModel(BaseQModel):

MiniMaxM2GPTQ

class MiniMaxM2GPTQ(BaseQModel):
    """GPTQ config for MiniMax-M2 (HF: MiniMaxM2*), featuring:       - Per-layer RMSNorm before/after attention       - Standard attention with q/k normalization parameters       - Sparse MoE block (MiniMaxM2SparseMoeBlock) with up to 256 experts"""

MiniMaxM3VLGPTQ

class MiniMaxM3VLGPTQ(BaseQModel):

Ministral3GPTQ

class Ministral3GPTQ(BaseQModel):

Mistral3GPTQ

class Mistral3GPTQ(BaseQModel):

MixtralQModel

class MixtralQModel(BaseQModel):

MLlamaQModel

class MLlamaQModel(BaseQModel):

MLlamaTextQModel

class MLlamaTextQModel(MLlamaQModel):

MobileLLMQModel

class MobileLLMQModel(LlamaQModel):

MossQModel

class MossQModel(BaseQModel):

MptQModel

class MptQModel(BaseQModel):

MuseGlimmerQModel

class MuseGlimmerQModel(BaseQModel):

NanbeigeQModel

class NanbeigeQModel(LlamaQModel):
    """Nanbeige's looped decoder reuses a Llama-shaped set of physical layers."""

NemotronHPuzzleQModel

  • model_type key(s): nemotron_h_puzzle
  • Creator: NVIDIA
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Traits: dynamic experts (n_routed_experts)
  • Definition: gptqmodel/models/definitions/nemotron_h_puzzle.py
  • Summary: NVIDIA Nemotron-Labs-3 Puzzle hybrid MoE; partner/internal checkpoint, no public HF base weights located.
class NemotronHPuzzleQModel(NemotronHQModel):
    """GPTQ definition for NVIDIA Nemotron-Labs-3 Puzzle hybrid MoE models."""

NemotronHQModel

class NemotronHQModel(BaseQModel):

NemotronLabsDiffusionQModel

class NemotronLabsDiffusionQModel(BaseQModel):

NemotronOmniQModel

class NemotronOmniQModel(NemotronHQModel):
    """GPTQ wrapper for Nemotron Omni models that embed a Nemotron-H language backbone."""

Olmo3QModel

class Olmo3QModel(BaseQModel):

OlmoeQModel

class OlmoeQModel(BaseQModel):
    """Quantization definition for the Transformers OLMoE decoder."""

OpenVLAQModel

class OpenVLAQModel(BaseQModel):
    """Quantization definition for OpenVLA checkpoints such as `openvla/openvla-7b`."""

OptQModel

class OptQModel(BaseQModel):

OuroQModel

class OuroQModel(BaseQModel):
    """GPTQ definition for ByteDance's looped Ouro decoder."""

Ovis2_5QModel

class Ovis2_5QModel(BaseQModel):

Ovis2_6_MoeQModel

class Ovis2_6_MoeQModel(Ovis2_5QModel):

Ovis2_6_NextQModel

class Ovis2_6_NextQModel(Ovis2_6_MoeQModel):

Ovis2QModel

class Ovis2QModel(BaseQModel):

OvisQModel

class OvisQModel(BaseQModel):

PanguAlphaQModel

class PanguAlphaQModel(BaseQModel):

Phi3QModel

class Phi3QModel(BaseQModel):

Phi4MMGPTQ

class Phi4MMGPTQ(BaseQModel):

PhiMoEGPTQForCausalLM

class PhiMoEGPTQForCausalLM(BaseQModel):

PhiQModel

class PhiQModel(BaseQModel):

Qwen2_5_OmniGPTQ

class Qwen2_5_OmniGPTQ(BaseQwen2_5_OmniGPTQ):

Qwen2_5_VLQModel

class Qwen2_5_VLQModel(BaseQwen2VLGPTQ):

Qwen2AudioGPTQ

class Qwen2AudioGPTQ(BaseQModel):

Qwen2MoeQModel

class Qwen2MoeQModel(BaseQModel):
    """Qwen2 MoE definition aligned to the upstream HF forward execution order."""

Qwen2QModel

class Qwen2QModel(LlamaQModel):

Qwen2VLQModel

class Qwen2VLQModel(BaseQwen2VLGPTQ):

Qwen3_5_MoeQModel

class Qwen3_5_MoeQModel(BaseQwen3VLGPTQ):

Qwen3_5_MoeTextQModel

class Qwen3_5_MoeTextQModel(Qwen3MoeQModel):
    """Text-only Qwen 3.5/3.6 MoE shells use the causal LM loader and keep routed     experts under `model.layers`."""

Qwen3_5QModel

class Qwen3_5QModel(LlamaQModel):
    """Qwen3_5 inherits the Llama-style layout but inserts Q/K RMS norm layers     ahead of the attention projections. We mark those helper modules as     non-quantized so the layer walker captures the complete structure."""

Qwen3_5TextQModel

class Qwen3_5TextQModel(Qwen3QModel):
    """Text-only Qwen 3.5 shells use the standard causal LM loader and keep their     decoder stack directly under `model.layers`."""

Qwen3_VL_MoeQModel

class Qwen3_VL_MoeQModel(BaseQwen3VLGPTQ):
    """Qwen3-VL MoE decoder with defused per-expert linear projections."""

Qwen3_VLQModel

class Qwen3_VLQModel(BaseQwen3VLGPTQ):

Qwen3MoeQModel

class Qwen3MoeQModel(BaseQModel):

Qwen3NextGPTQ

class Qwen3NextGPTQ(BaseQModel):
    """GPTQ config for Qwen3-Next (HF: Qwen3Next*), supporting:       - Mixed token mixers per layer: 'full_attention' (self_attn.*) and 'linear_attention' (linear_attn.*)       - Dense MLP (Qwen3NextMLP) and Sparse MoE (Qwen3NextSparseMoeBlock)       - Dynamic expert indexing via config.num_experts"""

Qwen3OmniMoeGPTQ

class Qwen3OmniMoeGPTQ(BaseQModel):

Qwen3QModel

class Qwen3QModel(LlamaQModel):
    """Qwen3 inherits the Llama-style layout but inserts Q/K RMS norm layers     ahead of the attention projections. We mark those helper modules as     non-quantized so the layer walker captures the complete structure."""

Qwen4ExpQModel

class Qwen4ExpQModel(BaseQModel):
    """Qwen3.8-Flash-Next / Qwen4 experimental multimodal MoE."""

Qwen4ExpTextQModel

class Qwen4ExpTextQModel(Qwen4ExpQModel):
    """Qwen4-Exp text-only checkpoints with the standalone decoder layout."""

QwenDriveQModel

  • model_type key(s): qwen_drive
  • Creator: Alibaba Qwen
  • Base weights (Hugging Face): no public checkpoint located — partner/internal checkpoints only
  • Definition: gptqmodel/models/definitions/qwen_drive.py
  • Summary: Quantization definition for the vlm.* part of Qwen-Drive.
class QwenDriveQModel(Qwen3_5QModel):
    """Quantization definition for the ``vlm.*`` part of Qwen-Drive."""

QwenQModel

class QwenQModel(BaseQModel):

Raon1MiniQModel

class Raon1MiniQModel(BaseQModel):
    """KRAFTON Raon-1 mini hybrid RDA/MLA model with routed MoE layers."""

RwgQModel

class RwgQModel(BaseQModel):

SmolVLMQModel

class SmolVLMQModel(BaseQModel):

SolarOpen2QModel

class SolarOpen2QModel(BaseQModel):
    """GPTQ definition for Solar Open 2's hybrid-attention MoE decoder."""

SolarOpenQModel

class SolarOpenQModel(BaseQModel):
    """GPTQ definition for Solar Open's GQA MoE decoder."""

Spark2_5QModel

class Spark2_5QModel(BaseQModel):
    """GPTQ definition for Spark-X2.5's hybrid-attention decoder."""

Starcoder2QModel

class Starcoder2QModel(BaseQModel):

TeleChat2QModel

class TeleChat2QModel(BaseQModel):

UnlimitedOCRQModel

class UnlimitedOCRQModel(BaseQModel):
    """Quantization definition for Baidu Unlimited-OCR checkpoints."""

VoxtralGPTQ

class VoxtralGPTQ(BaseQModel):

Xing4_0QModel

class Xing4_0QModel(BaseQModel):
    """Quantization layout for XingChen-AGI's Xing4.0 MLA/HC MoE model."""

XverseQModel

class XverseQModel(LlamaQModel):

Zamba2QModel

class Zamba2QModel(BaseQModel):

ZambaQModel

class ZambaQModel(BaseQModel):

ZDTaichu5QModel

class ZDTaichu5QModel(Qwen3_5TextQModel):
    """Quantization definition for ``TaichuAI/ZDTaichu5.0-9B``."""

Documentation home · Quickstart · Troubleshooting · Repository · PyPI

Source baseline: GPT-QModel 7.8.0 / ca52558. Maintained by ModelCloud.ai.

Home

Getting started

Quantization methods

Complementary configs

Using a model

Platforms

Internals

Clone this wiki locally