[Bug]: Can dbrx not use Vllm?

[bank010]

Your current environment

python -m vllm.entrypoints.openai.api_server --model /data1/LLM/model/AI-ModelScope/dbrx-instruct --trust-remote-code

🐛 Describe the bug

ValueError: Model architectures ['DbrxForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'AquilaForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'ChatGLMModel', 'ChatGLMForConditionalGeneration', 'DeciLMForCausalLM', 'DeepseekForCausalLM', 'FalconForCausalLM', 'GemmaForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'InternLM2ForCausalLM', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'QuantMixtralForCausalLM', 'MptForCausalLM', 'MPTForCausalLM', 'OLMoForCausalLM', 'OPTForCausalLM', 'OrionForCausalLM', 'PhiForCausalLM', 'QWenLMHeadModel', 'Qwen2ForCausalLM', 'RWForCausalLM', 'StableLMEpochForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM']

[ywang96]

Which version of vllm are you on? The support of dbrx-instruct is currently only merged to the main branch.

[bank010]

Which version of vllm are you on? The support of dbrx-instruct is currently only merged to the main branch.

Name: vllm
Version: 0.3.3
Summary: A high-throughput and memory-efficient inference and serving engine for LLMs
Home-page: https://github.com/vllm-project/vllm
Author: vLLM Team
Author-email:
License: Apache 2.0

[ywang96]

Version: 0.3.3

You will need to build vllm from source of the main branch in order to use dbrx-instruct.

[robertgshaw2-neuralmagic]

v0.4 should be good to go on this