Skip to content

Conversation

ftgreat
Copy link
Contributor

@ftgreat ftgreat commented Oct 12, 2023

Recently we released Aquila2 models. Models are opensourced in HuggingFace, like AquilaChat2-34B

Could we update README.md with Aquila2,thanks.

Signed-off-by: ldwang <ftgreat@gmail.com>
@ftgreat ftgreat marked this pull request as draft October 12, 2023 12:44
@ftgreat ftgreat marked this pull request as ready for review October 12, 2023 12:44
Copy link
Collaborator

@WoosukKwon WoosukKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the updates!

@WoosukKwon WoosukKwon merged commit 6368e77 into vllm-project:main Oct 12, 2023
@huyz-git
Copy link

I failed to run Aquila2 with vllm:

ValueError: Model architectures ['AquilaForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'FalconForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MPTForCausalLM', 'OPTForCausalLM', 'QWenLMHeadModel', 'RWForCausalLM']

If I modify vllm/model_executor/model_loader.py file and map this model architechture into AquilaForCausalLM, then I got:

File "/home/xxx/vllm/vllm/model_executor/models/aquila.py", line 370, in load_weights
    assert param_slice.shape == loaded_weight.shape
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

@lu-wang-dl
Copy link
Contributor

Make a PR to fix the issue: #1339

@lonngxiang
Copy link

I failed to run Aquila2 with vllm:

ValueError: Model architectures ['AquilaForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'FalconForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MPTForCausalLM', 'OPTForCausalLM', 'QWenLMHeadModel', 'RWForCausalLM']

If I modify vllm/model_executor/model_loader.py file and map this model architechture into AquilaForCausalLM, then I got:

File "/home/xxx/vllm/vllm/model_executor/models/aquila.py", line 370, in load_weights
    assert param_slice.shape == loaded_weight.shape
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

same error;version 0.2.0

ValueError: Model architectures ['AquilaForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'FalconForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MPTForCausalLM', 'OPTForCausalLM', 'QWenLMHeadModel', 'RWForCausalLM']

hongxiayang pushed a commit to hongxiayang/vllm that referenced this pull request Feb 13, 2024
Signed-off-by: ldwang <ftgreat@gmail.com>
Co-authored-by: ldwang <ftgreat@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants