AssertionError when load WizardLM_WizardCoder-15B-V1.0

[ishotoli]

I pulled the latest code and ran pip install . -e, and then got the following error when executing python -m vllm.entrypoints.api_server --model /home/foo/workshop/text-generation-webui/models/WizardLM_WizardCoder-15B-V1.0/ command:

INFO 06-26 04:42:39 llm_engine.py:59] Initializing an LLM engine with config: model='/home/foo/workshop/text-generation-webui/models/WizardLM_WizardCoder-15B-V1.0/', dtype=torch.float16, use_dummy_weights=False, download_dir=None, use_np_weights=False, tensor_parallel_size=1, seed=0)
Traceback (most recent call last):
  File "/home/foo/anaconda3/envs/aigc/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/foo/anaconda3/envs/aigc/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/foo/workshop/vllm/vllm/entrypoints/api_server.py", line 82, in <module>
    engine = AsyncLLMEngine.from_engine_args(engine_args)
  File "/home/foo/workshop/vllm/vllm/engine/async_llm_engine.py", line 212, in from_engine_args
    engine = cls(engine_args.worker_use_ray,
  File "/home/foo/workshop/vllm/vllm/engine/async_llm_engine.py", line 49, in __init__
    self.engine = engine_class(*args, **kwargs)
  File "/home/foo/workshop/vllm/vllm/engine/llm_engine.py", line 93, in __init__
    worker = worker_cls(
  File "/home/foo/workshop/vllm/vllm/worker/worker.py", line 45, in __init__
    self.model = get_model(model_config)
  File "/home/foo/workshop/vllm/vllm/model_executor/model_loader.py", line 48, in get_model
    model.load_weights(
  File "/home/foo/workshop/vllm/vllm/model_executor/models/gpt_bigcode.py", line 288, in load_weights
    load_tensor_parallel_weights(param, loaded_weight, name,
  File "/home/foo/workshop/vllm/vllm/model_executor/weight_utils.py", line 101, in load_tensor_parallel_weights
    assert param.shape == loaded_weight.shape
AssertionError

Am I missing something?

[Xingxiangrui]

thank you. I met the same Error when I use LLM.generate and load the WizardCoder-15B from local.

After changing the two new files in #313, the bug have been fixed.