Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when preparing Vicuna v0 weights? Can I use Vicuna v1.1? #25

Closed
JosephPai opened this issue Apr 18, 2023 · 6 comments
Closed

Error when preparing Vicuna v0 weights? Can I use Vicuna v1.1? #25

JosephPai opened this issue Apr 18, 2023 · 6 comments

Comments

@JosephPai
Copy link

Hi,

Thanks for releasing the interesting work.
I'm trying to deploy it on my server.
However, I encountered some difficulties when preparing Vicuna weights.

When apply the delta weights of Vicuna to the original LLaMa weights, I always got vocab mismatch error like this:
RuntimeError: The size of tensor a (32000) must match the size of tensor b (32001) at non-singleton dimension 0
I've searched issues in the FastChat repo but didn't find effective solution.
The author of FastChat suggests to directly move to Vicuna v1.1 since they have fixed a lot of issues in the new version.

I'd like to ask

  1. Do you have any experiences/suggestions to solve the issues I encountered?
  2. Do you think it's feasible to directly move to Vicuna v1.1? I noticed that in the new version, they have some changes, like the separator has been changed from ### to </s>. I'm not sure if it is compatible with MiniGPT-4.

Thanks!

@linkct
Copy link

linkct commented Apr 18, 2023

I encountered the same issue. Note that per the Vicuna official repo, Vicuna-v0 is only compatible with FastChat version <= v0.1.10, so pip install fschat==0.1.10 solved it for me.

@MrToy
Copy link

MrToy commented Apr 18, 2023

same issue,looking forward to providing vicuna v1.1

@JosephPai
Copy link
Author

JosephPai commented Apr 18, 2023

I encountered the same issue. Note that per the Vicuna official repo, Vicuna-v0 is only compatible with FastChat version <= v0.1.10, so pip install fschat==0.1.10 solved it for me.

@linkct This solution works for me. Thanks!

@LARRYMIN
Copy link

can change the source code of fastchat.model.apply_delta

if delta_state_dict[name].size(0)==32001:
    state_dict[name] += delta_state_dict[name][:32000, :]
else:
    state_dict[name] += delta_state_dict[name]

@feymanwang
Copy link

can you show the line number to add?

@WynMew
Copy link

WynMew commented Apr 25, 2023

can you show the line number to add?

line 107 I guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants