Missing added_tokens.json

[videah]

I am getting a vocab size mismatch between the model (32001) and tokenizer.model (32000) as there is no added_tokens.json included with the delta weight release

[Michaelvll]

Could you try installing the transformers with the same commit hash mentioned in the instruction? The transformers recently changed their tokenizer for the LLaMA model, causing that issue.
pip3 install git+https://github.com/huggingface/transformers@c612628045822f909020f7eb6784c79700813eda

[J-Griffin]

I get same thing (32001 vocab mismatch) when following Method 1 OR Method 2 install steps.

[merrymercy]

We did some updates. Could you try the latest huggingface/transformers and our latest checkpoints?https://huggingface.co/lmsys/vicuna-13b-delta-v0

[avijit-thawani]

I'm getting the same error even after a hard refetch from vicuna. Is there an easier off-by-one fix for the input and output embedding matrix, given that vicuna is 32001 and llama is 32000?
Update: tried both param.data += delta.state_dict()[name][1:] and param.data += delta.state_dict()[name][:-1] but I get the following error after submitting "Human: Hi" in the cli mode:

probability tensor contains either `inf`, `nan` or element < 0

[cbrendanprice]

as a temporary workaround, i manually created the added_tokens.json file in the model directory with the following content:

{
  "[PAD]": 32000
}

and it worked as expected. hope this helps.

[djaffer]

facing same issue

[merrymercy]

We released a new training script and a new version of weights v1.1 (https://github.com/lm-sys/FastChat-dev/blob/tokenize-conv-sep/docs/weights_version.md) which can solve a lot of these tokenization issues.

Could you try them again with the latest transformer / pytorch?
https://github.com/lm-sys/FastChat#fine-tuning

Close this for now. Feel free to reopen.