RuntimeError when loading InternVL3-14B model: Embedding size mismatch

[wkzcml-1]

Problem Description

When trying to load the InternVL3-14B model using the transformers library, I encountered the following error:

RuntimeError: Error(s) in loading state_dict for Embedding:
	size mismatch for weight: copying a param with shape torch.Size([151674, 5120]) from checkpoint, the shape in current model is torch.Size([151936, 4096]).

Additional Environment Information

Transformers Package Details:​

Name: transformers
Version: 4.52.0.dev0
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /home/tiger/.local/lib/python3.11/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: peft

Python version

Python 3.11.2

Code Snippet Used

model = InternVLForConditionalGeneration.from_pretrained(
    internvl3_14B_dir, 
    trust_remote_code=True
)

[Rocketknight1]

Hi @wkzcml-1, your code snippet refers to a local directory that we don't have access to. Can you give us a complete snippet that we can copy and paste to see the error, preferably using a model on the Hub instead?

[yonigozlan]

Hi @wkzcml-1 ! The link you provided is not the official transformers implementation, that would be this one, hope it solves your issue!

[wkzcml-1]

Hi @wkzcml-1 ! The link you provided is not the official transformers implementation, that would be this one, hope it solves your issue!

Thank you. By the way, I couldn't find the OpenGVLab/InternVL3-14B-hf link in OpenGVLab's Collections, which might cause inconvenience for users.

[wkzcml-1]

Hi @wkzcml-1, your code snippet refers to a local directory that we don't have access to. Can you give us a complete snippet that we can copy and paste to see the error, preferably using a model on the Hub instead?

Thank you. The reason is that the model I used is not the official Transformers implementation.