Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: copying a param with shape compatibility #3419

Closed
stefan-it opened this issue Mar 11, 2024 · 2 comments 路 Fixed by #3451
Closed

[Bug]: copying a param with shape compatibility #3419

stefan-it opened this issue Mar 11, 2024 · 2 comments 路 Fixed by #3451
Assignees
Labels
bug Something isn't working

Comments

@stefan-it
Copy link
Member

stefan-it commented Mar 11, 2024

Describe the bug

Hi everyone,

I've trained a lot of Flair models in the past for my hmBench project.

Unfortunately, with the latest Flair master version, at least one model is no longer working 馃槦

from flair.models import SequenceTagger

tagger = SequenceTagger.load("stefan-it/hmbench-ajmc-en-hmbert-bs8-wsFalse-e10-lr5e-05-poolingfirst-layers-1-crfFalse-2")

The following error is thrown:

File ~/Repositories/flair/flair/nn/model.py:105, in Model._init_model_with_state_dict(cls, state, **kwargs)
    101     kwargs["embeddings"] = embeddings
    103 model = cls(**kwargs)
--> 105 model.load_state_dict(state["state_dict"])
    107 return model

File ~/.venvs/dev/lib/python3.11/site-packages/torch/nn/modules/module.py:1671, in Module.load_state_dict(self, state_dict, strict)
   1666         error_msgs.insert(
   1667             0, 'Missing key(s) in state_dict: {}. '.format(
   1668                 ', '.join('"{}"'.format(k) for k in missing_keys)))
   1670 if len(error_msgs) > 0:
-> 1671     raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
   1672                        self.__class__.__name__, "\n\t".join(error_msgs)))
   1673 return _IncompatibleKeys(missing_keys, unexpected_keys)

RuntimeError: Error(s) in loading state_dict for SequenceTagger:
	size mismatch for embeddings.model.embeddings.word_embeddings.weight: copying a param with shape torch.Size([32001, 768]) from checkpoint, the shape in current model is torch.Size([32002, 768]).

I have found a working commit, where the model can be loaded without any error: 42ea3f6 - in combination with transformer-smaller-training-vocab==0.2.3.

Now my question is, if there's any chance to "repair" these models.

@helpmefindaname
Copy link
Collaborator

Hi @stefan-it
I am sorry that it took me so long to find time for this, the fix is actually very simple, can you take a look at: #3451 ?

@stefan-it
Copy link
Member Author

Many thanks @helpmefindaname ! I can confirm that the fix is working :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants