Error Loading Model RuntimeError: Error(s) in loading state_dict for GPT2LMHeadModel:

[raviolli]

Traceback (most recent call last):
  File "train_summary_loop.py", line 59, in <module>
    summarizer = GeneTransformer(max_output_length=args.max_output_length, device=args.device, tokenizer_type='gpt2', starter_model=summarizer_model_start)
  File "/home/tait-dev-0/summary_loop/summary_loop/model_generator.py", line 30, in __init__
    self.reload(starter_model)
  File "/home/tait-dev-0/summary_loop/summary_loop/model_generator.py", line 39, in reload
    print(self.model.load_state_dict(torch.load(from_file)))
  File "/home/tait-dev-0/anaconda2/envs/summary_loop/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for GPT2LMHeadModel:
	Missing key(s) in state_dict: "transformer.h.0.attn.masked_bias", "transformer.h.1.attn.masked_bias", "transformer.h.2.attn.masked_bias", "transformer.h.3.attn.masked_bias", "transformer.h.4.attn.masked_bias", "transformer.h.5.attn.masked_bias", "transformer.h.6.attn.masked_bias", "transformer.h.7.attn.masked_bias", "transformer.h.8.attn.masked_bias", "transformer.h.9.attn.masked_bias", "transformer.h.10.attn.masked_bias", "transformer.h.11.attn.masked_bias". 

[raviolli]

install transformers = 2.9.0

[jqJordan]

Hey I am having the same problem, but at line 96, where it loads from the fluency_news_bs32.bin. I tried multiple versions of the transformer, 2.9.0, 2.8.0, 2.9.1. It didn't resolve the issue. Any idea why? Thank you.

[ashutosh-dwivedi-e3502]

Hi, I can confirm that the issue persists even with multiple version of transformers.
Can you provide a fix for this?

[s103321048]

I'm also facing the same issue. I've also tried multiple version (from 2.8.0 - 3.2.0) of transformers but it seems like changing version won't help.
Found a relate issue here not so sure what version did the author train on?

[s103321048]

Add strict=False in line 39 of model_generator.py

    def reload(self, from_file):
        print(self.model.load_state_dict(torch.load(from_file),strict=False))

Since there are no such keys in state_dict, just let the model ignore it.

[ahtsham58]

I solved the issue by adding strict=False as a parameter. Thanks man

[theDikshaSaxena]

@s103321048 thanks man, it solved my issue