KeyError: 'generator'

[Akira4ever]

Due to Internet reasons, I downloaded multitask_unity_large.pt and tokenizer.model from https://huggingface.co/facebook/seamless-m4t-large/tree/main by myself. And I modified the pathname in fairseq2/models/utils/model_loader.py and fairseq2/models/nllb/loader.py for the model and tokenizer.

But run with error:

m4t_predict <input_text> t2tt <tgt_lang> --src_lang <src_lang>

Traceback (most recent call last):
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/models/utils/checkpoint_loader.py", line 83, in load_checkpoint
checkpoint = converter(checkpoint)
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/seamless_communication/models/vocoder/loader.py", line 29, in _upgrade_checkpoint
old_state_dict = checkpoint["generator"]
KeyError: 'generator'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/anaconda3/envs/seamless/bin/m4t_predict", line 8, in
sys.exit(main())
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/m4t_scripts/predict/predict.py", line 72, in main
translator = Translator(args.model_name, args.vocoder_name, device, dtype)
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/seamless_communication/models/inference/translator.py", line 79, in init
self.vocoder: Vocoder = self.load_model_for_inference(
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/seamless_communication/models/inference/translator.py", line 90, in load_model_for_inference
model = load_model_fn(model_name_or_card, device=device, dtype=dtype)
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/models/utils/model_loader.py", line 188, in call
checkpoint = load_checkpoint(
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/models/utils/checkpoint_loader.py", line 85, in load_checkpoint
raise_error(ex)
File "/data/anaconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/models/utils/checkpoint_loader.py", line 70, in raise_error
raise RuntimeError(
RuntimeError: The load of the checkpoint of the model 'vocoder_36langs' has failed. See nested exception for details.

[cndn]

Hey @Akira4ever - The code assumes vocoder checkpoint has states stored in 'generator' field. Could you print out checkpoint.keys() in the line throwing error above and see what fields it has in your checkpoint?

Looking at what you've done, one worry I have is you might hardcode file path of the UnitY model in fairseq2/models/utils/model_loader.py but the model loader file is shared with vocoder. If you'd like loading from local path instead of downloading on the fly, you could also update the yaml file with local paths as #50

[Akira4ever]

Thanks! I indeed changed file path of the model and maybe that's the reason. : (

[Akira4ever]

you might hardcode file path of the UnitY model in fairseq2/models/utils/model_loader.py but the model loader file is shared with vocoder.

And it's because I didn't download the vocoder model from huggingface...But #50 also helps me. Thank you!