Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: 'shared-multilingual-translation' #7

Open
AmanSinghal927 opened this issue Sep 16, 2020 · 1 comment
Open

KeyError: 'shared-multilingual-translation' #7

AmanSinghal927 opened this issue Sep 16, 2020 · 1 comment

Comments

@AmanSinghal927
Copy link

translator = from_pretrained(tag='mm-all-iter1')

| [src] dictionary: 40897 types
| [tgt] dictionary: 40897 types
/content/ilmulti/ilmulti/translator/translator.py:37: UserWarning: utils.load_ensemble_for_inference is deprecated. Please use checkpoint_utils.load_model_ensemble instead.
  self.models, model_args = fairseq.utils.load_ensemble_for_inference(model_paths, self.task, model_arg_overrides=eval(args.model_overrides))
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-22-f8e48d2129df> in <module>()
----> 1 translator = from_pretrained(tag='mm-to-en-iter1')

7 frames
/content/ilmulti/ilmulti/translator/pretrained.py in from_pretrained(tag, use_cuda)
     60     from .mt_engine import MTEngine
     61 
---> 62     translator = build_translator(config['model'], use_cuda=use_cuda)
     63     segmenter = build_segmenter(config['segmenter'])
     64     tokenizer = build_tokenizer(config['tokenizer'])

/content/ilmulti/ilmulti/translator/translator.py in build_translator(model, use_cuda)
    169     args.enhance(**keyword_arguments)
    170 
--> 171     fseq_translator = FairseqTranslator(args, use_cuda=use_cuda)
    172     return fseq_translator
    173 

/content/ilmulti/ilmulti/translator/translator.py in __init__(self, args, use_cuda)
     35         # print('| loading model(s) from {}'.format(args.path))
     36         model_paths = args.path.split(':')
---> 37         self.models, model_args = fairseq.utils.load_ensemble_for_inference(model_paths, self.task, model_arg_overrides=eval(args.model_overrides))
     38         self.tgt_dict = self.task.target_dictionary
     39 

/usr/local/lib/python3.6/dist-packages/fairseq/utils.py in load_ensemble_for_inference(filenames, task, model_arg_overrides)
     28     )
     29     return checkpoint_utils.load_model_ensemble(
---> 30         filenames, arg_overrides=model_arg_overrides, task=task,
     31     )
     32 

/usr/local/lib/python3.6/dist-packages/fairseq/checkpoint_utils.py in load_model_ensemble(filenames, arg_overrides, task)
    154         task (fairseq.tasks.FairseqTask, optional): task to use for loading
    155     """
--> 156     ensemble, args, _task = _load_model_ensemble(filenames, arg_overrides, task)
    157     return ensemble, args
    158 

/usr/local/lib/python3.6/dist-packages/fairseq/checkpoint_utils.py in _load_model_ensemble(filenames, arg_overrides, task)
    165         if not os.path.exists(filename):
    166             raise IOError('Model file not found: {}'.format(filename))
--> 167         state = load_checkpoint_to_cpu(filename, arg_overrides)
    168 
    169         args = state['args']

/usr/local/lib/python3.6/dist-packages/fairseq/checkpoint_utils.py in load_checkpoint_to_cpu(path, arg_overrides)
    141         for arg_name, arg_val in arg_overrides.items():
    142             setattr(args, arg_name, arg_val)
--> 143     state = _upgrade_state_dict(state)
    144     return state
    145 

/usr/local/lib/python3.6/dist-packages/fairseq/checkpoint_utils.py in _upgrade_state_dict(state)
    319 
    320     # set any missing default values in the task, model or other registries
--> 321     set_defaults(tasks.TASK_REGISTRY[state['args'].task])
    322     set_defaults(models.ARCH_MODEL_REGISTRY[state['args'].arch])
    323     for registry_name, REGISTRY in registry.REGISTRIES.items():

KeyError: 'shared-multilingual-translation'

@jerinphilip
Copy link
Owner

Okay, do a bunch of package changes (fairseq==0.8.0 to a fork). I have managed a final self-contained colab-notebook implementation.
https://colab.research.google.com/drive/1KOvjawhzPXOQ6RLlFBFeInkuuR0QAWTK?usp=sharing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants