ImportError: cannot import name 'metrics' from 'fairseq' (unknown location)

[PussyCat0700]

Hi, I am quite new to fairseq and when I was trying to pretrain a model from scratch using fairseq-hydra-train as instructed in README.md:
$ cd avhubert $ fairseq-hydra-train --config-dir /path/to/conf/ --config-name conf-name \ task.data=/path/to/data task.label_dir=/path/to/label \ model.label_rate=100 hydra.run.dir=/path/to/experiment/pretrain/ \ common.user_dir=pwd``
I got an error like this:
Traceback (most recent call last):
File "", line 1, in
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/multiprocessing/spawn.py", line 125, in _main
prepare(preparation_data)
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/multiprocessing/spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/runpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/xxx/anaconda3/envs/avhubert/bin/fairseq-hydra-train", line 5, in
from fairseq_cli.hydra_train import cli_main
File "/home/xxx/av_hubert/fairseq/fairseq_cli/hydra_train.py", line 11, in
from fairseq_cli.train import main as pre_main
File "/home/xxx/av_hubert/fairseq/fairseq_cli/train.py", line 28, in
from fairseq import (
File "/home/xxx/av_hubert/fairseq/fairseq/checkpoint_utils.py", line 26, in
from fairseq.models import FairseqDecoder, FairseqEncoder
File "/home/xxx/av_hubert/fairseq/fairseq/models/init.py", line 225, in
import_models(models_dir, "fairseq.models")
File "/home/xxx/av_hubert/fairseq/fairseq/models/init.py", line 207, in import_models
importlib.import_module(namespace + "." + model_name)
File "/home/xxx/anaconda3/envs/avhubert/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/xxx/av_hubert/fairseq/fairseq/models/wav2vec/init.py", line 6, in
from .wav2vec import * # noqa File "/home/xxx/av_hubert/fairseq/fairseq/models/wav2vec/wav2vec.py", line 25, in from fairseq.tasks import FairseqTask File "/home/xxx/av_hubert/fairseq/fairseq/tasks/init.py", line 15, in from .fairseq_task import FairseqTask, LegacyFairseqTask # noqa File "/home/xxx/av_hubert/fairseq/fairseq/tasks/fairseq_task.py", line 13, in from fairseq import metrics, search, tokenizer, utilsImportError: cannot import name 'metrics' from 'fairseq' (unknown location)

I thought the problem was with fairseq first, but as I tested the following command in avhubert conda enviornment:
python -c 'from fairseq import metrics'
No exception was thrown and it didn't seem to be the problem.

I have run git submodule init and git submodule update as instructed in README.md, and pip install --editable ./ has been run under fairseq directory with no problem. I have generated mfcc features from audio inputs and have done clustering. What can I do to solve the problem?

BTW: The configuration file I am using is base_lrs3_iter1.yaml

[PussyCat0700]

Update: Though I have tried to delete the whole project and clone from origin again, the problem is still there

[chevalierNoir]

Hi,

In your command, common.user_dir should be specified as common.user_dir=`pwd` . Other than that, when you clone the project again, could you try starting a new conda environment and follow the exact instructions as is in the readme?

[PussyCat0700]

Hi,

Surprisingly, I refactored imports in fairseq myself and it worked. So I guess it was just the problem with the imports of fairseq I was using, exactly as the error log said. I am not really sure why I didn't get the right version of fairseq though I have tried to clone the project and install submodules as in README.md several times without touching anything in fairseq submodule. Anyway, after adjusting imports in fairseq submodule manaully, it seems that the problem is solved now.

Thanks for your reply. Given that the problem is solved, I'll just mark the issue as closed now.

[secnavo]

Hi,

Surprisingly, I refactored imports in fairseq myself and it worked. So I guess it was just the problem with the imports of fairseq I was using, exactly as the error log said. I am not really sure why I didn't get the right version of fairseq though I have tried to clone the project and install submodules as in README.md several times without touching anything in fairseq submodule. Anyway, after adjusting imports in fairseq submodule manaully, it seems that the problem is solved now.

Thanks for your reply. Given that the problem is solved, I'll just mark the issue as closed now.

Hello, I have the same problem, how can I solve it by adjusting imports in fairseq submodule manaully? @PussyCat0700

[PussyCat0700]

Hi,
Surprisingly, I refactored imports in fairseq myself and it worked. So I guess it was just the problem with the imports of fairseq I was using, exactly as the error log said. I am not really sure why I didn't get the right version of fairseq though I have tried to clone the project and install submodules as in README.md several times without touching anything in fairseq submodule. Anyway, after adjusting imports in fairseq submodule manaully, it seems that the problem is solved now.
Thanks for your reply. Given that the problem is solved, I'll just mark the issue as closed now.

Hello, I have the same problem, how can I solve it by adjusting imports in fairseq submodule manaully? @PussyCat0700

Hello Secnavo:

I later uninstalled the submodule totally and simply tried out pip install fairseq==0.12.2 and it worked fine for me.
However, one problem you may find is that fairseq of this version cannot load dictionary properly. In that case you can make dictionary callable in their source code. Good luck!

[PussyCat0700]

Update: Found another solution that also works fine for me on another project requiring fairseq as a submodule. This solution is much easier and doesn't require any refactoring.
In short, just run this in terminal before executing fairseq-hydra-train and things will work out:
export PYTHONPATH=$PYTHONPATH:<your absolute path to fairseq submodule>