Missing requirements? Fairseq/omegaconf not optional; failure to load transformers version on colab

[daltonj]

Hi,

I am trying to follow the example to use GENRE with transformers.
https://github.com/facebookresearch/GENRE/blob/main/examples/transformers.ipynb

I put this into google colab for testing.
https://colab.research.google.com/drive/1hG4yRbrIe2XOZN1F_IZv7Xm0qIac40dU?usp=sharing

The example did not include installing GENRE, which i presume is a prerequisite for running things.

The error I get is:
ModuleNotFoundError Traceback (most recent call last)
in ()
4
5 with open("/content/kilt_titles_trie.pkl", "rb") as f:
----> 6 trie = pickle.load(f)
7
8 def prefix_allowed_tokens_fn(batch_id, sent):

1 frames
/content/GENRE/genre/base_model.py in ()
10
11 import torch
---> 12 from fairseq import search, utils
13 from fairseq.models.bart import BARTHubInterface, BARTModel
14 from omegaconf import open_dict

ModuleNotFoundError: No module named 'fairseq'

So, I installed fairseq, but there appears to be a version / dependency with fairseq on omegaconf?

ModuleNotFoundError Traceback (most recent call last)
in ()
4
5 with open("/content/kilt_titles_trie.pkl", "rb") as f:
----> 6 trie = pickle.load(f)
7
8 def prefix_allowed_tokens_fn(batch_id, sent):

4 frames
/content/fairseq/fairseq/dataclass/configs.py in ()
22 )
23
---> 24 from omegaconf import II, MISSING
25
26

ModuleNotFoundError: No module named 'omegaconf'

I don't think it's intended that the transformers version should require this. Also, what's the issue with omegaconf and fairseq, which presumably be a problem using it from that side?

[nicola-decao]

Are you getting the error when loading the pickle file? Am I understanding correctly?

[daltonj]

Correct.

I tracked it down a bit deeper. It's a python version issue. The default in colab is python 3.6.9, which causes the problem.

[nicola-decao]

@daltonj GREAT! 💪

[daltonj]

There's still some python version issues. By default you get errors that KILT is not available. It has to be downloaded and installed as a separate step. Is it needed for inference?

I also seem to have some issues using this with the latest python 3.8; so I assume only 3.7 is supported?