Could not import function from cython file named "data_utils_fast.pyx"

[sazzadhrz]

Initializing sentencepiece model for SRC and TGT
Initializing model for translation
2023-08-29 16:56:08 | INFO | fairseq.tasks.text_to_speech | Please install tensorboardX: pip install tensorboardX
2023-08-29 16:56:25 | INFO | fairseq.tasks.translation | [SRC] dictionary: 122706 types
2023-08-29 16:56:25 | INFO | fairseq.tasks.translation | [TGT] dictionary: 32296 types
2023-08-29 16:58:12 | INFO | fairseq.tasks.fairseq_task | can_reuse_epoch_itr = True
2023-08-29 16:58:12 | INFO | fairseq.tasks.fairseq_task | reuse_dataloader = True
2023-08-29 16:58:12 | INFO | fairseq.tasks.fairseq_task | rebuild_batches = False
2023-08-29 16:58:12 | INFO | fairseq.tasks.fairseq_task | creating new batches for epoch 1
Traceback (most recent call last):
  File "/mnt/d/NMT/IndicTrans2/fairseq/data/data_utils.py", line 313, in batch_by_size
    from fairseq.data.data_utils_fast import (
ModuleNotFoundError: No module named 'fairseq.data.data_utils_fast'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/d/NMT/IndicTrans2/go.py", line 19, in <module>
    model.translate_paragraph("এটি মূলত তাদের জন্য যারা উপস্থিত থাকতে পারেন নি বা উপস্থিত ছিলেন না।", "ben_Beng", "eng_Latn")
  File "/mnt/d/NMT/IndicTrans2/inference/engine.py", line 278, in translate_paragraph
    postprocessed_sents = self.batch_translate(sents, src_lang, tgt_lang)
  File "/mnt/d/NMT/IndicTrans2/inference/engine.py", line 252, in batch_translate
    translations = self.translate_lines(preprocessed_sents)
  File "/mnt/d/NMT/IndicTrans2/inference/engine.py", line 180, in fairseq_translate_lines
    return self.translator.translate(lines)
  File "/mnt/d/NMT/IndicTrans2/inference/custom_interactive.py", line 236, in translate
    for batch in make_batches(
  File "/mnt/d/NMT/IndicTrans2/inference/custom_interactive.py", line 61, in make_batches
    itr = task.get_batch_iterator(
  File "/mnt/d/NMT/IndicTrans2/fairseq/tasks/fairseq_task.py", line 318, in get_batch_iterator
    batch_sampler = make_batches(dataset, epoch)
  File "/mnt/d/NMT/IndicTrans2/fairseq/tasks/fairseq_task.py", line 300, in make_batches
    batches = dataset.batch_by_size(
  File "/mnt/d/NMT/IndicTrans2/fairseq/data/fairseq_dataset.py", line 145, in batch_by_size
    return data_utils.batch_by_size(
  File "/mnt/d/NMT/IndicTrans2/fairseq/data/data_utils.py", line 319, in batch_by_size
    raise ImportError(
ImportError: Please build Cython components with: `python setup.py build_ext --inplace`

To Reproduce please run the following code from the root path

from inference.engine import Model

ckpt_dir = "/mnt/d/Datasets/NMT/ai4bharat/indic-en-preprint/fairseq_model/"

model = Model(ckpt_dir, model_type="fairseq")

model.translate_paragraph("এটি মূলত তাদের জন্য যারা উপস্থিত থাকতে পারেন নি বা উপস্থিত ছিলেন না।", "ben_Beng", "eng_Latn")

Note: I have made the model run in CPU by replacing the "model.cuda()" with "model.cpu()"

[PranjalChitale]

facebookresearch/fairseq#2133