[t2t_vocabulary.py:32] Failed to load tensor2tensor

[Zminghua]

In mesh_tensorflow/transformer/t2t_vocabulary.py, it need to import subword_text_encoder_ops from tensor2tensor.data_generators.ops. However, there is no subword_text_encoder_ops.py in tensor2tensor repository. Tensor2tensor only contains subword_text_encoder_ops.cc file.

[anatoly-khomenko]

Hello,

Having the same issue when running Text-to-text Transfer Transformer (T5) model.
I have found also that in my virtual environment folder ops does not exist here: venv/lib/python3.6/site-packages/tensor2tensor/data_generators

tensor2tensor==1.15.2 installed as a dependency, as well as mesh-tensorflow==0.1.5

Is there any issue if tensor2tensor.data_generators.ops module can not be loaded?

Thank you!

[adarob]

You can safely ignore this warning if you'd not using one of these generators (which T5 does not). I will update the warning to be something less scary when I get a chance.

…

On Thu, Nov 28, 2019 at 8:35 AM Anatoly Khomenko ***@***.***> wrote: Hello, Having the same issue when running Text-to-text Transfer Transformer (T5) model. I have found also that in my virtual environment folder *ops* does not exist here: venv/lib/python3.6/site-packages/tensor2tensor/data_generators tensor2tensor==1.15.2 installed as a dependency, as well as mesh-tensorflow==0.1.5 Is there any issue if tensor2tensor.data_generators.ops module can not be loaded? Thank you! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#33?email_source=notifications&email_token=AAIJV2AYQ5NO6K7TBJSA4UTQV7XO5A5CNFSM4JI55XM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFNCEYI#issuecomment-559555169>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIJV2HUUOOUPPUCVCX7T6LQV7XO5ANCNFSM4JI55XMQ> .

[wintersurvival]

You can safely ignore this warning if you'd not using one of these generators (which T5 does not). I will update the warning to be something less scary when I get a chance.
…
On Thu, Nov 28, 2019 at 8:35 AM Anatoly Khomenko @.***> wrote: Hello, Having the same issue when running Text-to-text Transfer Transformer (T5) model. I have found also that in my virtual environment folder ops does not exist here: venv/lib/python3.6/site-packages/tensor2tensor/data_generators tensor2tensor==1.15.2 installed as a dependency, as well as mesh-tensorflow==0.1.5 Is there any issue if tensor2tensor.data_generators.ops module can not be loaded? Thank you! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#33?email_source=notifications&email_token=AAIJV2AYQ5NO6K7TBJSA4UTQV7XO5A5CNFSM4JI55XM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFNCEYI#issuecomment-559555169>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIJV2HUUOOUPPUCVCX7T6LQV7XO5ANCNFSM4JI55XMQ .

when running transformer code with "infer" mode, it would call subword_text_encoder_ops. so we cannot get bleu score if subword_text_encoder_ops cannot be found.

[wintersurvival]

Does subword_text_encoder_ops laid in .so file? it cannot import.