How can I get the words embeddings?

[stygian2a]

Hello!
Thank you for sharing this code!

Is there an easy way to get the embedding of a particular word?
Those found in table 5. of the paper.
Thank you!

[glample]

Hi!

Yes, I would suggest looking at the code in the notebook:
https://github.com/facebookresearch/XLM/blob/master/generate-embeddings.ipynb

Then doing something like that should work:

word_id = dico.index('cat')
model.embeddings.weight[word_id]

[stygian2a]

Thank you! How can I differentiate words from different languages (ie 'chat' in french means cat)?

[glample]

You can just replace "cat" by "chat" in the code above. There is only one share vocabulary, that contains the words for all languages. The vocabulary doesn't keep track of which word is used in which language.

[stygian2a]

Got it, thx for everything!

[vvssttkk]

i want get models for russian language, the mlm_xnli15_1024.pth will do?

[glample]

Yes, it contains Russian. But these two models will give you a better performance, and also support Russian:

https://dl.fbaipublicfiles.com/XLM/mlm_17_1280.pth
https://dl.fbaipublicfiles.com/XLM/mlm_100_1280.pth

[vvssttkk]

plus, for the bpe i should use this tokenization for mlm_17_1280.pth?

[vvssttkk]

so, when i run to_bpe(sentences) i get next error
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/sentences.bpe'

tmp folder i should create my hands? and where i should take sentences.bpe?

sorry, but your notebook have many errors at path to another users

[vvssttkk]

also, what does it mean?
/private/home/aconneau/projects/XLM/data/wiki/17/175k/ i think this path are not available

[vvssttkk]

plus, at folder tools i didn't give fastBPE
i should install from here. Is it's true?

[vvssttkk]

solved errors and create new pr describing the steps