Allow for local embedding files

[joelkuiper]

Right now the code path is to download some pretrained embedding from a remote source. However I have some domain specific embeddings that are in the standard gensim format. Would it be possible to support loading word embeddings simply from disk?

[alanakbik]

Hi Joel,

yes that is possible. If your embeddings are stored as gensim KeyedVectors, you can simply point the WordEmbedding class in the constructor to the head file.

embeddings = WordEmbeddings('path/to/KeyedVectors/file')

If your file is not a KeyedVector, you can use gensim to create it from a normal word embedding text file. First, load the text file:
vectors = gensim.models.KeyedVectors.load_word2vec_format('path/to/text/file', binary=False)

Second, save as KeyedVector:
vectors.save('path/to/keyed/vector', pickle_protocol=4)

Done! Then, you can init the WordEmbeddings object as above by pointing to this file.

Does that work for you?

[joelkuiper]

Oh I must've missed that somewhere. Cool 👍

[yuquanle]

I download the glove vector from https://nlp.stanford.edu/projects/glove/. Then I follow the above. But the final results is null (like this "tensor([])").

[alanakbik]

Hi @yuquanle could you give more information on how you are converting and loading the word embeddings? Could you paste your whole script? Also, which version of Flair are you using?