"unexpected EOF. The file might be corrupted" when loading model

[pwichmann]

I repeatedly run into the following error on one of my machines:

Traceback (most recent call last): File "/var/www/scminer/live_extractor/views.py", line 138, in process_text tagger = FlairSequenceTagger.load('ner') File "/root/anaconda3/envs/scminer_live/lib/python3.6/site-packages/flair/models/sequence_tagger_model.py", line 488, in load tagger: SequenceTagger = SequenceTagger.load_from_file(model_file) File "/root/anaconda3/envs/scminer_live/lib/python3.6/site-packages/flair/models/sequence_tagger_model.py", line 131, in load_from_file state = torch.load(model_file, map_location={'cuda:0': 'cpu'}) File "/root/anaconda3/envs/scminer_live/lib/python3.6/site-packages/torch/serialization.py", line 303, in load return _load(f, map_location, pickle_module) File "/root/anaconda3/envs/scminer_live/lib/python3.6/site-packages/torch/serialization.py", line 476, in _load deserialized_objects[key]._set_from_file(f, offset, f_is_real_file) RuntimeError: unexpected EOF. The file might be corrupted.

Environment:
• Ubuntu 16.04
• Anaconda 5.2.0
• Python 3.6
• Latest version of flair (Version: 0.2.1 according to pip show)

I have tried reinstalling flair multiple times and in different virtual environments.
I know it should generally work since it runs on another machine of mine.
Reached end of my wit.

Any idea what might cause this error?

[pwichmann]

[alanakbik]

Hi, pwichmann, thanks for your interest!

One possibility is that something happened during the first download of the model (abort, corruption, not enough disc space etc.), see: https://discuss.pytorch.org/t/loading-pretrained-model-error/10315

Flair models are stored at ~/.flair/models/ - could you try deleting this folder and calling the tagger again? This should trigger a new download.

[pwichmann]

UPDATE:
It was indeed a memory error (insufficient disc space) where a previous model download was incomplete and was not corrected in subsequent FLAIR uses.

I could correct the error following the advice described above. Note that the link does not point to where flair is installed (e.g. in your virtual env). It is indeed '''~/.flair'''.

Many thanks, @alanakbik !
Amazingly fast response and merci for the great tool.

[alanakbik]

Great, thanks!

[krzysztoffiok]

Hi,

i have most probably the same issue with one of the LM (XLNetEmbeddings()).

I looked into '''~/.flair''' there is a folder called "embeddings" and inside i see only:
en-fasttext-news-300d-1M glove.gensim.vectors.npy
en-fasttext-news-300d-1M.vectors.npy news-backward-0.4.1.pt
glove.gensim news-forward-0.4.1.pt

Is there any other place where flair stores the downloaded LMs? I have downloaded and used successfully other i.e. RoBERTaEmbeddings(), OpenAIGPT2Embeddings() so they should be somewhere there but i can't find them.

I'd appreciate some help!