Model name not found or config.json missing

[hzitoun]

Transformers version
2.3.0
Pytorch version
1.4

When running readme.md code

import torch
from transformers import XLMModel, XLMTokenizer
modelname="xlm_bert_fra_base_lower" # Or absolute path to where you put the folder

# Load model
flaubert, log = XLMModel.from_pretrained(modelname, output_loading_info=True)
# check import was successful, the dictionary should have empty lists as values
print(log)

# Load tokenizer
flaubert_tokenizer = XLMTokenizer.from_pretrained(modelname, do_lowercase_and_remove_accent=False)

sentence="Le chat mange une pomme."
sentence_lower = sentence.lower()

token_ids = torch.tensor([flaubert_tokenizer.encode(sentence_lower)])
last_layer = flaubert(token_ids)[0]
print(last_layer.shape)

Output

OSError: Model name 'xlm_bert_fra_base_lower' was not found in model name list 
(xlm-mlm-en-2048, xlm-mlm-ende-1024, xlm-mlm-enfr-1024, xlm-mlm-enro-1024, xlm-mlm-tlm-xnli15-1024, xlm-mlm-xnli15-1024, xlm-clm-enfr-1024, xlm-clm-ende-1024,
 xlm-mlm-17-1280, xlm-mlm-100-1280). We assumed 'xlm_bert_fra_base_lower' 
was a path or url to a configuration file named config.json 
or a directory containing such a file but couldn't find any such file at this path or url.

Tried downloading the model as tar file (lower and normal), extracting it and putting its absolute folder path as modelname but keeps getting the same error....

I may be missing something so stupid but I don't see a config.json file in the archive file

What is wrong?

[mcoavoux]

Hi,

the archive flaubert_base_lower.zip is not directly usable with the transformers package, make sure you have downloaded the following archive: https://zenodo.org/record/3567594/files/xlm_bert_fra_base_lower.tar that contains the config.json file.

[hzitoun]

Hi,

the archive flaubert_base_lower.zip is not directly usable with the transformers package, make sure you have downloaded the following archive: https://zenodo.org/record/3567594/files/xlm_bert_fra_base_lower.tar that contains the config.json file.

Hi, thanks for your quick reply! Now it works :)

Does that mean that only models starting with xlm work with the transformers lib ?

Could we load the Flaubert from torch.hub ?

[formiel]

Hi @hzitoun,

We are uploading Flaubert’s pretrained models to Hugging Face’s repo, so very soon you’ll be able to load them in transformers without having to download them manually. We will upload them to PyTorch Hub as well, thanks for the suggestion!

[hzitoun]

@formiel that is great!

Are you planning as well to be part of the fairseq toolkit? (I don't know if only FAIR researchers can do that).

A bit like what CamemBERT guys did. Would give something like:

import torch
flaubert = torch.hub.load('pytorch/fairseq', 'flaubert.v1')
flaubert.eval() 
masked_line = 'Le ciel est <mask> aujourd'hui :)'  #a bit of imagination :) 
flaubert.fill_mask(masked_line, topk=5)

Finally, when running the example in readme, I'm getting the following warning:

/Users/distiller/project/conda/conda-bld/pytorch_1579022061893/work/aten/src/ATen/native/RangeFactories.cpp:153: 
UserWarning: The number of elements in the out tensor of shape [1] is 1 
which does not match the computed number of elements 6. Note that 
this may occur as a result of rounding error. The out tensor will be resized
 to a tensor of shape (6,).

Any idea?

Thank you for being so reactive! Eager to try your model on some semantic search tasks, will compare results to CamemBERT too :)

[formiel]

Are you planning as well to be part of the fairseq toolkit? (I don't know if only FAIR researchers can do that).

Thanks for the suggestion. That's also planned but unfortunately we have no ETA yet.

Finally, when running the example in readme, I'm getting the following warning

We have integrated Flaubert into Hugging Face's transformers. Could you please try again by following this example?

[schwabdidier]

assume that you got your answer @hzitoun ?