biobert for keras version of huggingface transformers

[dmollaaliod]

Hi, are there any instructions/hints about how to load biobert as a keras layer using the huggingface transformers? I tried several possibilities but none of these worked. All that I found out is how to use the pytorch version but I am interested in the keras layer version. Below are two of my attempts (I saved the biobert files into folder "biobert_v1.1_pubmed").

Attempt 1:

biobert_model = TFBertModel.from_pretrained('bert-base-uncased')
biobert_model.load_weights('biobert_v1.1_pubmed/model.ckpt-1000000')

Error message:

AssertionError: Some objects had attributes which were not restored:
    : ['tf_bert_model_4/bert/embeddings/word_embeddings/weight']
    : ['tf_bert_model_4/bert/embeddings/position_embeddings/embeddings']
   (and many more lines like above...)

Attempt 2:

biobert_model = TFBertModel.from_pretrained("biobert_v1.1_pubmed/model.ckpt-1000000", config='biobert_v1.1_pubmed/bert_config.json')

Error message:

NotImplementedError: Weights may only be loaded based on topology into Models when loading TensorFlow-formatted weights (got by_name=True to load_weights).

Any help appreciated! My experience with huggingface's transformers library is almost zero.

[dmollaaliod]

Someone answered in stackoverflow, it worked for me :)