Fine-tuning SPECTER?

[jacklxc]

1. Is there any way that instead of training from SciBERT, but directly fine-tune on SPECTER? It seems that the format of the model weights of SPECTER is different from SciBERT.

2. How do I fine-tune SPECTER like SciBERT on classification tasks?

[ZzyChris97]

I have the same problem, how did you solve this

[armancohan]

The model that is on huggingface should be easily fine-tunable like SciBERT.
You can follow instructions here https://huggingface.co/docs/transformers/training but instead of bert-base-uncased use allenai/specter as the pre-trained model name.

[gabriead]

How does a custom training dataset has to look like? I understand from the repo that it consists of title+abstract+id for each paper
in metadata.json but I don't understand was data.json does? Are those the positive and negative examples of papers for each paper?