BIO-R-BERT

Model

Dataset

• DDI (Drug-Drug Interaction) 2013 dataset (link)
  • Relation Extraction task on Bioinformatics
  • 175 MEDLINE abstracts and 730 DrugBank documents
  • 5 DDI types (Negative, Mechanism, Effect, Advice, Int)
  • Use the preprocessed dataset from this repo
  • Didn't replace the name of drug to DRUG0, DRUG1, or DRUGN, comparing to other researches

How to use BioBERT for Transformers library

>>> from transformers import BertModel, BertTokenizer
>>> model = BertModel.from_pretrained('monologg/biobert_v1.1_pubmed')
>>> tokenizer = BertTokenizer.from_pretrained('monologg/biobert_v1.1_pubmed')

Dependencies

• python>=3.5
• torch==1.1.0
• transformers>=2.2.2
• scikit-learn>=0.20.0

$ pip3 install -r requirements.txt

How to run

You must give --do_lower_case option if pretrained model is uncased model.

$ python3 main.py --do_train --do_eval

Results

F1 micro score on 4 Positive types (Mechanism, Effect, Advice, Int)

	F1 micro (%)
CNN	69.75
AB-LSTM	69.39
MCCNN	70.21
GCNN	72.55
Recursive NN	73.50
RHCNN	75.48
SMGCN	76.64
BIO-R-BERT	82.66
_BIO-R-BERTGaus	83.35

References

• BioBERT
• Huggingface Transformers
• R-BERT Repository
• DDI-recursive-NN Repository
• AB-LSTM Repository
• MCCNN Repository
• RHCNN Repository