This is the experimental code for the paper Development of comprehensive annotation criteria for patients’ states of clinical texts.
- Python 3.8+
- pandas 1.2.4
- numpy 1.20.1
- torch 1.10.1+cu113
- scikit-learn 0.24.1
- transformers 4.11.3
- seqeval 1.2.2
Download UTH-BERT here.
To train and evaluate a NER model, run
python main_ner.py --bert_path UTH-BERT --batch_size 16
To train and evaluate a RE model, run
python main_re.py --bert_path UTH-BERT
If you have a Google account, you are able to run our code in Google Colab. Please run the following code in Google Colab. Note that if you want to run an experiment with the same experimental setup as ours, you maight have to subscribe Colab Pro.
import os
! git clone https://github.com/aih-uth/UTH-29
! wget https://ai-health.m.u-tokyo.ac.jp/labweb/dl/uth_bert/UTH_BERT_BASE_512_MC_BPE_WWM_V25000_352K_pytorch.zip
! unzip UTH_BERT_BASE_512_MC_BPE_WWM_V25000_352K_pytorch.zip
! pip install transformers seqeval
os.chdir("./UTH-29")
! python main_ner.py --bert_path ../UTH_BERT_BASE_512_MC_BPE_WWM_V25000_352K --batch_size 16
! python main_re.py --bert_path ../UTH_BERT_BASE_512_MC_BPE_WWM_V25000_352K
Fold | NER | RE |
---|---|---|
1 | 0.917 | 0.859 |
2 | 0.913 | 0.849 |
3 | 0.926 | 0.855 |
4 | 0.925 | 0.847 |
5 | 0.917 | 0.849 |
Avg. | 0.920 | 0.852 |
CC BY-NC-SA 4.0
- Ma, Y., Hiraoka, T., & Okazaki, N. (2020). Named Entity Recognition and Relation Extraction using Enhanced Table Filling by Contextualized Representations. arXiv preprint arXiv:2010.07522. Software available from https://github.com/YoumiMa/TablERT.
- pytorch-crf. Software available from https://pytorch-crf.readthedocs.io/en/stable/.
- Hiroki Nakayama. seqeval: A python framework for sequence labeling evaluation, 2018. Software available from https://github.com/chakki-works/seqeval.
If you use our code in your work, please cite the following paper:
Emiko Shinohara, Daisaku Shibata, Yoshimasa Kawazoe. Development of comprehensive annotation criteria for patients’ states from clinical texts. J Biomed Inform. 2022 Sep; 104200 (in Press).