Source code of IncreProtoNet Model for Incremental Few-shot Relation Classification

Inplementation of Our paper "A Two-phase Prototypical Network Model for Incremental Few-shot Relation Classification" in COLING 2020

Requirements

• pytorch = 1.0.1
• json
• argparse
• sklearn
• transformers==3.0.2

Model Architecture

Usage

Preparing Dataset and Pretrain files

Due to the large size of glove files (pre-trained word embeddings). Please download pretrain.tar from Tsinghua Cloud and put it under the data directory. Then run tar xvf pretrain.tar to decompress it.

BaseModel Training

Open the directory "IncreProtoNet/BaseModel", and run the python file "train_demo.py". The saved model are in the directory "IncreProtoNet/BaseModel/checkpoint/". The specific commands and several important hyper-parameters are as follows:

Glove Embedding Command

 python3 train_demo.py --embedding_type glove --lr 1e-1

Bert Embedding Commain

 python3 train_demo.py --embedding_type bert --lr 1e-2 --max_length 60

Result

embedding_type	learn_rate	pl_weight	train_iter	ACC
glove	1e-1	1e-1	25000	78.23
bert	1e-2	1e-1	8000	88.45

Incremental Model Training

Open the directory "IncreProtoNet/IncrementalFewShotModel", and run the python file "train_demo.py". The saved model are in the directory "IncreProtoNet/IncrementalFewShotModel/checkpoint/". The specific commands and several important hyper-parameters are as follows:

Glove Embedding Command

1-shot

python3.6 train_demo.py --embedding_type glove --pl_weight 0.1 \
--base_lr 0.1 --train_baseN 30 --triplet_num 10 --margin 5.0 \
--learn_rate 1e-1 --K 1 

5-shot

python3.6 train_demo.py --embedding_type glove --pl_weight 0.1 \
--base_lr 0.1 --train_baseN 30 --triplet_num 10 --margin 5.0 \
--learn_rate 1e-1 --K 5

Setting	Base	Novel	Both
1-shot	70.96	48.38	69.36
5-shot	72.54	61.57	71.54

BERT Embedding Command

1-shot

python3 train_demo.py --embedding_type bert \
--pl_weight 0.1 --base_lr 0.01 --train_baseN 30 --triplet_num 10 --margin 10.0 \
--learn_rate 1e-3 --K 1 --val_step 100 --val_iter 100 --max_length 60

5-shot

python3 train_demo.py --embedding_type bert --pl_weight 0.1 --base_lr 0.01 \
--train_baseN 30 --train_novelN 10 --triplet_num 20 --margin 3.0 --learn_rate 2e-3 \
--K 5 --val_step 100 --val_iter 100 --max_length 60 --test_Q 1 

Setting	Base	Novel	Both
1-shot	82.10	60.15	80.65
5-shot	84.64	65.77	82.26