HCRP

This is the implementation of our paper Exploring Task Difficulty for Few-Shot Relation Extraction.

Requirements

• python 3.6
• PyTorch 1.7.0
• transformers 4.0.0
• numpy 1.19

Datasets

We experiment our model on two few-shot relation extraction datasets,

1. FewRel 1.0
2. FewRel 2.0

Please download data from the official links and put it under the ./data/.

Evaluation

Please download trained model from here and put it under the ./checkpoint/. To evaluate our model, use command

FewRel 1.0

python train.py \
    --N 10 --K 1 --Q 1 --test_iter 10000\
    --only_test True --load_ckpt "checkpoint/hcrp.pth.tar"

FewRel 2.0

python train.py \
    --N 10 --K 1 --Q 1 --test_iter 10000\
    --val val_pubmed --test val_pubmed --ispubmed True\
    --only_test True --load_ckpt "checkpoint/hcrp-da.pth.tar"

Training

FewRel 1.0

To run our model, use command

python train.py

This will start the training and evaluating process of HCRP in a 10-way-1-shot setting. You can also use different args to start different process. Some of them are here:

• train / val / test: Specify the training / validation / test set.
• trainN: N in N-way K-shot. trainN is the specific N in training process.
• N: N in N-way K-shot.
• K: K in N-way K-shot.
• Q: Sample Q query instances for each relation.

There are also many args for training (like batch_size and lr) and you can find more details in our codes.

FewRel 2.0

Use command

python train.py \
    --val val_pubmed --test val_pubmed --ispubmed True --lamda 2.5

Results

FewRel 1.0

	5-way-1-shot	5-way-5-shot	10-way-1-shot	10-way-5-shot
Val	90.90	93.22	84.11	87.79
Test	93.76	95.66	89.95	92.10

FewRel 2.0

	5-way-1-shot	5-way-5-shot	10-way-1-shot	10-way-5-shot
Val	78.90	83.22	68.99	74.45
Test	76.34	83.03	63.77	72.94

Cite

If you use the code, please cite the following paper: "Exploring Task Difficulty for Few-Shot Relation Extraction" Jiale Han, Bo Cheng and Wei Lu. EMNLP (2021)

@inproceedings{han2021exploring,
    title = {Exploring Task Difficulty for Few-Shot Relation Extraction},
    author = {Han, Jiale and Cheng, Bo and Lu, Wei},
    booktitle = {Proc. of EMNLP},
    year={2021}
}