Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 

README.md

Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering

This code is for ECML-PKDD 2020 paper "Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering".

In this paper, we introduce a model-agnostic framework, DaFeC, to improve domain adaptation performance for few-shot classification via clustering. We first build a representation extractor to derive features for unlabeled data from the target domain (no test data is necessary) and then group them with a cluster miner. The generated pseudo-labeled data and the labeled source-domain data are used as supervision to update the parameters of the few-shot classifier. In order to derive high-quality pseudo labels, we propose a Clustering Promotion Mechanism, to learn better features for the target domain via Similarity Entropy Minimization and Adversarial Distribution Alignment, which are combined with a Cosine Annealing Strategy. Experiments are performed on the FewRel 2.0 dataset. Our approach outperforms previous work with absolute gains (in classification accuracy) of 4.95%, 9.55%, 3.99% and 11.62%, respectively, under four few-shot settings.

You can find the paper here.

Requirements

Python=3.7

PyTorch=1.1.0

CUDA=9.0

Transformers=2.7.0

Preparation

The training and dev dataset have been included in the ./data directory and the test set is not public. You should evaluate your models in the offical website.

Due to the large size, the pre-trained Glove files and BERT pretrain checkpoint are not included. You can download them in here.

Usage

  1. Training Representation Extractor
python train_demo.pys --save_ckpt=path_to_your_saved_model
  1. Generating pseudo-labeled data
python train_demo.py --load_ckpy=path_to_your_saved_model --cluster
  1. Training few-shot classifier (BERT-PAIR as the example)
python train_demo.py --model=pair --pair --encoder=bert --hidden_size=768 --optim=adamw --lr=2e-5 --train=train_wiki_and_pseudo_pubmed --save_ckpt=path_to_your_saved_model
  1. Evaluation Please evaluate the performance of your model in the FewRel 2.0 official website.

Citation

@inproceedings{cong2020DaFeC,
 author = {Cong, Xin and Yu, Bowen and Liu, Tingwen and Cui, Shiyao and  Tang, Hengzhu and Wang, Bin},
 booktitle = {Proc. of ECML-PKDD},
 title = {Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering},
 year = {2020}
}

Related Repo

The dataset and baselines are adapted from FewRel.

About

The code of "Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering", ECML-PKDD 2020.

Resources

Releases

No releases published

Packages

No packages published

Languages