Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding

Notes

The code implementation of this paper is mainly referenced in the following papers.

1. [Arxiv] Whitening Sentence Representations for Better Semantics and Faster Retrieval

2. [Coling 2022] Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective

3. [EMNLP 2020] On the Sentence Embeddings from Pre-trained Language Models

Usage

python main_sl.py 
or
sh mnli_run.sh

Citation

@article{gao2022kernel,
  title={Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding},
  author={Gao, Songyang and Dou, Shihan and Zhang, Qi and Huang, Xuanjing},
  journal={arXiv preprint arXiv:2210.07547},
  year={2022}
}