Hyperspherical Consistency Regularization

This repository contains the implementation code for paper:

Hyperspherical Consistency Regularization
Cheng Tan, Zhangyang Gao, Lirong Wu, Siyuan Li, Stan Z. Li. In CVPR, 2022.

Brief introduction

In this work, we explore the relationship between self-supervised learning and supervised learning, and study how self-supervised learning helps robust data-efficient deep learning. We propose hyperspherical consistency regularization (HCR), a simple yet effective plug-and-play method, to regularize the classifier using feature-dependent information and thus avoid bias from labels. Specifically, HCR first projects logits from the classifier and feature projections from the projection head on the respective hypersphere, then it enforces data points on hyperspheres to have similar structures by minimizing binary cross entropy of pairwise distances' similarity metrics.

We consider the pairwise distance as the key geometry property, and force points on the classifier's hypersphere to have a similar structure as the projection head's, as follows:

Dependencies

• torch

Usage

The following Python code is all you need.

from hcr import HCR

hcr_reg = HCR(classifier_network, lr)

for epoch in epochs:
    for batch in dataloader:
        # get logits and projections as the input of HCR
        hcr_reg.update(logits, projections)

Citation

If you are interested in our repository and our paper, please cite the following paper:

@InProceedings{Tan_2022_CVPR,
    author    = {Tan, Cheng and Gao, Zhangyang and Wu, Lirong and Li, Siyuan and Li, Stan Z.},
    title     = {Hyperspherical Consistency Regularization},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {7244-7255}
}

Or,

@article{tan2022hyperspherical,
  title={Hyperspherical Consistency Regularization},
  author={Tan, Cheng and Gao, Zhangyang and Wu, Lirong and Li, Siyuan and Li, Stan Z},
  journal={arXiv preprint arXiv:2206.00845},
  year={2022}
}

Contact

If you have any questions, feel free to contact me through email (tancheng@westlake.edu.cn). Enjoy!