Pytorch implementation of Maximum Manifold Capacity Representations Loss

This is not an official implementation from the authors. Official implementation from the authors.

Maximum Manifold Capacity Representation Loss (MMCR Loss) is a novel objective function for self-supervised learning (SSL) proposed by researchers in Center for Neural Science, NYU.

This repository aims to offer a convenient MMCR loss module for PyTorch, which can be easily integrated into your projects using git clone or pip install.

How to install

pip3 install mmcr

or

git clone https://github.com/skyil7/mmcr
cd mmcr
pip install -e .

Usage

import torch
from mmcr import MMCRLoss

loss = MMCRLoss()

input_tensor = torch.randn((8, 16, 128))  # batch_size, n_aug, feature_dim
loss_val = loss(input_tensor)

print(loss_val)

How it works

$$\mathcal{L} = \lambda\frac{\sum^{N}_{i=1}\lVert z_{i} \rVert_{*}}{N} - \lVert C\rVert_{*}$$

Where $\lambda$ is a trade-off parameter, $\lVert z_i\rVert_*$ is local nuclear norm of the $i$-th sample's augmented matrix, and $\lVert C\rVert_*$ is the global nuclear norm of centroid matrix $C$.

Arguments

• lmbda: Trade-off parameter $\lambda$. default is 0.
• n_aug: number of augmented views. If your input tensor is 3-dimensional $(N, k, d)$, you don't need to specify it.

Original Implementation from the author

• This repository was developed with reference to the official implementation provided by the authors.