SSL

Patterns - Dimitris

review Chowdhury et al.

Vision

SimCLR — A Simple Framework for Contrastive Learning of Visual Representations ICML 2020 | code

• contrastive learning with cropping and color as augmentations
• maximizes agreement between differently transformed views of the same sample via a contrastive cosine similarity loss in the latent space

BYOL - Bootstrap Your Own Latent A New Approach to Self-Supervised Learning NeurIPS 2020

• not contrastive, done without negative pairs
• iteratively update the representations, use exponential history mean network as the target

Barlow twins - Barlow twins: Self-supervised learning via redundancy reduction PMLR | code *

Audio

survey - Audio Self-supervised Learning: A Survey Patterns Review 2022 *

COLA - Contrastive learning of general-purpose audio representations ICASSP 2021 | Google | code *

BYOL-Audio - BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation | code

• employed normalization and augmentations to modify BYOL for audio SSL
• might be useful: lots of parameters mentioned in the paper

Health domain

Time series

Neighborhood-based

NCL - Neighborhood Contrastive Learning Applied to Online Patient Monitoring ICML 2021 | code

TNC - Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding ICLR 2021 | code

Step2Heart

SelfHAR

CLOCS