awesome_video_self_supervised

awesome video-based self-supervised learning methods in recently years

✨ :important work(5) 🚀 :interesting work(4) 👍 :good work(3) 🐫 :another paper(2) :not recommed(1)

Year	Important work	Interesting work	good work
2020		VTDL, CoCLR, DCS, SpeedNet	ClusterFit, Evolving Losses, Video_Playbck
2019		Time Cycle
2018			Time Contrastive

2020

Arxiv (with code or interesting)

1.Self-supervised Temporal Discriminative Learning for Video Representation Learning 🚀

Author: Jinpeng Wang, Yiqi Lin, Andy J. Ma, and Pong C. Yuen

Institutions: SYSU， HKBU

Code: github

Paper: arxiv

Summary: Novel Temporal Consistent Augmentation (TCA).

2.Temporally Coherent Embeddings for Self-Supervised Video Representation Learning

Author: Joshua Knights, Ben Harwood, Daniel Ward, Anthony Vanderkop, Olivia Mackenzie-Ross, Peyman Moghadam

Institutions: Robotics and Autonomous Systems, Data61 CSIRO, Brisbane, QLD 4069, Australia

Code: github

Paper: arxiv

3. Representation Learning with Video Deep InfoMax

Author: R Devon Hjelm, Philip Bachman

Institutions: Microsoft Research

Paper: arxiv

Summary: Introduce Deep InfoMax into video.

4. Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases 🚀

Author: Senthil Purushwalkam, Abhinav Gupta

Institutions: Carnegie Mellon University, Facebook AI Research

Code:

Paper: arxiv

Summary: Explore dataset bias in contrastive learning.

CVPR

1.ClusterFit: Improving Generalization of Visual Representations 👍

Authors:Xueting Yan∗ Ishan Misra∗ Abhinav Gupta Deepti Ghadiyaramy Dhruv Mahajany

Institutions: Facebook AI

Code:

Paper: arxiv

2.Evolving Losses for Unsupervised Video Representation Learning 👍

Authors:AJ Piergiovanni, Anelia Angelova, Michael S. Ryoo

Institutions: Google Research

Code:

Paper: arxiv

3.Multi-Modal Domain Adaptation for Fine-Grained Action Recognition (Oral) 🐫

Authors:Jonathan Munro, Dima Damen

Institutions: University of Bristol

Code:

Paper: arxiv

4.SpeedNet: Learning the Speediness in Videos (Oral) 🚀

Authors: Sagie Benaim, Ariel Ephrat, Oran Lang, Inbar Mosseri, William T. Freeman, Michael Rubinstein, Michal Irani, Tali Dekel

Institutions: Google Brain

Code:

Paper: arxiv

5.Video Playback Rate Perception for Self-supervised Spatio-Temporal Representation Learning 👍

Authors: Yuan Yao1∗, Chang Liu1∗, Dezhao Luo2, Yu Zhou2 and Qixiang Ye1†

Institutions: University of Chinese Academy of Sciences, Chinese Academy of Sciences

Code:

Paper: cvf

ECCV

1.Video Representation Learning by Recognizing Temporal Transformations 🐫

Authors: Simon Jenni, Givi Meishvili, and Paolo Favaro

Institutions: University of Bern, Switzerland

Code:

Paper: arxiv

2.Self-supervised Video Representation Learning by Pace Prediction 🐫

Authors: Jiangliu Wang, Jianbo Jiao, and Yun-Hui Liu

Institutions: The Chinese University of Hong Kong, University of Oxford

Code:

Paper: arxiv

ICLR

AAAI

NIPS

1. Self-supervised Co-Training for Video Representation Learning 🚀

Authors: Tengda Han, Weidi Xie, Andrew Zisserman

Institutions: VGG Group, Oxford

Code: github

Paper: arxiv

Others

2019

CVPR

oral

1.Learning Correspondence from the Cycle-Consistency of Time 🚀

Authors:Xiaolong Wang and Allan Jabri and Alexei A. Efros

Institutions:

Code: github

Paper: arxiv

poster

1. Self-supervised Spatiotemporal Learning via Video Clip Order Prediction 🐫

Authors: Dejing Xu1 Jun Xiao1 Zhou Zhao1 Jian Shao1 Di Xie2 Yueting Zhuang1

Institutions: 1Zhejiang University 2Hikvision Research Institute

Code: github

Paper: openaccess

workshop

ICCV

oral

poster

workshop

1.Skip-Clip: Self-Supervised Spatiotemporal Representation Learning by Future Clip Order Ranking

Authors: Alaaeldin El-Nouby1,4∗ Shuangfei Zhai 2 Graham W. Taylor1,3,4 Joshua M. Susskind2

Institutions: 1 University of Guelph 2 Apple Inc.

Code:

Paper: arxiv

ICLR

AAAI

oral

poster

1.Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles 🐫

Authors: Dahun Kim, Donghyeon Cho, In So Kweon

Institutions: Dept. of Electrical Engineering, KAIST, Daejeon, Korea

Code:

Paper: arxiv

NIPS

Others

2018

1.Time-Contrastive Networks: Self-Supervised Learning from Video 👍

Authors: Pierre Sermanet1*@ Corey Lynch1R* Yevgen Chebotar2*, Jasmine Hsu1 Eric Jang1 Stefan Schaal2 Sergey Levine

Institutions: Google Brain, University of Southern California

Code:

Paper: arxiv

Others