Materials for transfer learning 中文版, English version
将一些本领域中的经典论文按照时间线进行分类、梳理，分为浅层域适应、深度域适应、对抗域适应和域适应领域四部分。
针对每一部分，列举了3-4篇经典论文，建议详读这些经典论文，泛读这些经典论文的后续论文，并对其中的部分算法进行实现。
预期学习时间为2-3个月, 详细计划安排见入门参考
围绕这些论文，曾有一个相应的讨论班，相关的日程和资料如下：
- Mingsheng Long THU
- Fuzhen Zhuang CAS
- Yu Zhang SUST
- Wen Li UESTC
- Jindong Wang MSRA
- Lei Zhang CQUST
- Judy Hoffman Georgia Tech
- Kate Aaenko Boston University
- Sinno Jialin Pan NTU
- Kuniaki Saito Boston University(Ph.D)
- Zhao Han CMU
- Boqing Gong Google Research
- Ma Qianzu cognitive model link
- Harry Shum you are what you read link
- Harry Shum how to read papers 7.18(pravite), content
- Jingdong Wang how to read papers 7.21(pravite, same password with the above)
- Qiang Yang， From 0 to 1, how does transfer learning reach today's peak？link
- VALSE Webinar 20-19 Transfer learning (It is strongly recommended! It is difficult for the beginers but helpful for improving! It is of high quality) video, introduction
- Mingsheng Long_NJU2019 Transfer Learning Theories and Algorithms
- Mingsheng Long Valse 2019 Transfer Learning_From Algorithms to Theories and Back
- Kaichao You BAAI Live 2019 Advances in Domain Adaptation: Setting, Method and Model Selection
- Mei Wang 2019 deep_domain_adaptation
- Andrew Y.Ng NIPS 2016 Nuts and bolts of building AI applications using Deep Learning
| number | Title | Conference/journel + year | Code | Keywords | Benenit for us |
|---|---|---|---|---|---|
| 84 | WHAT MAKES INSTANCE DISCRIMINATION GOOD FOR TRANSFER LEARNING? (paper) | ICLR 2021 | contranstive learning, TL | new findings | |
| 83 | Self-Supervised Policy Adaptation during Deployment (paper) | ICLR 2021 | RL, adaptation | ||
| 82 | Tent: Fully Test-Time Adaptation by Entropy Minimization (paper) | ICLR 2021 | test-time adaptation | ||
| 81 | Improving Zero-Shot Voice Style Transfer via Disentangled Representation Learning (paper) | ICLR 2021 | voive style transfer | ||
| 80 | Scalable Transfer Learning with Expert Models (paper) | ICLR 2021 | multi-source | ||
| 79 | Distance-Based Regularisation of Deep Networks for Fine-Tuning(https://openreview.net/pdf?id=IFqrg1p5Bc) | ICLR 2021 | fine-tune | ||
| 78 | Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosis (paper) | AAAI 2021 | UDA, application | similar idea with us | |
| 77 | How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches? (paper) | AAAI 2021 | code | UDA, the thidr term of theory | |
| 76 | Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation (paper) | AAAI 2021 | group alignment, UDA | similar idea with us | |
| 75 | Bi-Classifier Determinacy Maximization for Unsupervised Domain Adaptation (paper) | AAAI 2021 | UDA | improvement for MCD | |
| 74 | A Free Lunch for Unsupervised Domain Adaptive Object Detection without Source Data (paper) | AAAI 2021 | source-free, Objective detection | ||
| 73 | Incremental Adversarial Domain Adaptation for Continually Changing Environments (paper) | ICRA 2018 | continual DA | new question | |
| 72 | ADAPTING TO CONTINUOUSLY SHIFTING DOMAINS (paper) | ICLR 2018 workshop | continual DA | new question | |
| 71 | Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay (paper) | arvix 2020 | continual DA | new question | |
| 70 | Continual Learning for Domain Adaptation in Chest X-ray Classification (paper) | MLR 2020(under review) | continual DA | new question | |
| 69 | Continual Domain Adaptation for Machine Reading Comprehension (paper) | CIKM 2020 | continual DA | new question | |
| 68 | Continual Unsupervised Domain Adaptation with Adversarial Learning (paper) | arvix 2020 | continual DA | new question | |
| 67 | Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning (paper) | arvix 2020 | continual DA | new question | |
| 66 | Unsupervised Domain Adaptation without Source Data by Casting a BAIT (paper) | arvix 2020 | source-free DA, prototype | good idea | |
| 65 | A Review of Single-Source Deep Unsupervised Visual Domain Adaptation (paper) | arvix 2020 | DA survey | good further directions | |
| 64 | Discover, Hallucinate, and Adapt: Open Compound Domain Adaptation for Semantic Segmentation (paper) | NeruIPS 2020 | open compound, DA | new problem | |
| 63 | Your Classifier can Secretly Suffice Multi-Source Domain Adaptation (paper) | NeruIPS 2020 | code | MS, prediction agreement | simple yet effective method, new findings |
| 62 | Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID (paper) | NIPS 2020 | code | contrastive learning, DA, Re-ID | contrastive learning + DA |
| 61 | Unsupervised Domain Adaptation without Source Data by Casting a BAIT(paper) | Arvix 2020 | source-free, two classifiers | good idea | |
| 60 | An Adversarial Domain Adaptation Network for Cross-Domain Fine-Grained Recognition(paper) | WACV 2020 | code | fine-grained, DA | new question |
| 59 | Class-incremental Learning via Deep Model Consolidation (paper) | WACV 2020 | |||
| 58 | Impact of ImageNet Model Selection on Domain Adaptation(paper) | WACV 2020 workshop | shallow methods with different deep features | 实验结果很迷惑 | |
| 57 | Measuring Information Transfer in Neural Networks (paper) | arvix 2020 | maybe useful for DA | ||
| 56 | Open-Set Hypothesis Transfer with Semantic Consistency (paper) | arvix 2020 | source free, open set | ||
| 55 | Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks(paper) | arvix 2020 | pretraining | good papers | |
| 54 | Measuring Information Transfer in Neural Networks(paper) | interesting paper | |||
| 53 | When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets(paper) | SSL, TL, experiments | many results related to multiple SSL methods can be seen in this paper | ||
| 52 | Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling (paper) | ICML 2020 | stein discrepancy | a new metric that is never used in DA | |
| 51 | Graph Optimal Transport for Cross-Domain Alignment (paper) | ICML 2020 | Graph, optimal transport, DA | ||
| 50 | Unsupervised Transfer Learning for Spatiotemporal Predictive Networks (paper) | ICML 2020 | |||
| 49 | Estimating Generalization under Distribution Shifts via Domain-Invariant Representations (paper) | ICML 2020 | code | new theory | recommend to read |
| 48 | Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation (paper) | ICML 2020 | code | ideas from theory | recommend to read |
| 47 | LEEP: A New Measure to Evaluate Transferability of Learned Representations (paper) | ICML 2020 | new metric for transferability | easy to use for other tasks | |
| 46 | Label-Noise Robust Domain Adaptation | ICML2020 | the author is a rising star | ||
| 45 | Progressive Graph Learning for Open-Set Domain Adaptation (paper) | ICML 2020 | code | open set DA | |
| 44 | Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (paper) | ICML 2020 | code | source-free DA | recommend to read, new trneds |
| 43 | Graph Optimal Transport for Cross-Domain Alignment (paper) | ICML 2020 | graph for DA | connenction with GCN | |
| 42 | Learning Deep Kernels for Non-Parametric Two-Sample Tests (paper) | ICML 2020 | code | extend MMD to deep | |
| 41 | Adversarial-Learned Loss for Domain Adaptation | AAAI 2020 | noisy label, adversarial learning | ||
| 40 | Transfer Learning for Anomaly Detection through Localized and Unsupervised Instance Selection | AAAI 2020 | transfer learning, anamaly detection | ||
| 39 | Dynamic Instance Normalization for Arbitrary Style Transfer | AAAI 2020 | dynamic instance normalization | ||
| 38 | AdaFilter: Adaptive Filter Fine-Tuning for Deep Transfer Learning | AAAI 2020 | gated output, fine-tune | ||
| 37 | Bi-Directional Generation for Unsupervised Domain Adaptation | AAAI 2020 | differert feature extractor, different classifiers | connection with ICML, the third term | |
| 36 | Discriminative Adversarial Domain Adaptation | AAAI 2020 | discriminative information with adversarial learning | ||
| 35 | Domain Generalization Using a Mixture of Multiple Latent Domains | AAAI 2020 | |||
| 34 | Multi-Source Distilling Domain Adaptation | AAAI 2020 | multi-source | ||
| 33 | Cross-Modal Cross-Domain Moment Alignment Network for Person Search (paper) | CVPR 2020 | cross-modal, DA, Person search | new problem | |
| 32 | Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision | CVPR 2020 | code | Entropy adversarial based | |
| 31 | Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective | CVPR 2020 | long-tailed | ||
| 30 | Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering | CVPR 2020 | code | cluster | |
| 29 | Stochastic Classifiers for Unsupervised Domain Adaptation | CVPR 2020 | stochastic two classifiers | simialer to MCD | |
| 28 | Progressive Adversarial Networks for Fine-Grained Domain Adaptation | CVPR 2020 | fine-grained | similar to mutil-aspect opinion analysis | |
| 27 | Model Adaptation: Unsupervised Domain Adaptation without Source Data | CVPR 2020 | Recommend to read, new problems | ||
| 26 | Towards Inheritable Models for Open-Set Domain Adaptation | CVPR 2020 | code | ||
| 25 | Unsupervised Domain Adaptation with Hierarchical Gradient Synchronization (paper) | CVPR 2020 | class gropu, DA | new idea | |
| 24 | Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptation (paper) | ECCV 2020 | code | SSDA, intar-domain discrepancy | good questions |
| 23 | Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification | ECCV 2020 | |||
| 22 | Extending and Analyzing Self-Supervised Learning Across Domains (paper) | ECCV 2020 | |||
| 21 | Dual Mixup Regularized Learning for Adversarial Domain Adaptation (paper) | ECCV 2020 | |||
| 20 | Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation (paper | ECCV 2020 | code | SSL reguralization, Anchors | new methods, good writings |
| 19 | Class-Incremental Domain Adaptation(paper) | ECCV 2020 | new problems | ||
| 18 | Simultaneous Semantic Alignment Network for Heterogeneous Domain Adaptation (paper) | ACM MM 2020 | code | ||
| 17 | Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition (paper) | ACM MM 2020 | similar idea with us | ||
| 16 | Do Adversarially Robust ImageNet Models Transfer Better? | arvix 2020 | code | Many experiments | |
| 15 | Visualizing Transfer Learning | arvix 2020 | interesting | ||
| 14 | A SURVEY ON DOMAIN ADAPTATION THEORY:LEARNING BOUNDS AND THEORETICAL GUARANTEES (paper) | arvix 2020 | theory | ||
| 13 | Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification (paper) | arvix 2020 | Good ideas | ||
| 12 | Towards Recognizing Unseen Categories in Unseen Domains (paper) | arvix 2020 | new problems | ||
| 11 | MiCo: Mixup Co-Training for Semi-Supervised Domain Adaptation (paper) | arvix 2020 | good framework | ||
| 10 | Dynamic Knowledge Distillation for Black-box Hypothesis Transfer Learning (paper | arvix 2020 | |||
| 9 | Learning from a Complementary-label Source Domain: Theory and Algorithms(paper) | arvix 2020 | code | novel idea | |
| 8 | A Review of Single-Source Deep Unsupervised Visual Domain Adaptation paper | arvix 2020 | Review | a good review! It contains many results of the state-of-the-art method | |
| 7 | Neural transfer learning for natural language processing（paper) | 2019 PDH thesis | NLP, transfer lerning | very detailed related work | |
| 6 | Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation (paper) | ICCV 2019 | code | Cluster assumption, DA | deal with misclassified samples |
| 5 | SpotTune: Transfer Learning through Adaptive Fine-tuning (paper) | CVPR 2019 | code | dynamic routing is a general method | |
| 4 | Parameter Transfer Unit for Deep Neural Networks (paper) | PAKDD 2019 best paper | good idea, recommened to read | ||
| 3 | Heterogeneous Domain Adaptation via Soft Transfer Network (paper) | ACM MM 2019 | |||
| 2 | DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patterns (paper) | IJCAI 2019 | DA, cross-domain recommendation | classical work | |
| 1 | Adversarial Domain Adaptation with Domain Mixup (paper) | IJCAI 2019 | mix-ip, DA | new idea | |
| 0 | Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation (paper) | ICML 2012 |
| number | Title | Conference/journel + year | Code | Keywords | Benenit for us |
|---|---|---|---|---|---|
| 16 | Wasserstein-2 Generative Networks (paper) | ICLR 2021 | GAN， wassertein | ||
| 15 | Prototypical Contrastive Learning of Unsupervised Representations(https://openreview.net/pdf?id=KmykpuSrjcq) | ICLR 2021 | prototype, constractive learning | maybe for UDA | |
| 14 | Self-Loop Uncertainty: A Novel Pseudo-Label for Semi-Supervised Medical Image Segmentation (paper) | MICCAI 2020 | ssl, pseudo label | ||
| 13 | Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning (paper) | NIPS 2020 | semi-supervised, weight smaples | it can be used in our work | |
| 12 | Safe semi-supervised learning: a brief introduction (paper) | safe ssl | new concept, maybe useful for negative transfer | ||
| 11 | Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data (paper) | ICML 2020 | code | ssl, unseen class | open set, maybe useful for negative transfer |
| 10 | (RECORD: Resource Constrained Semi-Supervised Learning under Distribution Shifpaper) | KDD 2020 | online, distribution shift | maybe useful for negative transfer | |
| 9 | Adversarial Examples Improve Image Recognition (paper) | CVPR 2020 | Adversarial examples, image recognition, batch normalization | Same idea can be explored in DA | |
| 8 | Collaborative Graph Convolutional Networks: Unsupervised Learning Meets Semi-Supervised Learning | AAAI 2020 | unsupervised learning, semi-supervised learning | ||
| 7 | Self-supervised Label Augmentation via Input Transformations | ICML 2020 | code | self-supervised | ideas can be used to many tasks |
| 6 | Learning with Multiple Complementary Labels (paper) | ICML 2020 | |||
| 5 | Deep Divergence Learning (paper) | ICML 2020 | divergence | ||
| 4 | Confidence-Aware Learning for Deep Neural Networks (paper) | ICML 2020 | code | confidence | |
| 3 | Continual Learning in Human Activity Recognition:an Empirical Analysis of Regularization (paper) | ICML workshop | code | Continual learning bechmark | |
| 2 | Automated Phrase Mining from Massive Text Corpora (paper) | ||||
| 1 | Adversarially-Trained Deep Nets Transfer Better(paper | arvix 2020 | new findings | ||
| 0 | Learning to Combine: Knowledge Aggregation for Multi-Source Domain Adaptation | arvix (paper) | same ideas with us |