meta-learning

This repository based on the survey paper: Meta-Learning in Neural Networks: A Survey

Documentations

• Awesome Meta Learning: A curated list of Meta Learning papers, code, books, blogs, videos, datasets and other resources.
• https://github.com/dragen1860/awesome-meta-learning

Github

• Dataset and Benchmark: https://github.com/google-research/meta-dataset
• Data Loader and Benchmark: https://github.com/tristandeleu/pytorch-meta
• MAML pytorch: https://github.com/dragen1860/MAML-Pytorch
• MAML Tensorflow (original): https://github.com/cbfinn/maml

Survey

• Related Fields:
  • Transfer Learning
  • Domain Adaptation (DA) and Domain Generalization (DG)
  • Continual learning (CL)
  • Multi-Task Learning (MTL)
  • Hyperparameter Optimization (HO)
  • Hierarchical Bayesian Models (HBM)
  • AutoML
• Previous Taxonomy
  • Optimization
  • Black Box / Model-based
  • Metric-Learning
• New Taxonomy
  • Meta-representation
    • Parameter Initialization
    • Optimizer
    • Black-Box Models (Recurrent, Convolutional, HyperNetwork)
    • Embedding Functions (Metric Learning)
    • Losses and Auxiliary Tasks
    • Architectures
    • Attention Modules
    • Modules
    • Hyper-parameteres
    • Data Augmentation
    • Minibatch Selection, Sample Weights, and Curriculum Learning When
    • Datasets, Labels and Environments
    • Discussion: Transductive Representations and Methods
    • Discussion: Interpretable Symbolic Representations
  • Meta-Optimizer
    • Gradient
    • Reinforcement Learning
    • Evolution
  • Meta objective and Episode Design
    • Many vs Few-Shot Episode Design
    • Fast Adaptation vs Asymptotic Performance
    • Multi vs Single-Task
    • Online vs Offline
    • Other Episode Design Factor
• Applications
  • Computer Vision and Graphics
  • Meta Reinforcement Learning and Robitics
  • Environment Learning and Sim2Real
  • Neural Architecture Search (NAS)
  • Bayesian Meta-learning
  • Unsupervised Meta-Learning and Meta-Learning Unsupervised Learning
  • Active Learning
  • Continual, Online and Adaptive Learning
  • Domain Adaptation and Domain Generalization
  • Hyper-parameter Optimization
  • Novel and Biologically Plausible Learners
  • Language and Speech
  • Meta-learning for Social Good
  • Abstract and Compositional Reasoning
  • Systems
• Challenges
  • Meta-generalization
  • Multi-modality of task distribution
  • Task families
  • Computation Cost
  • Cross-modal transfer and heterogeneous tasks

How to summarize papers?

## What?

## Why?

## How?

## Results? (What did they find?)

## Ideas to improve?