Studying information transfer in multi-task learning using random matrix theory

Theoretical analysis of multi-task learning using random matrix theory, together with the simulations and empirical experiments that accompany it.

Code structure

src/
├── notebooks/            # Random matrix theory simulations (theory vs. experiment)
│   ├── *.ipynb           # One notebook per setting studied
│   ├── TwoLayerNet.py    # Simple two-layer net used by the notebooks
│   └── figures/          # Generated plots (.eps) used in the paper
└── text_classification/  # Empirical multi-task text classification experiments

src/notebooks/ — RMT simulations

Each notebook sets up a data/transfer setting, derives the theoretical prediction, and compares it against simulation:

• covariate-shift.ipynb — transfer under covariate (input distribution) shift.
• model-shift.ipynb — transfer under model (parameter) shift.
• model-shift-with-adjustment-sps.ipynb — model shift with a sample-size adjustment.
• covariate-and-model-shift.ipynb — combined covariate and model shift.
• multitask.ipynb — the multi-task learning setting.
• comparing-estimators.ipynb — comparison of different estimators.
• TwoLayerNet.py — a small two-layer neural network (torch) shared by the notebooks.
• figures/ — .eps figures produced by the notebooks.

src/text_classification/ — empirical experiments

An Emmental-based pipeline for multi-task text classification, used to study information transfer across tasks on real data (mr, sst, subj, cr, mpqa, trec) with lstm, cnn, or mlp models.

• run.py — entry point: builds the model, tasks, and runs training/evaluation.
• task.py — defines the Emmental tasks (losses, outputs, task flow).
• modules.py — model building blocks (CNN, LSTM, embedding/average layers).
• data.py — assembles datasets and dataloaders from the raw loaders.
• dataloader.py — per-dataset loading, text cleaning, and word-embedding loading.
• utils.py — small file/JSON output helpers.
• download_data.sh — downloads the text classification datasets.
• run_text_classification.sh — convenience wrapper around run.py.
• requirements.txt — Python dependencies.
• README.rst — detailed run instructions for this experiment.

See src/text_classification/README.rst for end-to-end usage of the empirical experiments.