Home

contilab wiki

contilab is a PyTorch framework for empirical continual learning research (lifelong / incremental learning). It benchmarks methods that fight catastrophic forgetting on vision and language models, compares sequential training with model merging and knowledge distillation, and reports 30+ evaluation metrics with proper statistics.

Start here

Page	What it covers
Getting Started	Install, first run, reading the report
Architecture	Modules, data flow, the ensemble axis, raw-log-first metrics
Methods	Scenarios and the CL strategies: Naive, Joint, EWC, LwF, ER, DER++, A-GEM, LoRA-per-task
Consolidation	Train-then-consolidate: Task Arithmetic, TIES-Merging, multi-teacher and teacher → student continual distillation
Metrics	Every metric with its formula and reference, statistics, figures
Benchmarks	Split/Permuted MNIST, Split CIFAR, text5, synthetic streams
Performance	vmap ensembles, torch.compile, bf16, measured throughput
Results	CPU study: 12 methods × 6 benchmarks (3–6 seeds), tuned comparison, rankings, findings
Configuration	Profiles, presets, experiment matrices, sweeps, overrides
Running Experiments	CLI, run directories, resume, reports, post-hoc metrics
Extending	Adding a strategy, metric, benchmark or consolidator
Reproducibility	Seeds, determinism, caching and what is recorded
FAQ	Common questions
Citing	BibTeX and CITATION.cff

Why contilab

• Fair comparisons. Every method runs through the same training loop, optimiser, epochs, batch size, memory budget and evaluation protocol.
• Fast seeds. Seeds that differ only by their random seed train together as one stacked-parameter ensemble, so three seeds of a small model cost little more than one.
• Metrics you can add later. Training stores raw data (accuracy matrices, continual-evaluation curves, predictions, features, activations, gradients); metrics are pure functions of those records.
• Vision and LLMs behind one interface. The same strategies run on an MLP, a reduced ResNet-18 or a Hugging Face causal LM with a classification head.