MASS

Recursive Self-Improvement through Multi-Agent Self-Supervision

📜 Paper  ·  🧠 Overview  ·  📊 Results  ·  🚀 Quick start  ·  🔁 Pipeline  ·  🧪 Tasks & evaluation  ·  📚 Documentation

MASS alternates between improving workflows and learning from them. RHI searches for multi-agent workflows while the model weights stay fixed. Post-training uses trajectories from the selected workflows to update those weights. The next cycle starts from the updated model.

L^(0) ── RHI → post-training ──> L^(1) ── RHI → post-training ──> L^(2)

We use the paper's notation $\mathcal{L}^{(k)}$ for the model after $k$ cycles. The configuration files and output directories use L0, L1, and L2 for the same generations. Each cycle uses the current model as executor, optimizer, and in-loop evaluator. The base model is Qwen3.6-27B, and each generation uses qwen-code 0.20.0 as its coding-agent runtime.

This is a research implementation for running and extending the MASS pipeline. The experiment support and implementation notes describe the available experiments and differences from the reported runs.

Results for the base model $\mathcal{L}^{(0)}$ and the models after one and two MASS cycles. Task and trial counts are per model. ScienceAgentBench and delivery rates are shown as percentages; other scores retain their reported scales.

Terminal-Bench 2.0 (official time limits) and SWE-bench Verified report mean pass rate ± sample standard deviation across three trials. Their task counts are the numbers attempted per trial; scores use tasks with results for all three models within each trial: 86/88/87 for Terminal-Bench and 499/499/496 for SWE-bench. Delivery rates are reported separately from the MLR-Bench and AstaBench scores. The AstaBench breakdown includes trial scores, report counts, and scores among delivered reports. See the benchmark resources for official papers and code or data.

The packaged MLR-Bench configuration covers an earlier 12-task subset. The 107-task configuration and evaluation adapters for AstaBench (E2E-Bench-Hard), DSBench, Terminal-Bench 2.0, and SWE-bench Verified are not included. See experiment support for the available runners.

• Workflow search: RHI, trajectory collection, and within-task ranking.
• Post-training: conversation rendering, assistant-only loss masks, LoRA, validation-loss checkpoint selection, and BF16/FP8 model export.
• 12 synthetic tasks: every task prompt and initial workflow, with the nine-task training split and three-task test split.
• Evaluation: synthetic-task pairwise judging and ScienceAgentBench/MLR-Bench adapters.
• Two MASS cycles: separate configurations for the first and second updates.

The code runs independently of the original research repository. Model weights, task datasets, trajectories, and checkpoints are downloaded or generated when needed.

From your cloned mass directory, preview the first cycle and run the offline tests. These commands need no GPU or model server:

python3 -m mass plan --config configs/paper.json
python3 -m unittest discover -s tests -v

plan prints the configuration and training command without executing them. The tests use small fixtures and make no model calls. The RHI integration test requires the core dependencies and is skipped when they are unavailable.

For an actual experiment, follow the setup guide. It covers Python environments, model downloads, vLLM serving, and GPU allocation. Once the model server is running, you can start with a single task:

python -m mass search --config configs/paper.json --tasks 51

This uses the configured search budget. The single-task example shows how to use a shorter search in a separate run directory.

The stages are separate commands so that you can inspect intermediate outputs and resume completed work.

Run a stage with python -m mass <stage> --config <config>.

The second cycle trains a fresh LoRA adapter on the merged first-cycle model. The run guide gives the complete command sequence.

For each eligible training task, ranks 1–15 enter training and rank 16 enters validation. The saved checkpoint with the lowest validation loss is selected. External judges are used for reporting, separately from this training loop.

The synthetic tasks cover finance, pharmacy, and robotics:

Task 221 is included in the repository but was excluded from teacher collection in the reported experiment; the remaining eight training tasks contributed SFT data. Test tasks do not enter SFT or checkpoint selection. See the task index for every prompt and initial workflow, or tasks/splits.json for the machine-readable split.

Use rollout and report for synthetic-task evaluation. The public benchmark guide describes ScienceAgentBench and MLR-Bench evaluation with $\mathcal{L}^{(k)}$ + qwen-code.

This repository supports the pipeline and benchmarks listed above; it does not include every paper ablation or analysis. The current RHI driver uses fixed-reference comparisons and champion arbitration, which differs from Algorithm 1's direct comparison with the retained best output. The experiment support and implementation notes describe these differences and the current evaluation limitations.

See CONTRIBUTING.md for changes, bug reports, and experiment extensions. NOTICE.md covers third-party software and data.