Collective Model Intelligence

Toward Collective Model Intelligence: Characterizing and Coordinating Models

Tech Report · Results · Try Mio · Open-Source Dataset · 中文

Mio (Models in One) organizes complementary models to work together, choosing which model to call as task needs and remaining budgets change. Mio 1.0 Lite applies this approach to cost-efficient models from China.

Across nine equally weighted public benchmarks, Mio 1.0 Lite achieves an average score of 77.15/100 at 0.0621 CNY/request, the highest average score and lowest average cost among 13 compared methods. Compared with the strongest baseline by average score, Mio scores 6.35 points higher at 94.0% lower average cost. Full benchmark results →

Collective Model Intelligence aims to improve task outcomes and resource efficiency by understanding, reusing, and coordinating capabilities across an evolving population of independently developed models. The key is complementarity: a specialist can supply a missing capability, and different models can contribute at different stages as task needs and remaining cost and latency budgets change.

• Model Eval builds capability profiles through query-level score estimation and cross-domain prediction.
• Mio (Models in One) organizes complementary models and selects or coordinates them using task state and remaining budgets.
• Mos (Model Serving OS) schedules and executes evaluation and application calls, and manages deployment and model-file distribution for self-managed models.

Model Space provides shared model identities and resource references, linking capability evidence, candidate pools, and execution.

RobustRouteBench contains 35,072 instances across General Text (63 candidate models), Multimodal (45), and Chinese (10): 14,590 training, 1,500 validation, and 18,982 test requests. The test split directly matches the report's evaluation population.

Use Python 3.10+. The examples need no model API key.

git clone https://github.com/SinapisAI/collective-model-intelligence.git
cd collective-model-intelligence
python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[data,orbit]'
python examples/quickstart.py --scenario text  # train and evaluate on validation

Other scenarios: --scenario multimodal or --scenario chinese.

Replay the 65 prediction runs across 13 methods on the official test split using the report's nominal-budget nAUC:

python -m examples.reproduce_report --mode replay --output runs/report-replay

Dataset and protocol · Reproduction guide · Comparison methods

@techreport{sinapisai2026collectivemodelintelligence,
  title       = {Toward Collective Model Intelligence:
                 Characterizing and Coordinating Models},
  author      = {{SinapisAI Team}},
  institution = {SinapisAI},
  year        = {2026},
  url         = {https://github.com/SinapisAI/collective-model-intelligence}
}

Citation metadata: CITATION.cff. Code uses Apache-2.0, with MIT-licensed ORBIT components. The dataset follows its upstream terms; report materials and embedded logos retain their attribution.