Machine Dialectology (MDia)

_{Machine Dialectology: from verbose reasoning traces to a measurable ecology of machine dialects. Click the figure to inspect the full-resolution manuscript asset.}

Large language models increasingly operate as heterogeneous systems of solvers, routers, critics, and tool users. Yet their intermediate reasoning is still usually expressed as human-facing prose. Natural language is readable and flexible, but it can spend generated tokens on discourse, pedagogy, and surface conventions that a machine listener may not need. Simply forcing shorter answers is not enough: aggressive compression can remove variables, evidence links, parse commitments, verification state, or output contracts that the listener does need.

Machine Dialectology (MDia) treats an intermediate trace as a message produced by one machine component and interpreted by another. Speaker models create persistent Language Symbolism Frameworks (LSFs)—compact dialect cards with symbols, grammar, reusable operators, validity rules, answer contracts, provenance, and empirical transfer profiles. Listener models execute those dialects, and MDia routes them under explicit accuracy, token, parseability, and reliability constraints.

Central thesis

A machine dialect is not intrinsically strong or weak. Its value is relational:

dialect utility = f(speaker, listener, task, route, budget).

This perspective separates brevity from communicative adequacy. The objective is not to minimize tokens in isolation; it is to preserve the task-relevant state that a specific listener can reliably interpret under a budget.

The October 2026 release provides measured MDia receiver profiles across five benchmarks and three LLM families: GLM, Qwen and DeepSeek. Reusable dialects encode task-relevant obligations, function-call procedures, evidence links, compact code state and Python execution state.

Savings refer to native generated tokens, including native reasoning once. Native input usage is reported separately in the complete tables, and TALE accounting includes both estimator and solver stages. Correct counts describe the recorded cohort. The BFCL row uses the source Qwen parent profile and a new common-cohort English comparison.

In the Qwen BFCL common-192 panel, the MDia parent improves the observed accuracy/generated-token point over 11 of 13 baseline points, spanning six baseline families. All seven registered baseline families remain in the full panel. The English comparison's generated-token saving has an ordinary whole-group bootstrap interval of 60.63–69.41%.

Clone this repository, use Python 3.11+ (tested with Python 3.12.14), and run:

python3 -m venv .venv-evidence
source .venv-evidence/bin/activate
python -m pip install -r requirements-evidence.lock.txt
python scripts/verify_mdia_release.py
python scripts/reproduce_mdia_release.py --output runs/mdia-results-20261005

Choose a fresh output directory. The command verifies the file manifest, recomputes aggregates, whole-group paired intervals and accuracy/generated/input frontiers, checks the saved numerical references, and exports the tables and PNG/SVG figure. It runs offline without API credentials. VALIDATION.json records the checks and runtime versions.

The package contains 16,576 case-policy observation rows, the complete 55-point Flash 256-case block, all 36 BFCL common-192 points / 34 contrasts, all 36 CRUXEval common-192 points / 52 contrasts, and 303 baseline summary rows. Original cohort and logical alias identities remain available throughout the numerical records.

• Results overview and instructions
• Full BFCL common-192 table
• Full Flash 256-case table
• Full CRUXEval common-192 table
• Paired settings and separate input accounting
• Reproduction protocol, seeds and source mapping
• Download the versioned package

These commands reproduce the numerical analyses and presentation from the released scalar records. For new model inference, acquire the benchmark inputs from their original sources and use the dataset/provider interfaces in Extending MDia.

The deterministic toy pipeline exercises the complete lifecycle with redistributable fixtures. It requires Python 3.10+, no API key, and no model download.

python -m venv .venv
source .venv/bin/activate
python -m pip install -e .

mdia pipeline --config configs/toy_mdia.yaml

The command creates a versioned run directory containing immutable split manifests, direct traces, dialect generations, transfer profiles, a frozen dialect bank, route plans, predictions, token accounting, rule results, and a reproducibility report. Re-running with the same configuration and seed selects the same cards and routes.

mdia --help
mdia pipeline --help

runs/<run_id>/
  manifest.json
  report.md
  splits/{induction,evolution_validation,router_validation,test}.json
  direct_traces.jsonl
  dialects/generation-*.json
  profiles/{evolution_validation,router_validation}.jsonl
  selection/frozen_dialect_bank.json
  execution/{route_plans,predictions,evaluations}.jsonl
  execution/token_accounting.json
  rules/{registry,results}.json

Use the canonical MDia framework to create and evolve dialect cards, profile their transfer across listeners, and compare budgeted routes on your own tasks. Start with the offline lifecycle below, then add datasets and providers through the extension interfaces. The released native policy identities and serving-specification examples illustrate reusable task contracts and receiver profiles.

CLSR is MDia's homogeneous-agent special case, available through configs/clsr.yaml. Both use the same supported implementation under src/mdia/. MDia is the general entry point for experimenting with heterogeneous speakers, listeners, dialects and deployment policies.

A dialect card implements the manuscript contract

Cards also carry stable IDs, generation and parent links, creator/speaker provenance, task tags, I/O contracts, fallbacks, and a digest of the complete specification. This makes a dialect a persistent research artifact—not a one-off shortened answer or an unversioned prompt string.

The public CLI separates eight stages that older research scripts mixed together.

The implementation preserves four immutable partitions:

The distinction is deliberate. A controller choice is not reported as a concrete-LSF route. A canonical run may record both, but the two decisions remain separate in artifacts and evaluation.

Before generation, routing may use the query, observable task metadata, listener profiles, frozen validation statistics, and the token budget. It may not use gold labels, held-out outcomes, or feedback from a candidate answer.

The July 30 MDia manuscript source reports the following aggregate picture across eight benchmark columns: MMLU-Pro, GPQA-main, MATH500, AIME 2024–25, BFCL-v3, LiveCodeBench-output, MultiHopRAG, and MuSR.

The rule bank is deliberately conservative. Supported labels are evidence statements for the audited archive, not claims of universal laws or human-equivalent language emergence. Weak, boundary, and unsupported analogies are retained to constrain overclaiming.

MDia is organized as an auditable research system rather than a directory of loosely coupled scripts.

Only redistributable fixtures and compact example artifacts should be committed. Restricted benchmark text, private prompts, credentials, provider ledgers, and remote paths do not belong in Git. See Reproduction and data policy before interpreting or publishing results.

src/mdia/          canonical schemas, lifecycle, routing, evaluation, and rules
configs/           offline, CLSR, and paper-oriented presets
examples/toy/      redistributable deterministic fixture
artifacts/example/ committed complete offline example output
artifacts/research/ benchmark scalar records, reference tables and figures
releases/          versioned benchmark reproduction package
docs/              architecture, pipeline, reproduction, extension, and rules
papers/            manuscript snapshot included with the repository
legacy/            archived v0/v1/v2 research workspaces; noncanonical
tests/             unit, leakage, integration, and offline end-to-end checks

MDia provides an experimental substrate for questions that ordinary prompt optimization does not expose:

• Which dialects become public teaching languages, and which remain private self-talk?
• Which listener properties predict openness, resistance, or compression failure?
• When should a router select one dialect, aggregate several, compose stages, or fall back to raw reasoning?
• Can dialect families be inherited and recombined without losing provenance or listener compatibility?
• How do machine dialect ecologies change across model generations, domains, modalities, and tool-use settings?
• Can learned dialects improve production inference while preserving auditable, human-readable final answers?

Principia is a principle-first idea-discovery system and a downstream application direction for MDia. Machine dialects can serve as reusable reasoning substrates for improving the reasoning–token frontier and for organizing compact symbolic operators, evidence states, and logic chains during research ideation. MDia remains the general communication and routing framework; Principia applies that capability to structured idea discovery.

Issues are welcome for reproducibility reports, provider or dataset adapters, routing failures, rule-validation questions, and paper-to-code discrepancies. A useful technical report should include the MDia version, configuration, run ID, model/provider revision, split hashes, evaluator version, selected dialect IDs/digests, and the smallest redistributable artifact needed to reproduce the observation.

Please do not post credentials, restricted benchmark content, private prompts, or unredacted provider logs.

Use CITATION.cff for software citation. For the homogeneous CLSR method, cite:

@inproceedings{pei2026clsr,
  title     = {When LLMs Develop Languages: Symbolic Communication for Efficient Multi-Agent Reasoning},
  author    = {Pei, Zhengqi and Huang, Qingming and Wang, Shuhui},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  year      = {2026}
}

• CLSR paper: arXiv:2606.29354
• CLSR conference page: ICML 2026
• MDia manuscript snapshot: papers/MDia_Jul17.pdf

Use .env.example only as a list of environment-variable names. Export credentials through your shell or secret manager; never place a secret in YAML, JSON, Python, command transcripts, experiment artifacts, or issue reports. Generated model text—especially under legacy/—must be treated as untrusted data and never executed without validation.

Code is released under the MIT License. Dataset and model licenses remain with their respective owners.

Academic collaboration
In collaboration with the Institute of Computing Technology, Chinese Academy of Sciences.
Contact: peizhengqi22@mails.ucas.ac.cn

Business collaboration
In collaboration with Beijing Chipflow Technology Co., Ltd.
Contact: peizhengqi@chipflow.net