Repository navigation
v0.3.1a0 (beta): cluster-locality default-on, adaptive expert paging
Pre-release
Pre-release
Beta (pre-release) of 0.3.1, core half. main stays at 0.3.0; this build is from dev. The rewritten Web UI follows in 0.3.1a1.
Install: pip install --pre project_chronos==0.3.1a0
Changed
- Cluster-locality training on by default for new runs:
train_chronos.py, the planner/presets andchronos bundle finetunesetlambda_cluster_locality = 0.3; the window is the power of two nearest the median SFT record length (clamped to [16, seq_len]). Turn it off with--lambda_cluster_locality 0/--no-cluster-locality. TheChronosConfigclass default stays 0, so stored recipes resume unchanged and older checkpoints load as λ = 0. - Inference adapts to the checkpoint (
paging=auto): weights trained with the constraint are paged one training cluster per SSD read, others one expert per read.cluster/expertforce a layout; each layout has its own cache directory.
Added
chronos/lazy_experts.py: one lazy-expert implementation shared by the Web UI, bundle andchronos serve.- Next-use residency planner for MLX lazy experts; decay and prefetch waits are configurable (UI, bundle CLI,
generate()). chronos bundle translate|eval --lazy-experts --paging --residency --decay --ram-ratio --decode-wait --verify-wait;chronos bundle finetune --[no-]cluster-locality --lambda-cluster --cluster-window --clusters.chronos serve --paging;/api/psreports the resolved layout.
Evidence and limits
- λ = 0.3 comes from a measured sweep on a tiny model with synthetic 4-topic streams (300 steps, from scratch): cluster reads per token 1.48 → 0.023, held-out CE 1.703 → 1.712. Its effect on real data and on fine-tuning previously unconstrained weights has not been measured.
- Residency planner on the 50M s13996 checkpoint (16 FLORES+ dev prompts, greedy, CPU): −35 % synchronous SSD loads (plain), −33 % (speculative); outputs identical. That checkpoint was trained without the constraint, so it pages per expert.
- Lazy paging is exact against full-memory greedy in the tests (MLX planned/window/speculative; Torch with
sync_on_demand). - Weights trained on data with unknown licences are research-only and are not part of this release.
Full list: CHANGELOG.md.