Repository navigation
Releases: janmus1c/ncm-mlx
Release list
NCM-MLX 0.2.0: released NCM checkpoints in MLX
NCM-MLX 0.2.0 runs the released Next-Concept Modeling checkpoints (HazyResearch; method by Kumbong et al., arXiv:2610.14162) natively in MLX on the Apple GPU. This is an independent, inference-only port, not affiliated with Stanford or HazyResearch.
Released checkpoints in MLX
pip install ncm-mlx
python -m ncm_mlx official fetch --task tinystories && python -m ncm_mlx official convert --task tinystories
python -m ncm_mlx official generate --task tinystories --n 2 --levels- Tasks:
tinystories,sudoku-easy,sudoku-medium,sudoku-hard. Downloads pinned to one checkpoint revision and SHA-256 checked. Conversion reads.ptfiles without PyTorch and refuses anything except tensor data. - Parity with upstream PyTorch: per-level logits agree to 3.2e-10 (TinyStories) and 1.2e-13 (Sudoku) in exact fp64; fp32 differences are at float rounding level. Greedy generation is identical (8/8 TinyStories samples, 64/64 Sudoku boards).
- Sudoku, greedy, all 2,000 published boards per difficulty: 99.40 / 96.30 / 86.20% easy/medium/hard (paper: 99.5 / 96.5 / 86.1%). Upstream PyTorch on the same machine also scores 99.40% on easy, with identical outputs.
- M5 Pro, p50 synchronized latency: TinyStories 128 tokens 39.8 ms (batch 1) on Metal vs 280 ms for upstream PyTorch on CPU; upstream cannot run on the Mac GPU (MPS lacks a required op). Desktop GPU load was present; see docs/EXPERIMENTS.md.
Limits
LM1B, OpenWebText, WMT14 and XSum are not converted. TinyStories generative perplexity is not computed. Sampled (non-greedy) outputs use MLX's random generator and do not match upstream sample for sample. From-scratch training and its recorded results are unchanged from 0.1.x.
Documentation: https://janmus1c.github.io/ncm-mlx/official-checkpoints/
NCM-MLX 0.1.1
NCM-MLX 0.1.1 is a packaging and metadata release of the independent MLX implementation of Next-Concept Modeling (method by Kumbong et al., arXiv:2610.14162). Model code, training behavior and recorded results are unchanged from 0.1.0.
Changes
- First release on PyPI:
pip install ncm-mlx(Apple Silicon) orpip install "ncm-mlx[cuda]"(Linux NVIDIA), published through PyPI trusted publishing. - Package metadata: project URLs, keywords, classifiers and absolute README links for PyPI.
- Author ORCID in citation metadata.
- Refreshed README first screen with an animated real refinement trace.
Related artifacts
- Trained weights (published separately, not in the package): ncm-mlx-sudoku, ncm-mlx-tinyworld
- Working paper v0.11: https://doi.org/10.5281/zenodo.23686842 (not peer reviewed)
- Documentation: https://janmus1c.github.io/ncm-mlx/
Limits
TinyWorld synthetic text and conditional Sudoku only. No demonstrated general-text quality, instruction following or upstream checkpoint compatibility.
Archived on Zenodo: https://doi.org/10.5281/zenodo.23780571
Working paper v0.11 — two-seed Sudoku assessment
Archived working paper v0.11 · Two-seed assessment
This working-paper revision adds the completed E20 seed replication and an assessment of what the current Sudoku study establishes.
| MLX implementation | Training seed | Easy | Medium | Hard |
|---|---|---|---|---|
| E19-funded | 0 | 98.10% | 83.40% | 40.30% |
| E20 | 1 | 97.25% | 79.60% | 33.25% |
Each run uses a 6.44M-parameter generator and 16.384M presentations; each reported difficulty contains 2,000 boards, scored greedily. The hard-set difference is 7.05 percentage points. Both schedules complete without a recorded reset or truncation. Two seeds do not establish population variability or identify its cause.
This is a partial implementation and small-model success, with a substantial hard-Sudoku deficit to Original NCM (Kumbong et al.). Generator-size comparisons exclude the separate codec; training exposure, representation and task setup differ. The original method and reference scores are attributed in the paper.
The two-seed assessment includes failure accounting, uncertainty, candidate explanations and discriminating next tests. It explicitly records missing per-board predictions, remote source/evaluation manifests and E20 benchmark contention. Retained-byte checks do not constitute independent rescoring or reproduction.
Controlled text generation is the next main research direction, with data, quality, baseline and local-cost gates. It is not a demonstrated capability. No new model training or accelerator evaluation is included in this update.
All six paper editions are attached. Portable validation: 47 tests pass; accelerator modules are excluded before collection. Repository/link checks, strict documentation build, numerical consistency checks, rendered PDF/Word review and secret/privacy scans pass. Earlier paper tags, DOI records and software v0.1.0 remain unchanged.
Publication status: unreviewed working paper; no peer-review or arXiv-acceptance claim. Paper and source · Research roadmap.
Working paper v0.10 — completed Sudoku evidence
The completed E19-funded run raises the MLX implementation's Sudoku solve rates to 98.10% easy, 83.40% medium and 40.30% hard, evaluated greedily on 2,000 published boards per difficulty. Its 6.44M generator is 3.95× smaller than Original NCM's 25.46M generator and comes within 1.40 percentage points on easy boards. Medium/hard remain 13.10 / 45.80 points behind; the codec is additional, and training recipes and budgets differ.
| MLX implementation run or cited reference | Generator presentations | Easy | Medium | Hard |
|---|---|---|---|---|
| E15, previous local best | 8,192,000 | 96.25% | 76.65% | 31.90% |
| E18 | 8,192,000 | 84.25% | 49.10% | 10.55% |
| E19 pilot | 3,058,560 | 90.20% | 59.75% | 16.55% |
| E19 funded | 16,384,000 | 98.10% | 83.40% | 40.30% |
| Original NCM (Kumbong et al., published reference) | 51,200,000 | 99.50% | 96.50% | 86.10% |
The funded run completed 128,000 updates, selecting step 127,000 by validation loss. The release records actual exposure, timings, artifact provenance, finite-board uncertainty, setup failures and negative results. E20 is a separate seed replication in progress, with no completed quality result.
Six paper editions are attached. Source, public figure data, the completed experiment report and roadmap planning scenarios are fixed by this new tag. Planning scenarios do not predict future solve rates. Paper v0.9 and software v0.1.0 remain unchanged.
Validation: 43 portable tests passed (3 MLX modules skipped), Ruff, repository/link validation, strict documentation build, paper layout checks, PDF/Word visual review, document archive/XML checks and secret/privacy scans. The Word build now preserves readable numeric columns and native equation annotations.
This is an unreviewed working paper, not an arXiv acceptance. The NCM method and Original NCM reference results are due to Kumbong et al.. The independent MLX implementation does not claim overall parity or an equal-budget efficiency advantage.
Archived paper DOI: 10.5281/zenodo.23640777.
v0.1.0 — Initial research release
NCM-MLX is an independent, experimental MLX implementation of Next-Concept Modeling for TinyWorld synthetic text and conditional Sudoku. The underlying method is due to Kumbong et al.; this release does not claim to reproduce the paper's NCM results or load upstream checkpoints.
Included
- Two-stage residual-VQ codec and coarse-to-fine generator training, sampling and evaluation.
- Exactly one generator forward per level, with exact CPU training/inference equivalence tests.
- Character-level AR baseline and synchronized benchmarks with explicit sampling metadata.
- Full-state pause/resume recovery, including optimizer, random and batch state.
- Verified Metal and NVIDIA CUDA workflows, pinned dependencies and checksummed external Sudoku downloads.
- Architecture, training, benchmark, reproduction and recovery documentation; contributor and security workflows.
- Working paper v0.8 in PDF, HTML, EPUB, Word, LaTeX and JATS XML.
Evidence and limits
E15 remains the best verified local Sudoku run: 96.25 / 76.65 / 31.90% solved on 2,000 easy/medium/hard boards per set. E17's 71.25 / 31.75 / 3.70% comes from a weights-only continuation with reset optimizer/random state. E18 is ongoing and supplies no completed result to this release. Generator-size comparisons exclude the separately trained codec; many comparisons have unequal training budgets or contended hardware.
There is no demonstrated general-text quality, instruction following, production chat service or upstream checkpoint compatibility. Both inference samplers remain uncompiled. No datasets or model weights are bundled.
Start here
Validation: 58 MLX/portable tests passed locally; an isolated no-MLX environment passed 23 portable tests with 3 MLX modules skipped. Citation schema, 20 configs, documentation links, wheel installation, paper layout and 21-page desktop/phone browser checks passed. Staged-tree and complete clean release-history secret/privacy scans passed. CI separately checks Python 3.11 and 3.12 without accelerator training.
Working paper v0.9 — reproducibility and citation update
Paper v0.9 improves the reproducibility and submission readiness of this independent MLX study of Next-Concept Modeling. It corrects the public-data figure command and E1 exposure, distinguishes E12's measured run from its current configuration, qualifies the memory-bandwidth discussion, expands E10–E17 settings, and discloses OpenAI Codex assistance.
Read and cite the archived preprint. The software v0.1.0 archive remains fixed at commit f2516a90e3d3ffac493ded861bfb7350cf611031; its original GitHub release and v0.8 paper assets are preserved.
Assets include six canonical editions and a separately compiled XeLaTeX source package/preview prepared for arXiv. The source package contains the main TeX file, five figures and the MIT notice. The canonical Typst PDF is 16 pages; the XeLaTeX preview is 15 pages because its typesetting differs. This is an unreviewed working paper. Next-Concept Modeling itself is credited to Kumbong et al..
Validation: citation schema and repository links; strict documentation build; privacy scan of staged files and generated document internals; rendered canonical PDF/Word and clean TeX Live 2025 compilation of the isolated source package. No new research measurements or changes to the reported outcomes are introduced.