Speculative Per Package Repos

Per-package repositories

Speculative design. Open, not settled — thinking and proposals, not shipped behavior. See Roadmap for what's in flight.

Status: speculative (2026-10-02). Split most of the monorepo into one repository per library, and keep a small core and a toolchain that every library builds against. The library machinery already assumes it: a library is a published package, the registry is the integration point, and library-template is one, living in its own repo (Speculative-Vdom-Markup §4.2 and §4.4). This page says which packages become repos, what travels with each, how they compose, and the order to move them in.

1. Why, and what it costs

Today every library is a directory in one pnpm workspace (pnpm-workspace.yaml). They share one version (SYSTEM_VERSION, the root package.json's), publish nothing (every package but utils is private, at 0.0.0), and gate on one pipeline (CI-and-Deployment).

What it buys:

• Independent release. A fix to one library ships on its own tag, with a computed version bump (pack-library --since), without waiting on the site or anything else.
• Smaller CI. A library PR runs that library's check, tests, pack and the oracle lanes it declares. The gate stops paying for the site build, the dominant cost (Speculative-Lazy-Engine §1), on every library change, and the test shards in ci.yml stop being a fix for one runner holding everyone's suites.
• Third-party libraries as first-class. Same layout, same install check, same census. Ours get no shortcuts, which is the dogfooding Speculative-Lazy-Engine §3 asks for.
• The registry and manifest as the only integration point. The monorepo shortcuts go (srcAliases, /src exports, packages/* globs, central tables), the ones that let packages see each other without declaring it.
• Lanes with real boundaries. A repo is the strongest file boundary a lane can have, and the LANDING mutex becomes per repo (Lanes).

What it costs:

• No atomic cross-package change. Renames, record-schema changes, correctness batches and compute-engine bumps become one PR per repo, in dependency order, with publish latency between. That needs a codemod runner and a bump bot before the first cut, not after.
• Every compute-engine minor breaks every library (a caret range on 0.x). Today that is one PR.
• Late regressions. A cross-library break shows in the nightly integration run, not on the PR that caused it, unless core PRs opt into a downstream check (§4).
• Duplicated plumbing. CI config, branch protection, secrets (the Wolfram entitlement), issue trackers and worktrees all multiply.
• Names freeze. The first publish makes today's package names permanent on npm, and renames wait (Component-Naming). Settle them first.
• Seams to repair first (§2.3). Several libraries still depend on each other in ways a repo boundary can't carry.

2. What becomes a repo

2.1 The rule

A boundary goes where the edge is a published-API edge: the lower side changes less often than the upper, and only the registry's contents cross it (heads, types, records). The hierarchy (hierarchy.ts) gives the layers, and change frequency picks the cut. engine, structures and boxes change on protocol decisions, and everything extends them. The libraries change when their mathematics does. reference is the most-touched package by far, because it holds the data for everything; once records live in their libraries it stops being that.

2.2 The split

Repo	Holds	Why
nucleus (enumeratio/enumeratio, keeps the wiki)	engine with ce-patches folded in, manifest, entry, boxes, structures, evaluation; the toolchain: formats, raster, wolfram, oracle, reference (harness, schema, crosswalk machinery, compute-engine's own heads' records), plausible, utils, bench, tools/perf	The infra and base layers plus tooling that imports no library. Everything a library imports lives here, and nothing here imports a library. One version: the system. evaluation and ce-patches stay because every host runs the first, and the second tracks compute-engine and is applied by libraries from the published package.
one repo per library	residues, numerals, number-theory, adeles, analytic, hypercomplex, geometric, diagram, groupalgebra, hecke, hopf, incidence, quiver, modular, braid	The extension layer: each extends only what hierarchy.ts lists. Each ships its records, code, oracle values, notation and docs.
combinatorics	combinatorics, statistics, polytope	One repo until Speculative-Combinatorics-Layering-and-Plausible lands: statistics fills combinatorics' table and takes its carriers from the host, and its generators write into combinatorics' records. The Plausible runner and workflow follow collections/.
interface (enumeratio/notatio)	frontend, components, cli, and the site (web/)	The presentation layer: imports any library, is never one's parent, and has the highest churn. The site is the distribution's first consumer. Package names don't change.
census (enumeratio/census)	the cross-library checks (§4)	The crawler the roadmap describes: installs the libraries and checks them together. Depends on everything, so it can't sit in core.

catalog folds into manifest/libraries as Speculative-Vdom-Markup already says; until then it stays in core, and its tests that need combinatorics move into that repo.

Where the cross-cutting docs go: the wiki stays on the core repo, which is the project's home. An org-level .github repo carries the community defaults (contributing pointer, issue templates). A library repo carries a short AGENTS.md with the rules that must hold everywhere (no conflict markers, no snapshots, no attribution, golden data), and links to Contributing for the rest. The Packages page gains a repo column, and the wiki's blob/main/packages/... links are rewritten once from a path map.

2.3 Seams to repair before a cut

• Declare order. adeles' Fibonacci widening clobbers number-theory's, and modular's Inverse clobbers combinatorics'. Two repos can't coordinate "declare me last". Speculative-Overloads-as-Data comes first for those pairs.
• statistics ↔ combinatorics: the table fill and the generator that writes into the other package's records.
• Two libraries claiming one name. groupalgebra and combinatorics both reach for SymmetricGroup; FromDigits is documented in two packages; records carry overrides naming other libraries. The loader's collision check (loadReferenceData) becomes a census check.
• /src shortcut exports, imported by census and engines.ts. Libraries need a real declare entry.
• Dev edges. The algebra libraries devDepend on hypercomplex without importing it. Any core-to-library devDependency (reference, census, catalog, frontend, components) has to go.
• Tables built from one tree: manifest/scripts/build.ts and the committed declarers-data.ts.
• Interface imports of libraries. frontend imports analytic's shaders and polytope. Libraries should ship their own plot components (the reserved <group>-components in Packages §1), or the toolchain and the interface form a repo cycle.

3. What travels with a library

The library-template layout, as one tree:

• Its records, reference/<Head>/: index.md (definition, signatures, references), examples.tsv, examples.values.<system>.tsv (oracle answers) and notation.json (the unmerged notation-data-analytic branch). Heads and their examples change in one PR, which is what the install check relies on. Records stay with the library; a data repo would bring the coupling back (Examples-as-Data §7 reached the same answer per package).
• Its code: kernels, declare, the ./notation entry. What's generated (compiled families, compiled statistics and maps, the manifest slice, forms) is built and shipped in dist/, never committed (#533).
• Its oracle data: the values files, its package goldens (adeles.golden.json), and the targets it names in enumeratio.mappings. CI runs only those lanes.
• Its docs: README.md and docs/, in the tarball's files; the site serves them under /docs/<slug>/ from the installed package.
• Its tests, including the pinned form columns in examples.tsv.
• Its package.json: the enumeratio field (namespace, index, system range, mappings, notation) plus requires in place of its hierarchy.ts row, and peerDependencies on compute-engine and the libraries it extends, so an engine holds one copy of each.

What it needs from core is a published toolchain, a CLI (working name @enumeratio/toolchain) over what now sits in scripts:

• pack and --since (pack-library.ts), which are the pack-library action today;
• check: the install check, the hierarchy guard against requires, notation collisions within the library;
• scan: oracle-scan.ts and the farm, adopt and prune steps, for the declared targets;
• forms: collect-forms.ts and pin-forms.ts, which import only boxes, formats, entry, oracle and reference, so they move to core;
• format-records, and the Plausible runner.

recordDirs and loadReferenceData take a root, and the engine builder exists (manifest/src/engine.ts, #607): buildEngine({ libraries }) declares the base, the libraries asked for and what they require, in an order planned from the hierarchy and each library's enumeratio.declare / enumeratio.late fields. The remaining hard-coded order (DECLARE_ORDER, DECLARE_PREFERENCE) goes when declarations stop depending on order. Core also publishes the oracle kernels as images (Sage, Julia, Oscar and Mathlib are multi-gigabyte), so N libraries don't each install them.

CI template: a reusable workflow in core, so a library's own file is a few lines, pinned to a system ref. It runs install, build, vp check, tests, pack with the stale check, --since the previous tag, then the install check. Nightly: the declared oracle lanes and DEEP_TESTS. On a v* tag: publish to npm with provenance. check.yml is the seed. The template itself is pure Epsil and ours are TypeScript kernels, so it needs a code-library variant (src/, declare, vp pack).

4. Integration

• Registry. npm is the transport and our scope the namespace, read through PackageHost like the GitHub host the template uses (catalog.ts, Namespaces). Third parties publish the same way and are found by the enumeratio-library keyword. Our TypeScript libraries are the code packages of §4.2: installed as dependencies by a host we control, not fetched at run time.

• The manifest. The contract holds: a packed index per library, definitions pinned by hash, examples as the install check, a lock. The aggregate SYMBOLS and PACKAGES table stops being built from one tree. Each library's packed index is the source, and a distribution assembles them (Manifest, "Delivery"). Core's manifest keeps the machinery and the engine's own heads. This is the largest piece of engineering in the plan.

• Only the nucleus knows compute-engine (direction, 2026-10-02; still being explored). The survey and proposed export list are in Speculative-Nucleus-Facade. Core is called the nucleus, and ce-patches lives inside it rather than as its own package. The nucleus pins one exact compute-engine and exposes a controlled subset, not the whole API, plus the hooks a library needs to plug in: formatters, code serialization and parsing, notation, and possibly components, ideally encapsulated rather than exposed raw. The full compute-engine API stays reachable through an undocumented, explicitly unstable entry, so a library can opt in temporarily to test something or work around a bug. The aim is libraries that hold very little code beyond Epsil expressions, records and notation data, so they need little API surface at all. Libraries peer-depend on the nucleus within a range and never import @cortex-js/compute-engine (a lint rule enforces it). The nucleus's version follows its facade, not compute-engine: an upgrade libraries can't see is a patch and flows through their ranges. Libraries move at their own pace; one not verified on a newer nucleus can't join a distribution built on it. Libraries still depend on Epsil's behaviour, but Epsil rarely breaks compatibility, and where it does that's acceptable for now.

• Versions and ranges. Independent semver per library, computed from the diff. enumeratio.system is a range against core's version, which lockLibraries already checks. The pins keep a definition exact, so a range only governs what a caret pulls in.

• A distribution is a lockfile. The site and the CLI list exact library versions, as theme/libraries.ts does for the template. Census writes the known-good set when a run over the latest versions is green, and a bot opens the bump PR. Libraries release independently; what ships together is a pinned set.

• Cross-library checks run in the census repo, nightly and on any library's release. It installs the latest of every library and runs:

  • fullEngine: every library declared in hierarchy order, collisions and contributions;
  • notation collisions across libraries;
  • the resolver test over every example, and the hierarchy guard read from each library's declared requires;
  • the crosswalk: each record's references plus the external snapshots (DLMF, Fungrim, Wikidata, OEIS, FindStat) and the link check;
  • the catalog map laws and Plausible across carriers (combinatorics × statistics, the algebra carriers), and the oracle Plausible sampling;
  • the Wolfram frontier and head-map audits.

  A failure files its issue on the library that owns the head; the nightly-fixup routine reads census. A core PR can opt into a downstream run of census against its packed tarballs.

• Site and CLI build against installed packages: docs from tarballs, reference pages from loadReferenceData over the installed records, no srcAliases. That is item 6 of Speculative-Lazy-Engine §4 as well, so it pays regardless.

4.1 The manifest per library (BL-79; approved by Dean 2026-10-04)

Today four tables are built from the one tree, and every consumer reads the aggregate:

table	what it holds	built by
SYMBOLS	per head: the packages documenting it, each package's overloads (type, overrides, on/symbols/types), params, attributes, findstat	manifest/scripts/build.ts, walking every package's records plus a bare compute-engine
package/<pkg>	per package: each head's summary, type and attributes, and example counts	the same build
NOTATIONS	each package's notation entry	the same build, from each package.json
DECLARERS	which libraries declare or redeclare each name	reference/scripts/collect-declarers.ts, by declaring each library into a fresh engine; committed

plus HIERARCHY, hand-written. The resolver, describe, the frontend's HEADS/PARAMS and the docs read the aggregate.

Proposal: the library's packed index is its manifest slice, and a distribution assembles the aggregate.

1. One index shape for every library. reference/index.json (LibraryIndex, §4.2 of Speculative-Vdom-Markup) already carries what an Epsil library's symbols are: signature, pin, params, defaults, attributes, summary, notation, mappings, example counts. A code library packs the same file, with the fields that only it has:

   • per symbol: overloads (each with type, overrides, on/symbols/types, as SymbolInfo has them), findstat, and whether it declares the head or extends one declared below it (#605's rule: one declaration, contributions by extend);
   • per library: requires (its hierarchy.ts row, now in package.json), and declares, the names its declare(ce) declares or widens: the DECLARERS row, computed at pack time by declaring the library into a fresh engine over what it requires, as collect-declarers.ts does today.

   An Epsil library's symbols have pins; a code library's have none and are versioned by the package.

2. Global and namespaced names. Our libraries declare global heads (Binomial, not combinatorics.Binomial); a third party's live in its namespace. The index says which: no namespace (or global: true, reserved to our scope) means its names are global and the resolver finds them by name, as manifestRegistry does now; a namespace means ns.Name, as the catalog does.

3. The distribution assembles. A distribution is a lockfile (§4): the site, the CLI and census each list exact versions. assembleManifest(indexes) merges the installed indexes into today's SYMBOLS, DECLARERS, NOTATIONS and PACKAGES: overloads concatenate with their library, documented is the libraries whose index carries the head's record, requires builds the hierarchy. Consumers don't change: the aggregate has the same shape, only its source moves. The site builds it once at build time from node_modules; a host that wants less can load one index per namespace.

4. Core keeps compute-engine's heads. The bare-engine part of build.ts and reference/entries become the compute-engine library's index, shipped with core and assembled like any other.

5. Collisions become a census check. Two indexes that both declare a head, or a duplicate signature, fail the assembly in census (#605's rule), instead of the build's collision check over one tree.

Steps, each verifiable against today's output:

1. Built. build.ts assembles from a per-package index (scripts/package-index.ts reads a package's records into a PackageIndex; assembleManifest in src/assemble.ts, exported for hosts, merges them with compute-engine's own heads). The generated tables come out byte-identical.
2. Built, with step 4 (#621). Each library's build runs manifest's collect-declares after packing and writes dist/declares.json (enumeratio.declares): the names it declares or widens, the heads those canonicalise to, and the carrier types it mints. loadLibraries reads it, the site's catalogue imports it, and createResolver and the registry assemble their tables from the libraries they're given; compute-engine's own canonical forms come from the manifest build. collect-declarers.ts and the committed declarers-data.ts are gone, and the assembled tables match them exactly (plus formats, frontend and statistics, which reference never loaded).
3. Built (#628). Each package's enumeratio.layer, area and extends (and entries, for an entry point placed apart) are in its package.json; the manifest build gathers them into HIERARCHY, and the guard test is unchanged.
4. Built (#621). Hosts assemble from the libraries they install; the tarball check passes on it.

Open: whether summary and example counts stay in the aggregate (171 KB of summaries are loaded per package today) or are fetched per library on demand; whether a code library's index should also carry its records' references (the crosswalk) or census reads those from the installed records; and the size of SYMBOLS once third-party indexes are assembled into it.

5. Migration

0. Split without splitting. Repair the §2.3 seams in place. Add one CI job that packs every package, installs the tarballs in a scratch directory, and runs census and the site build from them. When it is green, a split holds nothing unknown except process. Settle the npm scope and the package names.
1. Pilot (done). library-template is public, tagged, and read by the site (#582). Harden it into the CI template: the toolchain by npx instead of an action that checks out core, the reusable workflow, tag-to-publish, a nightly lane, and the code-library variant. Exit: a third party can fork it and pass.
2. Publish what a leaf imports, then cut one. Publish engine, boxes, structures, manifest, entry and the toolchain's pack and check, versioned by the system. Then extract hopf. It imports only engine, boxes and structures. Its dependents are the CLI, census, reference and the site, all of which a published package can serve. It has no declare-order edge, and its records, notation and docs are already in its tree. The others fail on their own: groupalgebra shares SymmetricGroup with combinatorics; adeles sits on three libraries and widens Fibonacci; statistics is welded to combinatorics; polytope has no records. Its siblings incidence, quiver and hecke follow in one batch by script. This reorders the obvious sequence on purpose: a leaf can't leave before the base it imports is published.
3. Finish the toolchain. scan, forms, format-records, the oracle images, the census repo, the distribution lockfile, and the bump bot. The site now reads extracted libraries from the registry beside the in-tree ones; pnpm mixes workspace and registry packages, so nothing waits.
4. The rest, in waves.
   • diagram then braid, hypercomplex then geometric, modular and groupalgebra once their clobbers are data. diagram and groupalgebra carry oracle values, so this wave exercises the scan.
   • analytic, the arithmetic chain top-down (adeles, number-theory, numerals, residues, since residues has the most dependents), and the combinatorics repo once #402 lands.
   • The interface and the site last, when libraries already arrive by registry and the core gate stops building the site.

History. git filter-repo --path <dir> per package, with --path-rename to the root, and every historical path of that package listed. Layouts moved more than once (packages/<name> to packages/symbols/<group>/<name>, records from reference/*.yaml to reference/<Head>/), and rename detection gives up on the mass moves, so a bare path loses history. Check with git log --follow. Keep filter-repo's commit map to translate branch tips, and rewrite #593 in messages to enumeratio/enumeratio#593.

In-flight work, per extraction.

1. The lane ledger blocks lanes whose boundary overlaps the package (rule 1 already enforces the overlap check).
2. Open PRs on that path land or rebase first, at the end, as usual.
3. Anything still open is replayed onto the new repo through the commit map.
4. The monorepo directory is replaced by the published package under the same name, workspace:* becoming a range, so no import changes; link: overrides serve local work and worktrees.
5. Lane files take repo:glob boundaries, and LANDING is held per repo. BACKLOG dispatch prompts get a path map.

A change across repos is two lanes with an after: link. A compute-engine bump is a train: core first, then census on the new engine, then the bot's per-library PRs.

6. Decisions and remaining questions

Decided (2026-10-02): every lean below stands. Libraries version independently against a system range; records stay in their library; generated cross-library data is built in census or the distribution, never committed; core bumps compute-engine first and a bot opens the per-library PRs; nightly census feedback is acceptable, with an opt-in downstream run for core PRs; we own the @enumeratio scope as an npm org. Phase 0 has started.

1. Lockstep or independent versions? Lean: independent per library, a system range, and the census-blessed distribution for anything that ships whole.
2. Do records stay in their library or move to a data repo? Lean: stay (§3). Compute-engine's own heads' records stay in core.
3. Where does generated cross-library data live (manifest table, crosswalk, forms table, declarers)? Lean: built in the distribution or census from the installed libraries, never committed. External-source snapshots are fetched and cached in census.
4. Compute-engine bumps. Direction: only the nucleus depends on compute-engine (§4), so a bump becomes a nucleus release, classified as patch-safe or library-visible; census reports which libraries pass on the new nucleus and a bot opens each library's range bump. Not adopted yet: bumps stay as they are until the facade exists. Open: which subset the facade exposes, and how components are encapsulated.
5. CI cost and feedback latency. Per-repo CI is smaller, but the nightly census holds all the cross-checking. Is next-morning feedback on a cross-library break acceptable, or do core and structures PRs always run the downstream job?
6. npm scope and publishing. Nothing is published yet; the @enumeratio scope is ours, an npm org. Public or private, trusted publishing with provenance, WTFPL. Rename before the first publish, or accept the names? (utils is the only package not marked private.)
7. Grouped repos. Is combinatorics + statistics + polytope the only cluster, or do diagram + braid and hypercomplex + geometric stay together too?
8. Core's edge. Do evaluation, ce-patches, formats and bench stay in core, or become their own repos?
9. Code libraries. Every library of ours is code. Is "trusted origin by scope" the whole policy, and what do third-party code libraries get beyond it?
10. Repo names. enumeratio/adeles is taken by the fork of the Sage package kept for the upstream patch, and compute-engine is a fork too. Plain names, or a prefix?
11. Lanes across repos. Where does the ledger live (outside any repo, or core's common dir), and does "fewer, bigger PRs" survive smaller CI?
12. Issues and the fixup routine. One tracker in core, or one per repo? The routine has to read N repos either way.
13. The interface split. Does it go last as above, or stay with core for good?