Namespaces

Namespaces, resources, and the shape of the catalog

Status: partly built: the resolver, contexts and lazy resolution (§3–§5) run in packages/catalog; the definitions-as-data discipline (§6) is in force for the combinatorial statistics. Anything a test pins is cited by the test rather than repeated (Roadmap §1).

This page asks the question that comes before upstreaming any head: which of our names should be heads at all.

The forcing problem is the Postgres-era project's catalogue. Bringing across its collections, carrier domains, stats, maps and aggregates by "declaring a symbol for each" is somewhere between implausible and hostile, and on the measurements below it is the wrong reading too.

1. What is actually there

The catalog has these populations: collections, carriers, statistics, maps. What matters is the gap between rows and names:

big_omega       many collections, but only two carriers  (IntegerFactorization, Numeric)
divisor_count   many collections
major_index     three collections, three carriers  — dyck_paths, permutations, standard_tableaux
                                                     (title "Major index" on all three)

A stat is not one name per row. Many rows collapse into few names, by two different mechanisms:

• Carrier scoping does most of the work. big_omega is one function on a numeric carrier; the many rows are collections over that carrier inheriting it, not many definitions. Most stat names live on exactly one carrier, and for these the row count is pure inheritance: scoping the definition to the carrier removes it.
• Genuine overloading covers the rest. A minority of stat names are defined on more than one carrier (major_index on three). These are one identity with several implementations resolved by argument type, the Postgres model, and the case that needs a domain-keyed signature.

So the pressure is much smaller than the row count, and most of what remains is inheritance rather than overload resolution.

2. The decomposition

So the catalog is not one undifferentiated pile of names. It is three populations with genuinely different shapes, and they want three different mechanisms.

2.1 Carriers — declare them all

Carriers are the small closed vocabulary that everything else is typed by. Collections return them; stats and maps are defined on them; restricted collections return domain-restricted versions of them.

They must be real declared symbols, not resolved strings. If IntegerComposition is only reachable as ƒ("IntegerComposition"), then every type annotation, every assumption, every membership test routes through a string literal — which stringly-types the type system, the exact opposite of the nominal-vs-structural machinery that makes the domain worth having. The point of SetPartitions returning an actual set-partition domain value is that assumptions fire and a constructed set partition is distinguishable from something merely structurally identical to one. A string can't carry that.

The carrier vocabulary is small enough to declare outright.

2.2 Stats and maps — overloads, not namespace pressure

Heads defined on carriers rather than on collections do not need namespacing. They need two things, neither of which is a namespace:

1. Definitions scoped to the carrier, so a collection over that carrier inherits them. This alone accounts for most stat names.
2. A domain-keyed signature for those that are genuinely multi-carrier, which is on the upstream question list (the Equal/2-is-not-a-signature objection): arity cannot distinguish major_index(permutation) from major_index(dyck_path), and it should not have to.

The Postgres-era project's base_map.scope already carried 'carrier' | 'collection', and most map rows are carrier-scoped: this is an imported model, not an invented one.

2.3 Collections — the genuinely open tail

The collections are a growing, individually low-notation, and mechanically derived. This is the population that wants a resolver rather than a declaration, and it is the only one.

2.4 What actually computes today

Resolving a name and being able to evaluate it are different things. The numbers here go stale within weeks, so this section says where each one lives rather than restating it; a test pins each, and [[Roadmap]] §1 is the index of the lists.

• Carriers: all declared (packages/symbols/combinatorics/combinatorics/src/carriers.ts).
• Statistics: every catalog statistic on the covered carriers (permutations, integer partitions, Dyck paths, set partitions) is defined, native, or a cardinality; packages/catalog/tests/statistics-coverage.test.ts refuses anything else. Other carriers are not claimed.
• Maps: packages/symbols/combinatorics/combinatorics/src/maps.ts defines them; src/frontier-maps.ts lists the ones the catalog has that are not built, with the reason.
• Collections: the tail, where only a fraction have kernels (packages/symbols/combinatorics/combinatorics/collections). This is what matters for sequencing.

A resolver that resolves every collection name and can evaluate a fraction of them is still worth having, because resolution carries the carrier, the grade axes, the title and the overload set — enough to document, search, type-check and cross-link a name whose kernel does not exist yet. So the namespace question and the kernel question are independent, and the namespace one is nearly free while the kernel one is most of the collections.

A resource that resolves without a kernel stays symbolic rather than erroring. That is deliberate: it is a real name with real metadata that this engine cannot evaluate, and holding is the honest report.

3. The resolver

Wolfram's shape, near enough:

ResourceFunction["HadamardGamma"][x]

One head, a string identifier, resolved through a registry into a function value that then applies. One head covers every arity and kind, because the resolution is data.

3.1 What compute-engine can already do

Measured against compute-engine, not assumed:

ce.box([["Resource", "'Perms'"], 3])
// ["Apply", ["Resource", "'Perms'"], 3]      — nested head canonicalises to Apply

ce.declare("Resource", { signature: "(string) -> function", evaluate: … })
ce.box([["Resource", "'x'"], 21]).evaluate()
// 42                                         — resolve-then-apply evaluates today

So the curried shape works at the MathJSON layer with no machinery at all. It does not parse from LaTeX (ce.parse("f(x)(3)") gives ["Multiply", 3, "f", "x"]), so the typed spelling stays flat: ƒ("WeakCompositions", 5).

3.2 Late declaration, and why the parse tree is repairable

An unknown name is demoted to a multiplication operand when an expression is canonicalised, but the parse tree stays repairable, so static analysis works for LaTeX input. Three separate facts, all measured:

// 1. A boxed expression DOES pick up a later declaration.
const b = ce.box(["LateHead", 3]);        // LateHead undeclared
ce.declare("LateHead", …);
b.evaluate();                              // 30

// 2. Multi-argument application already parses undeclared — the comma defeats Multiply.
ce.parse("\\operatorname{Foo}(3,4)");       // ["Foo", 3, 4]

// 3. The ambiguous unary case keeps its structure BEFORE canonicalisation.
ce.parse("\\operatorname{Foo}(3)", { canonical: false });
// ["InvisibleOperator", "Foo", ["Delimiter", 3]]
ce.parse("\\operatorname{Foo}(3)");         // ["Multiply", 3, "Foo"]  — canonical form commits

Only the unary application of an unknown name is ambiguous at all, and even there the information survives: InvisibleOperator is exactly the repairable juxtaposition node, and it is canonicalisation, not parsing, that throws the alternative away.

That makes the whole pipeline work on the AST, with no source-text lexer:

const raw = ce.parse(src, { canonical: false });
// walk for ["InvisibleOperator", <symbol>, ["Delimiter", …]]
// declare the candidates the registry knows
ce.box(raw.json); // re-canonicalises the SAME tree, now correctly

End to end on \operatorname{LateHead}(3) + \operatorname{Other}(2) + a(b+c), with only LateHead and Other in the registry:

raw        ["Add",["InvisibleOperator","LateHead",["Delimiter",3]],
                  ["InvisibleOperator","Other",["Delimiter",2]],
                  ["InvisibleOperator","a",["Delimiter",["Add","b","c"]]]]
candidates LateHead, Other, a
repaired   ["Add",["Multiply","a",["Add","b","c"]],["Other",2],["LateHead",3]]
evaluate   a * (b + c) + 50

a(b+c) correctly stays a multiplication, because a is not a name we claim. The registry is what disambiguates, which is the right place for that decision to live.

This is what lets the Resource(…) wrapper go away. §3 introduced it as one head standing in for a whole catalog, and it is still the right fallback for a name you want to address without installing. But if unknown names can be discovered in the tree and declared before canonicalisation, then the ordinary spelling is just the name — CarlitzCompositions(4) — and the resolver becomes an implementation detail of the loading step rather than something a reader has to type. Direct injection into the global namespace, on demand. Resource then survives only for the case it is actually good at: naming something explicitly, from a context you have not blessed.

Why Wolfram never has this problem: f[x] is application by syntax, unconditionally, so its parser never needs to know whether f is defined. LaTeX has no such luxury — a(b+c) genuinely is multiplication in ordinary notation — so some signal has to break the tie, and "is this a name we know" is the only one available. The ambiguity is inherited from the notation, not invented by compute-engine. What compute-engine could do better is simply not discard InvisibleOperator so eagerly; it already builds the node.

3.3 On spelling — the glyph and the separator

ƒ (U+0192, Option+f on a Mac) is deliberately ugly, and the ugliness is the argument for it: it is never chosen as a variable on purpose, it renders visibly curly rather than as an italic f, and so it can squat a namespace marker without ever colliding with a user's symbol. That holds up.

It is also not the only option, and need not be the stored one. The dictionary mechanism means the head can be named honestly in MathJSON — Resource, say — and carry ƒ as its LaTeX trigger. Typed short, stored legible, serialised back short.

This is a load-bearing use of library-contributed notation: it gates the ergonomics of an entire namespace strategy, not just cosmetics.

The separator is harder, and is not settled. Measured: of ~, $, ::, and the unicode lookalikes ー and ․, only _ is legal in a compute-engine symbol name — everything else falls back to a string literal. _ round-trips (\mathrm{Notatio_{Carlitz}}(3) parses back to ["Notatio_Carlitz", 3]) but renders as a subscript, colliding with the existing subscripted-symbol convention (i_k, j_m, ζ_H). __ is also legal and renders as a superscript, colliding with powers.

~ is unavailable today because LaTeX binds it to a non-breaking space — but that is a better argument for it than against it. A qualified name is precisely one unbreakable run: the separator is a space that must never break. If a character has to be chosen, the one whose existing meaning is "this is all one unit" is the semantically right one, and the conflict is an implementation detail of the LaTeX dictionary rather than a clash of meaning. Wolfram's backtick has no such justification, and has the additional property that a qualified name cannot be written inside Markdown code spans — which is a real cost for a project whose documentation is Markdown.

This stays open. What is settled is the shape: whatever the character, it separates a context from a name, and it is only needed when disambiguating.

4. Contexts — the part that is actually the good idea

A resolver alone gives lazy loading. It does not give a place to put things, and that is the more valuable half.

The proposal: a lazily-populated context namespace, filled as needed, with a search path — so that resolution can find a name without it living in the global symbol table, and so that names can be promoted toward the global table as they earn it.

enumeratio~WeakCompositions        namespaced, resolved on demand
ada~WeakCompositions               by curator
WeakCompositions                   promoted — on the search path, or global

(~ throughout this document stands in for whatever separator is chosen — see §3.3. The registry's SEPARATOR constant (packages/catalog/src/types.ts) is one line to change.)

The ladder has three rungs: namespaced (exists, addressable, off main), blessed (curated set, on the default search path), promoted (global, earns a real head and conventional notation). Only the top rung costs global namespace, and it is reached by evidence rather than by being declared first.

Contexts are the piece compute-engine has no equivalent of. Scopes exist, but they are lexical evaluation scopes, not a namespace mechanism: there is no way to hold a body of names off to one side, opt into a subset, and refer to them in short form without accepting all of them globally. That is what Wolfram's Context plus $ContextPath provide, and the absence is what forces the choice between "declare everything" and "declare nothing". It belongs on the upstream pile beside the notation ask, though it is a much larger request and one we can prototype entirely on our side first.

Two observations on prior art:

• Wolfram keeps Context and ResourceFunction as separate mechanisms, and the reason is that contexts are local and eager while resources are remote and lazy. A lazy context — the thing proposed here — is the synthesis of the two, and is genuinely not what Wolfram built. That is a point in its favour, not against it, but it does mean there is no reference implementation to copy.
• Once the identifier carries a curator (ada/WeakCompositions, ada`WeakCompositions), the string is a namespace. The choice between encoding it in a string argument and encoding it in a context path is not cosmetic: with a string, promotion is a rename (every existing reference breaks); with a context path plus a search path, promotion is a path change (existing fully-qualified references keep working). That argues for the context form, and it argues for deciding early, because the migration is much cheaper before anything is written down.

5. Lazy resolution — the mechanism

This is the part that actually matters, and §3.2 makes it tractable.

5.1 The loop

parse non-canonically
  -> walk the AST for application candidates and unknown heads
  -> resolve those names against the registry          (async: fetch, import, whatever)
  -> declare what resolved
  -> canonicalise the same tree
  -> evaluate

Evaluation itself is synchronous — that is not negotiable and should not be fought. But nothing above requires it to be async: the resolution happens before evaluation, in a phase that is allowed to await. The sync boundary is respected by moving the work outside it, not by suspending inside it.

5.2 Completeness, and the residual case

Static analysis over the source AST cannot see a head that is synthesised during evaluation — and handlers can do that (verified: a handler boxing ["Carrier", n] evaluates fine, provided Carrier is declared). So the pipeline is complete exactly when:

every head that can appear at runtime but not in the source is already loaded.

Two things make that condition hold rather than hope:

• Carriers are loaded eagerly. They are the dominant category of runtime-synthesised head — a collection's elements are carrier values — and §2.1 declares them all.
• The rest is lintable. Handler bodies are ours; a check that our own handlers only construct heads from their own library or its declared dependencies is a lint over our sources, not a guarantee we need from upstream.

That last point is worth stating plainly because it is easy to assume otherwise: compute-engine does not constrain what a library's symbols reference. LibraryDefinition carries a requires field, but there is no public API to register one and nothing inspects handler bodies — a JavaScript evaluate can box any head it likes. The discipline is ours to impose and ours to check.

5.3 When resolution fails anyway

If an unresolvable head is reached at evaluation time, the honest behaviour is to hold, not to throw: the expression stays symbolic and carries enough information to say what was missing. A held expression can then be re-resolved and re-evaluated by the outer loop — which is the same fixpoint the pipeline already runs, entered one step later.

The residual design case is a curator-supplied carrier: a published collection over a domain that is not ours. That is the real dependency edge, and it is manageable because it is rare and because holding gives it somewhere to fail safely.

6. Definitions as data

A symbol's definition is data where it can be: almost every head we add is defined in Epsil, as an expression tree, and the accelerated TypeScript handler stays the one that runs (see Epsil-only definitions for how definitions and kernels relate). The Epsil definition is slow, and is kept for three independent reasons:

1. It is the specification. A FullForm AST that rewrites MajorIndex down to bare primitives says what the head means, unambiguously, in a form a reader can expand.
2. It is an oracle we own. The handler and the definition must agree on every value, a differential test that catches truncations and off-by-ones the example suite does not, without a second system to compare against. A definition is worth adding only where that differential runs.
3. It makes the core explicit. Reduce every head until nothing reduces further and what remains is the primitive set. Everything above that line is data, which can be lazily loaded, versioned, fetched, curated and promoted. §4 and §5 pay off in proportion to how much of the catalog is above it.

6.1 The frontier

The reduction does not bottom out at zero. Some heads are primitive by construction and some are irreducibly numeric (ζ evaluates; it does not rewrite into anything simpler). So the artefact is a reduction tree with a named frontier: the heads that do not reduce, each labelled with why (primitive, numeric kernel, foreign call). Choosing it too small makes definitions unreadably deep; too large and the "core" stops meaning anything. Every head either reduces or is on the declared frontier, and a lint holds that closed.

Not everything is expressible: the diagram algebras, the Laurent-polynomial Burau kernel and Bareiss elimination are algorithms over mutable state, and belong on the frontier with an honest label.

6.2 Showing it

A reference page shows, per head, every implementation it has: the bindings block of the record (BindingOrigin, packages/entry/src/types.ts), a list because a head is made of several things:

origin	what it is	stored as
reference	the defining expression, in Epsil	the expression itself
native	the TypeScript that actually runs	a pointer, checked
compiled	a compute-engine compile target (glsl, …)	a pointer, derivable
component	a web component: the element IS the answer	the element name
mapped	the equivalent in Wolfram / SymPy / Sage	a pointer into MAPPINGS

A native row is a pointer, never a copy, and a test checks the file exists: a copied implementation rots. Each row carries an environment (engine, browser, gpu, node, external), because some heads only mean anything somewhere: a plotted ζ bottoms out in <notatio-complex-plot>, and the rendered phase portrait is the value. Zeta has implementations in the engine, on the GPU (compute-engine's GLSLTarget), as a custom element in the browser, and in external systems it maps to.

A reference row shows its expression in TreeForm (<notatio-out form="tree" raw>): the tree of heads opened one level at a time, each head with an entry linked to it. raw matters: the definition is shown as authored, not canonicalised. In TreeForm a head with a reference row shows ≝, and clicking it unfolds the application in place, one reduction step, foldable back; a frontier head shows its primitive reason. The step stops exactly where the frontier says it should.

6.3 The core, derived

The combinatorial statistics are defined as expressions (packages/symbols/combinatorics/combinatorics/src/statistics/), and core.ts computes what they rest on by walking every definition and subtracting the heads we define ourselves. The core is the list tests/statistics-core.test.ts pins: list and arithmetic primitives of compute-engine (At, Filter, Count, Map, Fold, Range, Sum, comparisons, and a few more). It is pinned so that reaching for one more primitive is a visible event; the core grows only to close a frontier entry.

The tower falls out of the same walk: a statistic sits one level above the deepest head it uses. Most reduce straight to the core; StandardTableauCount is n! over the HookProduct head rather than a second walk of the Young diagram. A flat tower means definitions are still repeating each other.

Rules that hold:

• A name collision shows up as a cycle. A partition statistic that is compute-engine's Length applied to the parts is not redefined: it is NATIVE_TO_ENGINE, a catalog name that needs nothing from us.
• Check the engine before declaring a limit. compute-engine has Fold, and a fold accumulator can be a growing list or a list of lists, which expresses patience sorting (LongestIncreasingSubsequence) and the other prefix folds.
• When a list has to BE a value rather than a promise, fold it. Map over a Range stays lazy; rebuilding the list with a fold evaluates eagerly.
• Every definition states its own floor. Range(1, 0) counts down rather than yielding the empty list, so a permutation too short to have the feature must be handled explicitly.
• Expression definitions are slow. Triple-loop statistics cost about n³ evaluations and are checked over smaller sizes. A definition is a specification, not a fast path, which is the argument for keeping both and the differential between them.

Every catalog statistic on the covered carriers is accounted for (defined, native to compute-engine, or a cardinality answered by Count) and the frontier is empty; packages/catalog/tests/statistics-coverage.test.ts holds that closed.

7. What is built

packages/catalog implements §3 and §4 as a working prototype, and its tests fail loudly when the catalog moves.

• What declareCatalog needs lives on each head's own record (grades/carrier/unbounded, catalogCarrier, statOn/mapOn); packages/reference/scripts/collect-catalog-records.ts writes src/catalog-records-data.ts, which resources.ts reads, and a currency test pins it.

• ResourceRegistry is the context registry: add, bless, unbless, resolve, candidates. Qualified spellings bypass the search path, which is the property that makes promotion a path change.

• declareCatalog(ce) declares exactly ONE head for the whole catalog, and answers a resource only when a head of its name is already declared — it asks the engine rather than recording the answer, so the registry cannot go stale against what is declared.

• CATALOG_LATEX carries the ƒ trigger, and cannot be installed by declareCatalog. It has to be merged into a LatexSyntax at construction.

• prepare(ce, source, registry, install, parse?) is §5.1 running: parse, find candidates, resolve, install, box. It takes LaTeX or MathJSON, and install may be async — which is the whole point, since evaluation may not be. compute-engine parses an unknown name applied to parentheses as an application, so the default parse stage is the canonical one. The stage is pluggable: rawParse keeps the unresolved InvisibleOperator reading for a front end that wants to decide application versus product itself (e.g. keeping a(b+c) a product for names nobody claims), and the candidate walk reads either tree.

Registering every collection declares nothing and costs one Map insert each — there is a test asserting exactly that. The lazy loop is tested end to end on a real catalog name: nothing declared up front, ["Count", ["Subsets", 4]] installs Subsets on demand and evaluates to 16.

Open

• String identifier vs context path. §4 argues for the path on promotion-cost grounds. Not settled, and the cheapest moment to settle it is before anything is written down.
• Which separator character. §3.3: only _ is legal in a symbol name today and it collides with the subscript convention. ~ is semantically right and currently taken by LaTeX.
• Where the frontier sits. §6.1: the primitive set is a decision, and §6 depends on it.
• Does a resolved resource keep identity across sessions? Wolfram resources have a UUID behind the name. If promotion is a rename, something stable underneath has to survive it.
• Where the registry lives. It is built from the records today; the aspiration is fetched on demand. The shape should not assume either.
• Multi-carrier stats need the domain-keyed signature to exist first. The single-carrier ones need only carrier-scoped definitions, which is ours to build; the multi-carrier ones are a hard dependency on the upstream question list.
• Aggregates are not placed in one of the three populations; they need the same census first.