Speculative Statistics as Fields

Design: statistics and maps as fields of a collection's elements

Speculative design. Open, not settled. Extends Speculative Statistics and Maps (signed off: no head per statistic or map, reached through CombinatorialStat / CombinatorialMap) and builds on the carrier and order rulings in Speculative Combinatorics Layering and Plausible. Nothing here is built.

Why again

That design is built for the ~84 statistics and ~25 maps we implement. It left phase 4 (the unimplemented tail) as "free once dispatch exists". It isn't free, and the symbol reference shows why: ~360 catalogue stubs (stub: carrier, statOn:/mapOn:) are pages for heads that don't exist. CombinatorialStat(x, 'Parity') comes back unevaluated; nothing computes behind the page. A reader can't tell a statistic we compute from one we merely catalogue, a stub can't carry an example (examples are tests, and there is nothing to test), and a statistic's page, its collection's page and its library's page are three unrelated places.

The proposal: a statistic or map is a field of the elements of the collection it applies to, listed and documented on that collection, generated lazily when asked, and owned by the library that owns the collection (or by a library that extends it). The stub pages go away; their catalogue rows become field rows.

What FindStat's data says (crawled 2026-10-03, all of it)

1997 statistics, 336 maps, 24 collections, ~1.7M (object, value) rows, 129 MB of JSON. Each statistic ships a title, description, references, Sage source (1716 of 1997 have one, ~260 chars median), generating functions and a StatisticData table (median ~1200 rows; objects as text, e.g. [2,1], {{1}}, [1,0,1,0]). Each map ships domain, codomain, properties (Bijection, Graded map, Involution…), Sage source and a MapData table.

Concentration matters for what to build first:

collection	statistics	maps from it
Permutations	405	57
Graphs	313	21
Dyck paths	293	47
Integer partitions	239	15
Set partitions	110	19
Posets, Cartan types, binary words, compositions, …	20–85 each

Four carriers we already have (permutations, Dyck paths, partitions, set partitions) hold ~1050 of the 1997 statistics, over half. We match 58 of them by value today. So the first ~1000 statistics need no new carrier, only definitions, and every definition arrives with an oracle: any definition we write is checked against ~1200 rows of FindStat's own values. That inverts the stub problem: instead of "catalogued, nothing to test", a field is born with its tests.

Not covered by our carriers at all: graphs (313), posets/lattices (119), Cartan types (75), perfect matchings, binary trees, ASMs, tableaux…. Those wait on carriers, and on the library split below.

Proposal

1. The field

A statistic is applied as CombinatorialStat(x, "Name") (and CombinatorialStat(x, "St000018") by FindStat id), the form that exists today; there is no x.Name sugar (Epsil prefers Name(x) and doesn't plan one). A collection's field set is its carrier's registry plus anything equivalence transports (registerEquivalence). Over a collection, CombinatorialStat(C, "Name") is the lazy collection of values, as today. Maps are the parallel CombinatorialMap(x, "Name"), returning a carrier value.

"Lazy" has two meanings and we want both: a field evaluates on request (no head declared per name at boot, so boot cost is the table, not 400 declarations), and a field's definition is loaded on request (the library that declares it is resolved by the kernel's resolver when the name is first seen; see §3). Discoverability, which sugar would have given, comes from the collection page's field tables (§4) and About/Information on a value.

2. Where a field's record lives

Beside the collection, not in a global catalogue: the carrier's package (or an extending library) holds fields/<Name>/ with index.md (summary, FindStat id and names, Sage reference), the Epsil definition, and examples.tsv. The record is not a symbol page: it renders as a row on its collection's page and as a page of its own per carrier and statistic, generated from the records and the FindStat data, whose call form is the CombinatorialStat(x, "Name") it documents (never a global head page). Two carriers that share a statistic's name have two pages: they are distinct implementations, even when they have a lot in common. Examples are x.Name rows with known: <FindStat value>, source: FindStat St000018, so known.test.ts makes FindStat the independent check.

A statistic that FindStat has and we don't is a declared but unimplemented field: shown on the collection page greyed, with FindStat's own title and description and a link, no example, no page of its own, and counted in a visible "N of M FindStat statistics implemented" figure. That keeps the catalogue honest without pretending a head exists.

3. The library split

Collections will split across libraries (Speculative Per Package Repos); fields follow with three rules:

1. A field lives in the library that owns its carrier, unless it needs something that library doesn't depend on.
2. A field that needs another library's carrier or heads lives in a library that extends both (as hecke extends structures). A statistic of a permutation that is defined through a poset belongs to neither alone.
3. Fields are resolved by carrier type, never by name. The kernel's resolver already maps a type or head to the library that declares it; add fields rows ("carrier permutation has field Inversions in library X, FindStat St000018") so x.Inversions loads X on first use and a docs page lists the fields of a collection across every library that contributes, with the contributing library named on each row.

A shared, data-only findstat library (ids, titles, descriptions, known tables, Sage source as mapped bindings) is the crosswalk target: it holds no definitions, so it never has to depend on, or ship with, the libraries that implement them. The 129 MB crawl does not ship; only the slice used as examples (a handful of small objects per statistic) is committed, as ordinary example rows in the owning library.

4. Docs shape

• Collection page gets a Statistics and a Maps section: one row per field (name, FindStat id, summary, library, status), expandable to its examples, and the existing distribution/Tally figures where cheap (the Mahonian numbers). The Maps section lists every map from and to the collection, so the page answers "what are the distinct maps in and out of this collection?" at a glance. The per-carrier, per-statistic and per-map pages hang off these rows.
• Symbol index drops the stub heads; the Statistics and Maps catalogue pages are generated from the same registry (as they are now) and group by collection.
• Search indexes fields as Collection.Field so "Inversions" finds the permutation field.
• Information / About on any value lists its fields, by carrier type.

5. What it does to what exists

• The 84 implemented statistics and 25 maps keep working; their records move from reference/entries to beside their collection (phase 3 of the existing design), and become fields.
• The ~360 stub records retire; 18 have a FindStat id today, so most of the rest are arithmetic statistics (Abundance, BigOmega) that FindStat doesn't cover. Those are arithmetic functions of integers and belong as ordinary heads in number-theory/arithmetic where they are one-argument heads, not combinatorial fields.
• Bare heads (Inversions) stay on the existing deprecation path.

Plan, in order of cost

1. Data, not code. Import FindStat as a read-only dataset: id → collection, title, description, sage source, and a small sample of StatisticData. No behaviour change. (Crawl is done.)
2. Examples and checks for the 58 matched from step 1's data: known rows with source: FindStat St….
3. Field syntax and the fields rows in the resolver; stubs retire; collection pages gain the sections. This is the design-sign-off step: it changes spelling and the docs shape.
4. The ~1000 on existing carriers, by hand or by translating Sage source into Epsil where mechanical (reading x.number_of_inversions() is not mechanical; a handful of idioms are). Each lands with its FindStat table as the oracle, so wrong definitions fail at once. This is the long tail and can be farmed out as lanes by collection.
5. Maps, with MapData as the oracle, same way.
6. New carriers (graphs, posets) as their own libraries, each with its stats.

Decisions (2026-10-06)

1. Name(x) / CombinatorialStat(x, "Name"), no x.Name sugar.
2. A section on the collection page, plus generated per-carrier-per-statistic and per-map pages keyed to the CombinatorialStat(x, "Name") / CombinatorialMap(x, "Name") form. The collection page lists all maps from and to it.
3. Arithmetic statistics (Abundance, BigOmega, AliquotSum…) are not combinatorial fields: their records say statOn: IntegerFactorization, so they are functions of an integer through its factorization, and become ordinary number-theory heads, Abundance(n). Their domain is the nonzero integers, signs included: a negative integer factors as a sign unit times positive primes (Wolfram: FactorInteger[-12] is {{-1,1},{2,2},{3,1}}), and the divisor-based statistics read the magnitude (DivisorSigma[1,-12] = 28, PrimeOmega[-12] = 3, PrimeNu[-12] = 2, MoebiusMu[-12] = 0, EulerPhi[-12] = 4). Zero is per head: FactorInteger[0] is {{0,1}} and EulerPhi[0], MoebiusMu[0] are 0 by convention, but DivisorSigma[1,0], PrimeOmega[0] and Divisors[0] stay unevaluated (every integer divides 0). So each statistic's record states its domain and its value at 0 and at negatives, as each head's does today.
4. A field page is per carrier. Same name on several carriers = distinct implementations, each its own page, each checked against its own FindStat table.
5. Attribution, and ask before the next crawl. FindStat is credited wherever its data or ids appear (sample rows, known sources, the collection tables), and we reach out to the maintainers (info@findstat.org) before the next crawl. Pre-release, the current crawl stands.

Settled afterwards

• A FindStat statistic on a carrier we haven't implemented is not listed. The fix is to implement the carrier (graphs, posets, … each as its own library, with its statistics), not to carry greyed rows.
• A map's FindStat properties (Bijection, Involution, Graded map, …) are recorded as its laws wherever the map is listed (a collection's or carrier's Maps section, or its own page), even though FindStat, not we, checked them: we inherit them and say where they come from. Where a law is one our own tests can check (inverse, orderIsomorphism), it is.
• Per-carrier statistic and map pages carry the examples and the detail; collection and carrier pages link to them and only list what each is (name, FindStat id, properties), so nothing is written twice.

Built so far

• findstat/data/{statistics,maps}.json in the combinatorics package: a committed snapshot (attributed) of FindStat's data, made by packages/reference/scripts/collect-findstat-data.ts snapshot from a slow crawl of the site's own per-item JSON.
• collect-findstat-data.ts examples: for each statistic of ours matched to a FindStat id, a few of its rows as CombinatorialStat(x, "Name") examples with known and source: FindStat St…. All 171 rows agree with FindStat. One finding on the way: a bare head such as Peaks is typed for a single carrier and errors on a Dyck path, which the CombinatorialStat form avoids.