Replies: 1 comment
-
|
Closing in favor of a new approach |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Problem
On SQLite/D1, a taxonomy-filtered collection listing —
getEmDashCollection("posts", { where: { category }, orderBy, limit })— applies thetaxonomy filter as a correlated
EXISTSon aSELECT * FROM ec_<collection> ... ORDER BY ... LIMIT ?. A stats-blind D1 planner drives from the collection's order index and probesthe taxonomy per row. For a selective term the
LIMITnever fills, so the query walksthe whole collection — tens of thousands of D1 rows read for a page returning a handful
(#1834). On D1, rows-read is the billed/throttled unit.
The planner can't fix this on its own: choosing the fast plan requires knowing the term's
selectivity, and D1 is stats-blind (
sqlite_stat1is never maintained). That one fact hasto come from us.
Proposal
Let the loader plan taxonomy-filtered listings automatically from cached per-term entry
counts.
collection + status + locale, so a term's count equals the rows that plan wouldactually fetch. No fixed "published" semantics are imposed — the count follows the query
(a draft query counts drafts, a French query counts French, one collection counts only
its slice).
content_taxonomies, seek thecomposite pivot index, join the collection by primary key, sort the small candidate set)
or keep the
EXISTSscan for a non-selective one (theLIMITfills fast). The choice isa
√(pageRows · total)cost-model crossover.index-nested-loop
EXISTS— theO(|small|·log|big|)plan, which drives from the mostselective term rather than the least.
back (the seek and scan paths are tested to return identical results). A stale or wrong
count only degrades plan quality, which is what makes eventual-consistency caching safe
here.
[content:<collection>, taxonomies]. The existing epoch graph already fires on exactlythe writes that move a count — content create/delete/status-change/attach bump
content:<collection>; term rename/delete bumpstaxonomies. No new SQL table, nowrite-path hooks, no reconciler.
translation_group, carrying name/slug/count plus acollection total, so a future public "term counts" API (and the admin term-cloud /
archive-nav counts) can reuse the same load.
A composite index on
content_taxonomies(taxonomy_id, collection, entry_id)(migration051) backs the seek — it seeks on both leading columns so a term shared across collections
reads only the queried collection's slice. Postgres is unchanged (keeps
EXISTS; it hasstatistics).
Evidence (real D1)
Against a real dataset (~24k published rows in the collection, ~76k
content_taxonomiesrows), rows-read:
The cost-model threshold picks correctly in both real single-term cases with margin.
Scope
composite index.
archive-nav counts (today status-agnostic, cross-collection, invalidated only by
taxonomy writes) onto this published, correctly-invalidated per-collection map. That's a
visible behavior change (public counts would drop drafts) and needs cross-collection
aggregation.
Seeking maintainer sign-off on: planning taxonomy listings automatically from cached
counts, the object-cache-based count primitive with its coarse per-collection
invalidation, and the out-of-scope split.
Beta Was this translation helpful? Give feedback.
All reactions