Compose per-type projecting stages in @lde/search-pipeline, retire projectGraph and the buffering searchIndexWriter, and prove the memory bound. The scalability landing — ADR 13. Depends on Stage<Out> and projectRoots.
Scope
searchStages(options): Stage<TypedSearchDocument>[] — one stage per root type. Sketch:
{ schema, types: [{ searchType, rootVariable, itemSelector, readers, batchSize? }], writerFor }
Each stage's project closure: projectRoots(quads, bindings.map(b => b[rootVariable].value), schema, searchType), tagged { searchType, document }. rootVariable couples the selector's projected variable to what the projection reads — settle this interface here, against a working Stage<Out>.
selectByClass(searchType) helper: ?x a <class> — one word for the object grain, where class is the source class. Not a default (three of DR's four catalog types have no source class).
- Move
TypedSearchDocument from @lde/search to here: it exists only because a terminal routes, which is glue, not projection.
searchIndexWriter → Writer<TypedSearchDocument>: keeps ADR 9's per-collection fan-out (route by searchType.class to per-type runs) and the run lifecycle; stops projecting and stops buffering.
- Delete
projectGraph, rootsByType, frameByType from @lde/search.
Acceptance
- Memory flat by counting (ADR 12): run a synthetic input at 10 and 10,000 roots,
batchSize: 10, against a recording fake Writer; assert project invoked ceil(roots/batchSize) times, and peak live document count identical at both sizes. Fails against today's searchIndexWriter; passes here.
- No behavioural regression: the DR-shaped multi-collection integration test (Typesense testcontainer) — same documents land in the same collections.
projectGraph gone from the workspace.
Design: ADR 13. Parent: #606.
Compose per-type projecting stages in
@lde/search-pipeline, retireprojectGraphand the bufferingsearchIndexWriter, and prove the memory bound. The scalability landing — ADR 13. Depends onStage<Out>andprojectRoots.Scope
searchStages(options): Stage<TypedSearchDocument>[]— one stage per root type. Sketch:projectclosure:projectRoots(quads, bindings.map(b => b[rootVariable].value), schema, searchType), tagged{ searchType, document }.rootVariablecouples the selector's projected variable to what the projection reads — settle this interface here, against a workingStage<Out>.selectByClass(searchType)helper:?x a <class>— one word for the object grain, whereclassis the source class. Not a default (three of DR's four catalog types have no source class).TypedSearchDocumentfrom@lde/searchto here: it exists only because a terminal routes, which is glue, not projection.searchIndexWriter→Writer<TypedSearchDocument>: keeps ADR 9's per-collection fan-out (route bysearchType.classto per-type runs) and the run lifecycle; stops projecting and stops buffering.projectGraph,rootsByType,frameByTypefrom@lde/search.Acceptance
batchSize: 10, against a recording fakeWriter; assertprojectinvokedceil(roots/batchSize)times, and peak live document count identical at both sizes. Fails against today'ssearchIndexWriter; passes here.projectGraphgone from the workspace.Design: ADR 13. Parent: #606.