Problem Statement
@lde/search-indexer calls itself the composition layer, but only exports
createSearchIndexer(config) and configFromEnvironment(). The composition itself –
registry selector, QLever import resolver, Typesense writer factory – is internal.
That is fine until a Deployment needs behaviour. searchIndexerPipeline documents the
route for that:
A deployment that needs more – a bespoke root selector, per-stage tuning, non-SPARQL
readers, or quad-level plugins – composes searchStages, searchIndexWriter and
Pipeline directly; this convenience owns no capability of its own.
Taking it means giving up all of createSearchIndexer, including the parts unrelated to
the reason for leaving: the Docker socket, network and container naming of the import
path, and collection naming. About 55 lines of fiddly wiring, copied out of a package the
Deployment already depends on – and frozen there the first time a default changes.
Solution
Export the three helpers unchanged, so adding one transform costs the transform rather
than a fork of the composition layer.
User Stories
- As a Deployment, I want to attach my own transform without re-implementing dataset
selection, distribution resolution and writer construction, so that adding behaviour
costs only the behaviour.
- As a Deployment, I want the QLever import path and collection naming wired for me, so
that I do not re-derive them from the reference docs and drift from the served API.
- As a Deployment needing nothing bespoke, I want
createSearchIndexer unchanged, so
that the simple case stays simple.
Implementation Decisions
A visibility change, not a redesign. The helpers already take an IndexerConfig (public)
and return public types; their signatures and names stay as they are.
Nothing else changes: no overrides parameter on createSearchIndexer, no hook on
SearchIndexerPipelineOptions, no change to the mounted module contract.
Document the composed path with a worked example, including two details that are easy to
get wrong and impossible to discover:
- the reader’s
subjectVariable must match the stage’s root variable (root for
selectByClass);
- a field populated by a transform must still declare a
path – projection skips a field
with neither a path nor a derive, so a transform-minted IR Alias is never read.
Testing Decisions
Each helper is asserted on what it builds from a given config, not how.
distributionResolverFrom: undefined without QLever config; the network-scoped and
unscoped forms produce their documented container naming.
writerFactoryFrom: in-place vs blue-green per mode; collection prefix applied.
datasetSelectorFrom: criteria carried through.
Prior art: the existing search-indexer tests already cover this through
createSearchIndexer; exporting lets the assertions move to the units that own them. Add
one test composing the helpers with searchStages, searchIndexWriter and Pipeline as
the documented example does, so that path cannot rot while createSearchIndexer passes.
Out of Scope
- An overrides parameter on
createSearchIndexer, or a readers/transform option on
SearchIndexerPipelineOptions. The documented route already reaches
SearchStageType.readers.
- Any change to the mounted schema-module contract.
- Shipping any transform: this package composes and contributes no domain logic.
Problem Statement
@lde/search-indexercalls itself the composition layer, but only exportscreateSearchIndexer(config)andconfigFromEnvironment(). The composition itself –registry selector, QLever import resolver, Typesense writer factory – is internal.
That is fine until a Deployment needs behaviour.
searchIndexerPipelinedocuments theroute for that:
Taking it means giving up all of
createSearchIndexer, including the parts unrelated tothe reason for leaving: the Docker socket, network and container naming of the import
path, and collection naming. About 55 lines of fiddly wiring, copied out of a package the
Deployment already depends on – and frozen there the first time a default changes.
Solution
Export the three helpers unchanged, so adding one transform costs the transform rather
than a fork of the composition layer.
User Stories
selection, distribution resolution and writer construction, so that adding behaviour
costs only the behaviour.
that I do not re-derive them from the reference docs and drift from the served API.
createSearchIndexerunchanged, sothat the simple case stays simple.
Implementation Decisions
A visibility change, not a redesign. The helpers already take an
IndexerConfig(public)and return public types; their signatures and names stay as they are.
Nothing else changes: no overrides parameter on
createSearchIndexer, no hook onSearchIndexerPipelineOptions, no change to the mounted module contract.Document the composed path with a worked example, including two details that are easy to
get wrong and impossible to discover:
subjectVariablemust match the stage’s root variable (rootforselectByClass);path– projection skips a fieldwith neither a path nor a derive, so a transform-minted IR Alias is never read.
Testing Decisions
Each helper is asserted on what it builds from a given config, not how.
distributionResolverFrom:undefinedwithout QLever config; the network-scoped andunscoped forms produce their documented container naming.
writerFactoryFrom: in-place vs blue-green per mode; collection prefix applied.datasetSelectorFrom: criteria carried through.Prior art: the existing
search-indexertests already cover this throughcreateSearchIndexer; exporting lets the assertions move to the units that own them. Addone test composing the helpers with
searchStages,searchIndexWriterandPipelineasthe documented example does, so that path cannot rot while
createSearchIndexerpasses.Out of Scope
createSearchIndexer, or a readers/transform option onSearchIndexerPipelineOptions. The documented route already reachesSearchStageType.readers.