Skip to content

Export the search indexer’s composition helpers #691

Description

@ddeboer

Problem Statement

@lde/search-indexer calls itself the composition layer, but only exports
createSearchIndexer(config) and configFromEnvironment(). The composition itself –
registry selector, QLever import resolver, Typesense writer factory – is internal.

That is fine until a Deployment needs behaviour. searchIndexerPipeline documents the
route for that:

A deployment that needs more – a bespoke root selector, per-stage tuning, non-SPARQL
readers, or quad-level plugins – composes searchStages, searchIndexWriter and
Pipeline directly; this convenience owns no capability of its own.

Taking it means giving up all of createSearchIndexer, including the parts unrelated to
the reason for leaving: the Docker socket, network and container naming of the import
path, and collection naming. About 55 lines of fiddly wiring, copied out of a package the
Deployment already depends on – and frozen there the first time a default changes.

Solution

Export the three helpers unchanged, so adding one transform costs the transform rather
than a fork of the composition layer.

User Stories

  1. As a Deployment, I want to attach my own transform without re-implementing dataset
    selection, distribution resolution and writer construction, so that adding behaviour
    costs only the behaviour.
  2. As a Deployment, I want the QLever import path and collection naming wired for me, so
    that I do not re-derive them from the reference docs and drift from the served API.
  3. As a Deployment needing nothing bespoke, I want createSearchIndexer unchanged, so
    that the simple case stays simple.

Implementation Decisions

A visibility change, not a redesign. The helpers already take an IndexerConfig (public)
and return public types; their signatures and names stay as they are.

Nothing else changes: no overrides parameter on createSearchIndexer, no hook on
SearchIndexerPipelineOptions, no change to the mounted module contract.

Document the composed path with a worked example, including two details that are easy to
get wrong and impossible to discover:

  • the reader’s subjectVariable must match the stage’s root variable (root for
    selectByClass);
  • a field populated by a transform must still declare a path – projection skips a field
    with neither a path nor a derive, so a transform-minted IR Alias is never read.

Testing Decisions

Each helper is asserted on what it builds from a given config, not how.

  • distributionResolverFrom: undefined without QLever config; the network-scoped and
    unscoped forms produce their documented container naming.
  • writerFactoryFrom: in-place vs blue-green per mode; collection prefix applied.
  • datasetSelectorFrom: criteria carried through.

Prior art: the existing search-indexer tests already cover this through
createSearchIndexer; exporting lets the assertions move to the units that own them. Add
one test composing the helpers with searchStages, searchIndexWriter and Pipeline as
the documented example does, so that path cannot rot while createSearchIndexer passes.

Out of Scope

  • An overrides parameter on createSearchIndexer, or a readers/transform option on
    SearchIndexerPipelineOptions. The documented route already reaches
    SearchStageType.readers.
  • Any change to the mounted schema-module contract.
  • Shipping any transform: this package composes and contributes no domain logic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions