Skip to content

feat(search)!: serve a surfaced inline reference as a nested document - #689

Merged
ddeboer merged 3 commits into
mainfrom
feat/nested-inline-references
Jul 31, 2026
Merged

feat(search)!: serve a surfaced inline reference as a nested document#689
ddeboer merged 3 commits into
mainfrom
feat/nested-inline-references

Conversation

@ddeboer

@ddeboer ddeboer commented Jul 31, 2026

Copy link
Copy Markdown
Member

Makes the API device of an Inline Reference real end to end: a Reference Type declared inline with output is now stored, reconstructed and served as a nested object carrying its own Search Fields. A multi-valued reference keeps each referent’s values grouped, so a consumer never pairs parallel arrays by index, and a Deployment no longer flattens referents onto the root.

Fix #674

What changed

@lde/search

  • SearchValue gains NestedDocument – the referent of a surfaced inline reference, carrying an id only when the referent is a named node (a nested document is read, not addressed).
  • nestedReferenceType(schema, field) – the one predicate the collection definition, result reconstruction and the API surfaces share for “does this field carry a nested document?”.
  • nestedFieldName(parent, name) – the nested Physical Field convention (media.contentUrl), the nested counterpart of physicalFields.
  • searchSchema gains the invariants the nested shape needs, so an unserviceable nesting fails at startup, naming the field: an inline reference is either a reading device (no Role) or surfaced (output), and every field of a Reference Type carries output only – searchable/filterable/facetable/sortable/labelSource are rejected, because no query compiler serves them and they would be silently ignored per query.
  • The projection now honours array for keyword and reference as it already did for every other kind: a declared array stores a list, a single-valued field the first value the graph carries. It previously wrote a list either way, so one declaration meant a list to the projection and a scalar to the collection definition and the API output type – at root the engine rejected the document, and inside a nested document (which the engine does not type-check) it survived to fail per query at the surface.
  • The port contract (@lde/search/testing) gains the nesting rule, so every engine implementation is held to it.

@lde/search-typesense

  • The collection definition stores a surfaced inline reference as object/object[] (turning on enable_nested_fields), with one nested Physical Field per output field of its Reference Type, all index: false – nested content is display weight on disk, so the RAM lever is unchanged. It takes the schema as an option, and a rebuild writer builds its definition at construction, so a nesting type without its schema throws when the writer is built rather than inside the first run under a held lock.
  • Reconstruction rebuilds one nested Search Document per referent, at the engine-adapter level rather than in a surface, so a second surface inherits it (the placement decision from ADR 11). parseSearchResponse now requires the schema, so a caller cannot silently reconstruct nothing for every nested field.

@lde/search-api-graphql

  • A surfaced inline reference’s GraphQL type is built from its Reference Type’s output fields (same per-kind rules as a root type, nullable id), so a client selects a nested object’s fields directly. Other references keep the id-plus-label pair.

Wiring

  • searchIndexerPipeline’s writerFor is called with the schema as well as the type, so a writer can declare the nesting; @lde/search-indexer passes it through.

Breaking changes

  • A keyword/reference field that does not declare array: true now projects a value rather than a one-element list. A Deployment whose source is multi-valued must declare array: true – including on an internal field a derive counts.
  • SearchValue widens: a consumer switching on a reference value must handle a nested document.
  • parseSearchResponse takes the schema as a required fourth argument.
  • buildCollectionDefinition throws for a type surfacing an inline reference unless the new schema option is passed.
  • Declarations that previously carried an ignored Role on an inline reference or inside a Reference Type now fail at searchSchema().

Verified end to end against a real Typesense container (collection creation, import, search, reconstruction), including a mixed named/blank-node multi-valued reference and a referent missing an optional field.

ddeboer added 3 commits July 31, 2026 14:16
- store a surfaced (output) inline reference as a nested object, with one
  nested Physical Field per output field of its Reference Type
  (nestedFieldName), so a multi-valued reference keeps each referent's
  values grouped instead of smeared across parallel arrays
- reconstruct it as a nested Search Document at the engine-adapter level,
  below every API surface, so a second surface inherits the nesting
- add NestedDocument to SearchValue; a referent carries an id only when it
  is a named node
- build the GraphQL reference type from the Reference Type's own output
  fields, with a nullable id, so a client selects a nested object's fields
  directly
- reject a nested declaration the engine cannot serve at startup: an inline
  reference, and every field of a Reference Type, carries the Role output
  and nothing else
- hand writerFor the schema, and take it as a collection-definition option,
  so a writer can declare the nesting
…t mid-run

- require the schema in parseSearchResponse, so a caller cannot silently
  reconstruct nothing for every nested field
- build the collection definition when a rebuild writer is constructed, so
  a surfaced inline reference without its schema throws there rather than
  inside the first run, under a held lock
- declare the nested keyword fixtures array, matching what the projection
  writes for a path-valued keyword
…, not a list

The projection wrote every path-valued keyword and reference field as an
array whatever its `array` flag said, so one declaration meant a list to
the projection and a scalar to the engine collection definition and the
API output type. At root the engine rejected the document; inside a nested
document, which the engine does not type-check, it survived to fail per
query at the surface instead.

- honour `array` for keyword and reference as every other kind already
  does: a declared array stores a list, a single-valued field the first
  value the graph carries
- declare `array` on the fixtures whose sources are multi-valued
@ddeboer
ddeboer merged commit 9072199 into main Jul 31, 2026
4 checks passed
@ddeboer
ddeboer deleted the feat/nested-inline-references branch July 31, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Serve inline references as nested output

1 participant