Filter by IRI across all reference fields (cross-field "related to" filter)

[ddeboer]

Context

SearchQuery.where is a list of Filter clauses, each naming exactly one declared field: { field, in } for keyword/reference, { field, range } for the numeric/date kinds, { field, is } for boolean. The semantics are AND across fields, OR within a field, and the GraphQL where input mirrors that one-to-one – one input field per filterable field.

There is no way to express OR across fields.

Problem

The question “show me everything related to this entity” has no answer in the current model. An IRI can appear in several reference fields of the same type – creator, about, contentLocation, locationCreated, genre – and a consumer that wants every document referencing a person, place or term must issue one query per reference field and merge the results client-side, losing a correct total, correct ranking and correct facet counts in the process.

This is the natural query behind an entity landing page (“heritage objects related to Vincent van Gogh”), and it is exactly the case NDE’s valeros API draft reserves syntax for: alongside creator.id:<iri> it defines *.id:<iri> – “by ID for all properties”.

Engine-side this is cheap: Typesense filter_by supports || across fields, so a cross-field reference filter compiles to one disjunction in the same query. The work is in the model and the contract.

Proposal

Add a cross-field reference filter to the query IR and the surface. Open questions worth settling first, roughly in dependency order:

1. IR shape. A new Filter variant rather than an overload of { field, in } – the existing variants are discriminated by operator and validated per field kind, and field: '*' would be a stringly special case that every consumer has to remember to handle. Something closer to { fields: readonly string[], in: readonly string[] } keeps it typed and lets the surface decide the field set.

2. Which fields participate. Every filterable reference field is the obvious default. The alternative – an explicit declaration-level list, i.e. the deployment names the fields that make up “related to” – is more control but another declaration concept. Default first, unless DR or object search needs the narrower set.

3. Facet semantics – the subtle part. Skip-own-filter (ADR 0005, Cut the faceted page to a single engine round-trip (overlap + ride the listing search) #556) removes a facet’s own clause when computing its buckets. A cross-field filter has no single owning field: it touches creator, about, contentLocation at once. So when computing the creator facet, does the cross-field clause drop entirely, drop only its creator disjunct, or stay whole? Each gives different counts and each is defensible; this needs deciding before it ships, not after.

4. GraphQL surface. <Type>Where has one input field per filterable field, so a cross-field filter needs either a reserved input field (ThingWhere { …, anyReference: StringFilter }) or a separate root argument (relatedTo: [String!]). Reserved names collide with declared field names – worth checking the declaration validation rejects a field named whatever we reserve.

5. Port contract. assertValidQuery must validate the new variant (fields exist, are filterable, are references), so a second adapter (Add an OpenSearch engine adapter for large-scale object search #572) cannot quietly ignore it.

Acceptance

• One query answers “documents referencing this IRI in any reference field”, with a correct total, ranking and facet counts.
• The cross-field clause compiles to a single engine round-trip, not one per field.
• Facet semantics under a cross-field filter are decided, documented in the ADR, and covered by tests.
• assertValidQuery rejects a cross-field clause naming unknown, non-filterable or non-reference fields.
• The SDL snapshot pins whatever the surface shape turns out to be.

Notes

• Surfaced while comparing our surface with the valeros API draft (above), whose *.id: parameter is the same capability expressed in a REST idiom. That draft is otherwise a resource API asking questions this family has already answered; this and hit highlighting (Support hit highlighting across the search family #610) are the two places it is ahead of us.
• Pre-1.0 contract point: this adds to the frozen where input (ADR 0004), so the shape is cheaper to settle before v1 than after – though unlike Support hit highlighting across the search family #610 an additive input field is a softer break.

[ddeboer]

A consumer has now asked for exactly this capability: netwerk-digitaal-erfgoed/prototypes-data-layers is building an entity page where clicking a term shows every object referring to it through any property (limburg/lol#28, with the *:/terms/… wildcard from the REST prototype as the prior art).

Working through what that consumer could do today, without any change here, turned up an alternative worth weighing before the IR variant gets built.

The workaround, and what it reveals

A consumer can union the reference IRIs into one derived field in its own schema, using the existing derive mechanism:

{
  name: 'references',
  kind: 'keyword',
  array: true,
  filterable: true,
  derive: (document) =>
    distinct(RELATABLE_FIELDS.flatMap((fieldName) => stringValues(document[fieldName]))),
}

where: { references: { in: [iri] } } then answers the question with a correct total, correct ranking, correct facet counts and one engine round-trip – the whole acceptance list, minus the per-query field subset.

The part that matters for the design here is open question 3. A cross-field clause has no owning field, which is what makes skip-own-filter ambiguous: touching creator, about and contentLocation at once, it is undecidable whether the clause drops entirely, drops one disjunct, or stays whole. A derived union field has exactly one owning field, so the existing rule applies unchanged and the ambiguity never arises:

• computing the references facet → the clause is its own, so it drops;
• computing the material facet → the clause is foreign, so it stays whole, and the buckets answer “of the documents related to this IRI, how do they break down by material”.

That is the behaviour we would most likely have argued our way to anyway, but here it falls out of ADR 0005 rather than being a fourth thing to decide and document.

Proposal: a declared union-of-references field

Rather than { fields, in } in the IR, let a reference field declare that it is part of the type’s relatable set – a role flag alongside filterable/facetable/sortable/searchable – and generate the union field from that declaration. What this buys:

• No new Filter variant, so no port-contract change and nothing a second adapter (Add an OpenSearch engine adapter for large-scale object search #572) can quietly ignore – assertValidQuery keeps validating one field per clause.
• Facet semantics settled by construction (above), not decided and then defended.
• Domain knowledge stays in the declaration. The field set is never a name list in this repo: the default is every filterable reference field, and a deployment that wants it narrower flags the fields it means. Which properties constitute “related to” is a profile judgement, and it should not become one here.
• Compiles to one filter_by term rather than a disjunction, so it is cheaper engine-side than the || form, at the cost of storing each reference IRI a second time.

What it does not give, and what would keep this issue open:

• Per-query field subsets (“related through material or genre only”). Narrowing to a single field is already expressible by ANDing that field’s own clause; an arbitrary subset is not. Worth deferring until something asks for it.
• Labelled facets on the union. A union spans Term, Person and Place, so it cannot resolve labels until Let a reference resolve labels from several label sources #693 lands, and a keyword union has no labelSource at all. Faceting stays on the concrete reference fields, which is where it belongs.
• Which field matched, which neither shape surfaces – though a consumer whose reference fields are output can intersect them with the IRI client-side, so it may never need to.

Surface, either way

If the disjunction shape wins after all, two points from question 4 hold regardless. Name it structurally – anyReference names a field kind, where relatedTo is Schema.org-flavoured and would collide with a real declared property. And whatever the name, add it to the reserved-field-name check in validateSearchType, so a colliding declaration fails validation the way id does instead of relying on nobody choosing it.