Memoize sourced_from update target resolution on Results - #1342
Merged
ellisandrews-toast merged 1 commit intoAug 13, 2026
Conversation
ellisandrews-toast
requested review from
BrianSigafoos-SQ,
bsorbo,
jwils,
jwondrusch,
marcdaniels-toast,
myronmarston and
rossroberts-toast
as code owners
August 12, 2026 17:08
myronmarston
approved these changes
Aug 13, 2026
ellisandrews-toast
deleted the
OLAP-741-memoize-sourced-from-update-targets
branch
August 13, 2026 13:54
ellisandrews-toast
added a commit
that referenced
this pull request
Aug 13, 2026
…es (#1343) ## Summary Second step toward #1273 (stacked on #1342). Allows an `indexing_only: true` relationship to reference a non-indexed object type, so that a `sourced_from` source type will no longer be required to have its own index. Relationships serve two distinct jobs today: 1. GraphQL navigation (query time): the generated relationship field resolves by searching the related type's index. This genuinely requires the related type to be indexed. 2. `sourced_from` routing (indexing time): the relationship's foreign key metadata routes source events to the destination type's documents. The source type's own index is never touched. The previous validation ("Only root document types can be used in relations") applied the job-1 requirement to both jobs. This PR scopes it to job 1: GraphQL-exposed relationships still require an indexed related type (with an updated error message that explains why), while `indexing_only: true` relationships may now reference a non-indexed type. ## New id validation Allowing non-indexed related types opens a validation gap. "Indexed types must have an `id` field" is enforced as a side effect of `t.index` — so until now, source types (being necessarily indexed) always had `id` guaranteed transitively. A non-indexed source type skips that check entirely, but relationships join on `id` non-negotiably: the source event's `id` drives per-source-record version tracking on destination documents, and nested `sourced_from` matches list elements by `id`. Without a check, a missing `id` would surface as confusing ingestion-time failures far from the actual mistake. This PR adds the corresponding schema-definition-time validation in `RelationshipResolver`: a non-indexed related type must define an `id` field, reported with the standard relationship error context. Indexed related types are unaffected (they're already validated via `t.index`). ## Note on scope This PR makes such schemas _definable_, but events for a non-indexed source type are still rejected at ingestion: the event envelope's `type` enum in `json_schemas.yaml` only includes indexed types. Widening that to "ingestable types" (indexed types + `sourced_from` source types) is the next PR in the stack.
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pure refactor, no behavior change: extracts the inline
Indexing::SourcedFromUpdateTargetsResolver.new(state).resolvecall inResults#build_runtime_metadatainto a memoizedResults#sourced_update_targets_by_source_type_namemethod, following the existingderived_indexing_type_namespattern.Motivation
First step toward #1273. Supporting unindexed
sourced_fromsource types requires the JSON schema event envelope (generated inelasticgraph-json_ingestion) to include source type names in itstypeenum — and that set is exactly the keys of this resolver's result. Memoizing it onResultslets a follow-up PR consume it fromResultsExtension(the same wayderived_indexing_type_namesis consumed today) without running a second resolver pass.Notes
build_runtime_metadatastill invokes the resolver eagerly before materializingall_types— the ordering matters so thatsourced_fromvalidation errors take precedence over errors raised during derived-type generation. A comment now documents this previously-implicit constraint; two existing specs guard it.