fix(recall): inherit observation entities through source_memory_ids#1397
Merged
nicoloboschi merged 2 commits intovectorize-io:mainfrom May 5, 2026
Merged
Conversation
`include_entities=True` returns `entities: null` for every observation in the recall response, even when those observations are linked through `source_memory_ids` to facts whose entities are populated. The per-memory endpoint (`get_memory_unit`) already handles this case: if an observation has no rows in `unit_entities`, it inherits the union of entities from its source memories. The recall path queried `unit_entities` directly and stopped there, so observation results lost both their per-result `entities` field and their contribution to the top-level aggregate map. The asymmetry made observation-only recall hostile to clients that needed entity context (URL recovery, entity-aware ranking). The documented workaround was to add `world` and `experience` to the `types` filter and rely on those facts to carry the entity payload. Mirror `get_memory_unit`'s fallback inside the recall entity-fetching block: for observation result IDs that produced no direct `unit_entities` rows, look up their `source_memory_ids`, fetch entities for the union of source IDs in a single batched query, and project the results back onto the original observation IDs (deduped by entity_id, preserving source-memory order). The downstream code that derives per-result `entities` and the top-level aggregate map both consume `fact_entity_map`, so the inheritance flows through both paths automatically. Add a regression test that seeds an observation linked via `source_memory_ids` to a fact carrying two entities, plus a second observation with its own direct `unit_entities` link, then asserts recall projects both per-result entity lists and the top-level map.
…QL helper
The first commit on this branch fixed the recall projection by mirroring
get_memory_unit's procedural fallback in Python: query unit_entities,
detect observations that came back empty, separately fetch
source_memory_ids, separately fetch entities for the union of source
IDs, then dedupe and merge in Python. That worked but had two issues
worth fixing before the PR lands.
First, the inheritance edge ("observation linked through its source
memories") is dialect-shaped: PG stores it on `memory_units.source_memory_ids`,
Oracle keeps it in the `observation_sources` junction table. The
procedural patch reached for `source_memory_ids` directly, which made
recall observation-entity inheritance silently PG-only.
Second, the same fallback already existed inline in get_memory_unit, so
shipping a second copy in recall left two places that had to stay in
sync forever, by hand.
Introduce `_entity_rows_for_units_sql`, a private engine helper that
returns a single dialect-correct UNION SELECT producing
`(unit_id, entity_id, canonical_name)` rows. Direct rows come from
`unit_entities`; observations that have no direct row inherit through
`source_memory_ids` (PG) or `observation_sources` (Oracle), guarded by
NOT EXISTS so the inheritance only fires when the direct path is empty.
This is the same conceptual shape as `_observations_via_source_match_sql`
on the document view fix branch — both are SQL primitives over the
observation-source edge.
Use the helper in two places that previously hand-rolled the same
inheritance logic:
- The recall entity-fetch block collapses from three queries plus a
Python dedupe loop to one fetch into the same `fact_entity_map`.
- get_memory_unit's two-query "fetch direct, fall back to sources"
pattern collapses to one fetch, with identical observable behavior.
Add a get_memory_unit assertion to the existing regression test so the
shared helper is exercised through both call sites and any future drift
between recall and the per-memory endpoint trips a test, not a
production report.
nicoloboschi
approved these changes
May 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Recall returns
entities: nullfor every observation result, even wheninclude_entities=trueis set, whileGET /v1/default/banks/{bank_id}/memories/{id}returns the same observation with its inherited entities populated. The asymmetry strips entity context (e.g. URLs, named projects) from observation-only recall and silently drops those entities from the top-levelentitiesaggregate map.Related: #1374 fixes the same conceptual gap on the document detail view (per-document graph + Observations tab were empty because the observation→source edge wasn't traversed). This PR is the recall-projection sibling: observation results lose their entities for the same underlying reason — code paths that need to traverse the observation→source edge weren't doing it.
Root cause
get_memory_unit(hindsight_api/engine/memory_engine.py) already handles observations explicitly: if a unit has no rows inunit_entities, it inherits entities from its source memories. The recall path's entity fetch only queriedunit_entitiesdirectly; observations have no direct rows there, so every observation in the recall result lost its entities.The inheritance edge is dialect-shaped: PG stores it on
memory_units.source_memory_ids, Oracle keeps it in theobservation_sourcesjunction (same setup #1374 navigates with_observations_via_source_match_sql).Fix
Two commits:
fix(recall): inherit observation entities through source_memory_ids— mirrorget_memory_unit's fallback inside recall'sinclude_entitiesblock: for observation result IDs with no directunit_entitiesrows, look upsource_memory_ids, fetch entities for the union of source IDs, project back. Downstream code derives the per-resultentitiesfield and the top-level aggregate map from the same map, so inheritance flows through both automatically.refactor(recall): consolidate observation entity inheritance in one SQL helper— replace the procedural fallback with_entity_rows_for_units_sql, a private engine helper that returns one dialect-correct UNION SELECT covering both direct and inherited entity rows. Mirrors the shape of_observations_via_source_match_sqlfrom fix(api): include observations in per-document graph and counts #1374. Also fixes silent Oracle-parity gap (the procedural patch reached forsource_memory_idsdirectly, which is PG-only).get_memory_unitis refactored to use the same helper, so the two call sites that hand-rolled the same inheritance now share one source of truth.Reproduction (before fix)
A mixed-type recall (
["observation","world","experience"]) populates the top-level entities map but only via world/experience contributions; observation results still come back withentities: null.Test plan
tests/test_recall_observation_entities.py) seeds:worldfact with two entities linked viaunit_entities,source_memory_idsand zero direct entity links,unit_entitieslink,and asserts that:
recall_asyncwith the expectedentities(inherited and direct),entitiesaggregate map is populated with all of them,get_memory_uniton the same observations also surfaces inherited and direct entities (guards against drift between the two call sites that share the helper).mainwithAssertionError: Observation entities must inherit through source_memory_ids; got []and passes after the fix.tests/test_observations.pyandtests/test_graph_filtering.pycontinue to pass.ruff checkandruff formatclean.