chore(db): remove deprecated entity schema from memory_links - #3177
Merged
Conversation
nicoloboschi
force-pushed
the
chore/drop-memory-links-entity-schema
branch
from
August 4, 2026 15:07
02cbfb0 to
cab9145
Compare
Entity edges are no longer materialized in memory_links. Retain stores memory-to-entity associations in unit_entities, and both read paths derive entity edges from that table on demand — the /graph endpoint from shared unit_entities rows and recall via the unit_entities self-join. Migration e9b2c7d1f3a4 deleted the stored entity rows and current writers only ever pass entity_id = NULL, leaving the entity-specific schema on memory_links as dead weight. New migration (PG + Oracle) drops the entity_id column and its FK, the entity index, 'entity' from the link_type CHECK, and the entity_id term in the function-based unique index (which collapses to (from_unit_id, to_unit_id, link_type)). It is written to avoid long locks on large tables: the residual delete is chunked with per-batch commits, indexes are swapped CONCURRENTLY, and the new CHECK is added NOT VALID then validated separately. Application code drops _NIL_ENTITY_UUID and the nil_entity_uuid DataAccessOps parameter, simplifies internal link tuples to four elements (from, to, link_type, weight), and removes the entity_id column/placeholder from the PG and Oracle bulk inserts and the chunk-storage lock ordering. The graph API keeps returning dynamically derived entity edges.
nicoloboschi
force-pushed
the
chore/drop-memory-links-entity-schema
branch
from
August 4, 2026 15:14
cab9145 to
437f6f1
Compare
This was referenced Aug 5, 2026
nicoloboschi
added a commit
that referenced
this pull request
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #3005.
Entity edges are no longer materialized in
memory_links. Retain stores memory-to-entity associations inunit_entities, and both read paths derive entity edges from that table on demand — the/graphendpoint from sharedunit_entitiesrows, and recall via theunit_entitiesself-join. Migratione9b2c7d1f3a4deleted the stored entity rows, and current writers only ever passentity_id = NULL, so the entity-specific schema onmemory_linkswas dead weight that made the table look like it still supported a second, materialized source of truth for entity edges.This removes that schema and the write-path plumbing behind it.
Schema changes (new migration, PG + Oracle)
link_type = 'entity'rows.entity_idcolumn and its FK toentities.idx_memory_links_entity/idx_ml_entity).link_typeCHECK without'entity'(temporal, semantic, and the legacy causal values remain valid).idx_memory_links_uniqueto(from_unit_id, to_unit_id, link_type)— non-entity rows already deduplicate on those three columns via the oldCOALESCE(entity_id, nil)key, so effective uniqueness is preserved.Production-safe by construction
memory_linksand its entity index can be very large on established banks, so the migration avoids long exclusive locks:CONCURRENTLY(PG) /ONLINE(Oracle), building the new unique index under a temporary name and renaming so duplicate protection is never dropped;NOT VALIDthenVALIDATEd separately, so writers are never blocked on a full-table scan.Downgrade restores the former schema shape but, as documented in the migration, cannot reconstruct the historical entity rows
e9b2c7d1f3a4already deleted.Application code
MemoryLink.entity_id/MemoryLink.entityandEntity.memory_links; update the docstring and CHECK metadata.(from, to, link_type, weight, entity_id)to(from, to, link_type, weight)across the temporal, semantic, causal, and graph-maintenance producers._NIL_ENTITY_UUIDand thenil_entity_uuidDataAccessOpsparameter; drop theentity_idcolumn/placeholder from the PG and Oracle bulk inserts (conflict target is now(from_unit_id, to_unit_id, link_type)).entity_idtiebreaker from the chunk-storage lock ordering and a stale runtime comment.unit_entities— unchanged.Tests
test_migration_drop_memory_links_entity.py(dedicated pg0 instance): asserts the column/index/entityCHECK value are gone at head, the unique index is three columns, and the downgrade → re-upgrade round-trip restores and re-drops the shape.test_link_utils.pyandtest_document_transfer.pyfor four-element tuples / noentity_idcolumn.Non-goals
Unchanged: the
unit_entitiesmodel,link_type = 'entity'in graph/API response types, entity resolution / co-occurrence / orphan cleanup, and temporal/semantic/causal link generation.