Skip to content

fix(storage): drop write-only connects_to and index synapse source_id/target_id - #25

Merged
acidkill merged 1 commit into
acidkill:mainfrom
RobertSigmundsson:fix/synapse-document-model
Jun 22, 2026
Merged

fix(storage): drop write-only connects_to and index synapse source_id/target_id#25
acidkill merged 1 commit into
acidkill:mainfrom
RobertSigmundsson:fix/synapse-document-model

Conversation

@RobertSigmundsson

Copy link
Copy Markdown
Contributor

Summary

This implements option A from #15 (tidy the document model). Three related, backward-compatible changes to the synapse area:

  1. Drop the connects_to edge table. add_synapse issued RELATE neuron:$src -> connects_to -> neuron:$tgt (in a try/except) and delete_neuron cleaned it up, but nothing in src/ ever traverses ->connects_to-> (grep returns zero). It was write-only. In fact it never even wrote: the neuron:$src form (a parameter in the record-id position of a record literal) does not parse, the except swallowed the error, and our live brain has 0 connects_to rows after thousands of synapse inserts. So this removes a per-insert round-trip that did nothing.

  2. Declare source_id / target_id on synapse. These are the fields add_synapse writes and get_synapses filters on, but they were not in the SCHEMAFULL definition. Declaring them makes the schema match what is persisted (1609 synapses in our brain carry them).

  3. Point the source/target indexes at the populated columns. idx_synapse_source / idx_synapse_target were defined on out / in, which only a RELATION table fills. synapse is a normal table, so those columns are always empty and the indexes covered nothing: every WHERE source_id = ... / target_id = ... lookup full-scanned. They now index source_id / target_id.

On the SCHEMAFULL parser (deliberately out of scope)

While here we noticed ensure_schema splits SCHEMA_SQL on ; and drops any fragment starting with --. Since every DEFINE TABLE ... SCHEMAFULL is preceded by a comment, those table-level statements are dropped and tables run SCHEMALESS (confirmed: INFO FOR TABLE synapse shows no source_id/target_id in the field registry, yet they persist). That is the other half of the "two bugs cancelling out" from #15, and it is why the undeclared fields worked. It affects every table, not just synapse, so we did not touch the parser here: fixing it would activate SCHEMAFULL across ~25 tables at once. Declaring source_id/target_id makes synapse correct either way. Happy to raise the parser as its own thread.

Changes

  • storage/surrealdb/schema.py: declare source_id/target_id on synapse; repoint idx_synapse_source/idx_synapse_target; remove the connects_to table.
  • storage/surrealdb/store.py: remove the connects_to RELATE in add_synapse and its cleanup in delete_neuron; fix the class docstring.
  • tests/unit/test_synapse_document_model.py: regression test.

Type of Change

  • Bug fix / cleanup (non-breaking)

Testing

  • ruff, mypy, unit suite green.
  • Verified against our live brain: connects_to = 0 rows; 1609 synapses carry source_id/target_id; the old indexes were on the empty out/in columns.

Note for existing deployments

ensure_schema swallows "index already exists", so on an existing DB the two indexes keep their old out/in definition until rebuilt (REMOVE INDEX + re-run, or REBUILD INDEX). Fresh installs get the corrected indexes directly. Say the word if you would prefer the schema use OVERWRITE for these two.

CHANGELOG (### Fixed)

  • Drop the write-only connects_to table (the RELATE never parsed and nothing traversed it) and persist synapse edges via the source_id/target_id document fields, now declared in the schema and covered by the source/target indexes (previously on the never-populated out/in, so lookups full-scanned).

Context: #15

…/target_id

Synapse edges are stored in the document model (source_id/target_id columns),
not as graph edges. This makes that consistent (option A from acidkill#15):

- Drop the connects_to table. add_synapse issued a RELATE into it and
  delete_neuron cleaned it up, but nothing traverses ->connects_to-> anywhere
  in src/. It was write-only, and it never even wrote: the
  "RELATE neuron:$src -> connects_to -> ..." form does not parse (a parameter
  in the record-id position) and the call sits in a try/except, so on a live
  brain connects_to has 0 rows. Removes a dead per-insert round-trip.
- Declare source_id/target_id on synapse. They are the fields add_synapse
  writes and get_synapses filters on, previously absent from the schema.
- Repoint idx_synapse_source/idx_synapse_target from out/in (only a RELATION
  table populates those, so they covered empty columns and lookups
  full-scanned) to source_id/target_id.

Adds tests/unit/test_synapse_document_model.py. The ensure_schema parser that
drops comment-prefixed DEFINE TABLE statements (so tables run SCHEMALESS) is
left untouched on purpose: it affects every table, not just synapse, and
flipping it would activate SCHEMAFULL across ~25 tables at once.

Co-Authored-By: Captain Nemo <noreply@anthropic.com>
@acidkill
acidkill merged commit ed98a65 into acidkill:main Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants