Summary
If a process exits after beginBulkParseLoad() drops parse-lane secondary
indexes but before endBulkParseLoad() restores them, opening the project
does not recreate those indexes.
DatabaseConnection.open() calls healBulkNodeLoad(), but that recovery
path checks/rebuilds FTS triggers only. It does not restore the indexes named by
BULK_PARSE_INDEX_NAMES / BULK_EDGE_INDEX_NAMES, despite the bulk-load
comment promising that a crash in the window is healed on the next open.
Reproduction
- Start a fresh index large enough to remain in bulk parse for a while.
- Stop the process after
beginBulkParseLoad() has dropped indexes.
- Reopen the project or run the documented recovery
sync.
- Inspect
sqlite_master.
After reopening, the database had only:
sqlite_autoindex_nodes_1
sqlite_autoindex_files_1
idx_edges_identity
Expected node lookup indexes and idx_unresolved_status were absent.
Impact measured on a large C/C++ graph
- 6.57 million nodes and 24.24 million pending references.
SELECT DISTINCT name FROM nodes became a full node scan plus temporary
B-tree.
- Resolver cache warm-up ran for more than 10 minutes and reached 5.5 GiB RSS
before being stopped.
- Recreating only
idx_nodes_name reduced that phase to about 27 seconds.
- Missing unresolved-reference indexes added separate 13-second and 11-second
scans even when a scoped lookup returned no rows.
The recovered sync can also finish without changing
project_metadata.index_state from indexing to complete, leaving a
healthy graph permanently reported as truncated.
Suggested direction
Persist an explicit bulk-load marker, or make open-time healing verify and
recreate every index dropped by both bulk-load paths. A successful recovery
sync should also validate counts/integrity and own the metadata transition back
to complete.
Summary
If a process exits after
beginBulkParseLoad()drops parse-lane secondaryindexes but before
endBulkParseLoad()restores them, opening the projectdoes not recreate those indexes.
DatabaseConnection.open()callshealBulkNodeLoad(), but that recoverypath checks/rebuilds FTS triggers only. It does not restore the indexes named by
BULK_PARSE_INDEX_NAMES/BULK_EDGE_INDEX_NAMES, despite the bulk-loadcomment promising that a crash in the window is healed on the next open.
Reproduction
beginBulkParseLoad()has dropped indexes.sync.sqlite_master.After reopening, the database had only:
Expected node lookup indexes and
idx_unresolved_statuswere absent.Impact measured on a large C/C++ graph
SELECT DISTINCT name FROM nodesbecame a full node scan plus temporaryB-tree.
before being stopped.
idx_nodes_namereduced that phase to about 27 seconds.scans even when a scoped lookup returned no rows.
The recovered sync can also finish without changing
project_metadata.index_statefromindexingtocomplete, leaving ahealthy graph permanently reported as truncated.
Suggested direction
Persist an explicit bulk-load marker, or make open-time healing verify and
recreate every index dropped by both bulk-load paths. A successful recovery
sync should also validate counts/integrity and own the metadata transition back
to
complete.