Skip to content

Consider caching prepared statements in build-edges.ts::applyEdgeTechniquesAfterNativeInsert chunk loop #1768

Description

@carlos-alm

Context

Titan grind phase 19 deduped the getOrCreateBatchStmt/runBatchInsert chunk-size statement-cache pattern used by batchInsertNodes/batchInsertEdges/markExportedSymbols in src/domain/graph/builder/helpers.ts. The mandatory codebase-wide duplicate scan (grind Step 2c) also checked for other prepared-statement-caching opportunities in db/ and builder/ code.

Finding

src/domain/graph/builder/stages/build-edges.ts::applyEdgeTechniquesAfterNativeInsert (around line 1832) chunks sourceIds into 500-sized groups to stay within SQLITE_LIMIT_VARIABLE_NUMBER, but re-prepare()s two SQL statements from scratch on every chunk iteration instead of caching by chunk size:

for (let i = 0; i < sourceIds.length; i += CHUNK_SIZE) {
  const chunk = sourceIds.slice(i, i + CHUNK_SIZE);
  const placeholders = chunk.map(() => '?').join(',');
  db.prepare(`UPDATE edges SET technique = 'ts-native' WHERE ... source_id IN (${placeholders})`).run(...chunk);
  db.prepare(`UPDATE edges SET confidence = ? WHERE ... source_id IN (${placeholders})`).run(...);
}

This is a different shape from the just-deduped pattern (two SQL templates per chunk, not one; chunked by an IN (...) id list rather than a multi-value INSERT/UPDATE ... OR list), and it lives in a hot, transactional edge-finalization path, so we did not touch it in the phase-19 grind PR to keep that change small, safe, and single-concern.

Suggested follow-up

If profiling shows applyEdgeTechniquesAfterNativeInsert chunking is hot on large incremental builds (many distinct sourceIds batches), consider caching the two prepared statements by chunk size the same way getOrCreateBatchStmt does, using two per-db WeakMap<BetterSqlite3Database, Map<number, SqliteStatement>> caches (or reusing getOrCreateBatchStmt directly, since its signature already accepts an arbitrary buildSql callback).

Not urgent — no evidence this is currently a bottleneck; filed for future evaluation only.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions