Skip to content

feat(brain): runtime embedding config + incremental per-symbol embedding - #501

Merged
ajianaz merged 1 commit into
developfrom
feat/brain-runtime-config-499
Aug 5, 2026
Merged

feat(brain): runtime embedding config + incremental per-symbol embedding#501
ajianaz merged 1 commit into
developfrom
feat/brain-runtime-config-499

Conversation

@ajianaz

@ajianaz ajianaz commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

What

Add runtime embedding backend selection via .cora.yaml config and incremental per-symbol embedding to skip unchanged symbols during re-index.

Why

Previously, embedding backend was selected at compile time only (#[cfg(feature = "pretrained-embed")]). Users could not switch between hashing 256d and pretrained 768d without recompiling. Additionally, embed_project() re-embedded all symbols on every index run, even when only 1 file changed — wasting CPU on large projects.

Closes #499

How

BrainConfig (src/config/schema.rs):

  • New brain.embedding field in .cora.yaml with values: auto (default), hashing, pretrained
  • Invalid values fall back to auto with a warning
  • BrainEmbeddingMode enum with Display + FromStr impls

Runtime dispatch (src/embed/mod.rs):

  • New Backend enum (Hashing | Pretrained) with dims() and provider_name()
  • resolve_backend() reads config string, caches via OnceLock
  • embed_code_dispatch() checks ACTIVE_BACKEND at runtime, falls back to compile-time default if unset
  • Graceful fallback: brain.embedding=pretrained without feature flag → warning + hashing

Incremental embedding (src/index/brain.rs):

  • Migration v7 adds embed_fingerprint TEXT column to symbols table
  • embed_project() computes DefaultHasher fingerprint from name + signature
  • Only symbols whose fingerprint changed are re-embedded
  • Fingerprints updated after successful embedding
  • Vector index retains all symbol IDs for search (only embeddings are incremental)

Wiring (src/main.rs, src/commands/watch.rs):

  • cora index, cora brain, cora watch all call resolve_backend() before embedding

Testing

  • cargo test --features tree-sitter passes (881 + 16 + 6 = 903 tests, 0 failures)
  • cargo fmt --all -- --check passes
  • cargo clippy --all-targets --features tree-sitter -- -D warnings passes
  • cargo build --release --features tree-sitter passes (hashing-only mode)
  • cargo build --release --features tree-sitter,pretrained-embed passes (dual mode)
  • Manual smoke-test: cora index on uteke (1106 symbols, 147 files)
    • Full index: 1106 symbols embedded ✅
    • No-change re-index: 147 files skipped, 0 symbols re-embedded ✅
    • Touch 1 file: 146 skipped, 1 indexed, only 2 symbols re-embedded ✅
    • cora brain "memory recall" returns 8 results with correct FTS5 + vector signals ✅
    • Both hashing-only and pretrained builds tested ✅

Related Issues

Closes #499

Checklist

  • Branch name follows convention (feat/brain-runtime-config-499)
  • Branch is from develop
  • Commit messages follow Conventional Commits
  • No secrets or credentials committed
  • One logical change per PR (no mixed concerns)

- Add BrainConfig to .cora.yaml with brain.embedding mode (auto|hashing|pretrained)
- Refactor embed dispatch from compile-time to runtime via resolve_backend()
- Migration v7: add embed_fingerprint column to symbols table
- Incremental embedding: only re-embed symbols whose name+signature changed
- Wire resolve_backend() in cora index, cora brain, cora watch commands

Closes #499
@ajianaz ajianaz added the enhancement New feature or request label Aug 5, 2026
@ajianaz
ajianaz merged commit bcc7553 into develop Aug 5, 2026
13 checks passed
@ajianaz
ajianaz deleted the feat/brain-runtime-config-499 branch August 5, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(brain): runtime embedding config + incremental per-symbol embedding

1 participant