Skip to content

brainlayer search cold-loads the embedding model for every CLI query #620

Description

@EtanHey

Problem

Each brainlayer search invocation cold-loads the sentence-transformers model before a single query. On the M1 incident host, a query loaded 391 tensors and completed in about 8.9 seconds.

That latency makes CLI search unsuitable as the fallback agent loop when MCP is unavailable.

Expected

Repeated CLI queries should reuse a warm embedding service/process or provide a fast text-first mode that does not load the model when semantic search is unavailable or unnecessary.

Acceptance

  • A documented warm/reusable path avoids per-invocation model construction.
  • Text-only fallback can return useful results without loading the embedding model.
  • Search output clearly reports whether semantic scoring was used.
  • Performance regression test or benchmark covers cold and warm paths.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions